Our crawler
UptenAIBot
Effective date: 7 October 2026
Operated by UptenAI, Inc., a Delaware corporation. Contact: ops@uptenai.com.
UptenAIBot is the crawler UptenAI uses to read an online store the way search engines and AI assistants do, so we can score it and suggest fixes. If you found it in your logs, this page explains what it does and how to stop it.
How it identifies itself
Every request it makes carries this user agent:
UptenAIBot/1.0 (+https://uptenai.com/bot)What it reads, and when
- A site is crawled only when its owner asks for a report, or when our team prepares a free report for a brand.
- At most 100 pages per audit: the homepage, a sample of collections and products, help, shipping, returns and policy pages, the sitemaps and robots.txt.
- One request per second to your site, with a 10 second timeout and a size limit on each page.
- It reads public pages only. It never signs in, never adds to a cart, never fills in forms and never runs your site’s scripts.
- Page copies are kept for 30 days to check fixes against, then deleted. What we extracted (titles, headings, structured data) stays with the report.
It obeys robots.txt
UptenAIBot follows the rules in your robots.txt for its own name, and the rules for every crawler when there is no group for it. Pages you disallow are not fetched.
How to stop it
Add this to your robots.txt and it will not read any page of your site:
User-agent: UptenAIBot
Disallow: /Or email ops@uptenai.com with your domain and we will stop crawling it.
The AI crawler access test
Part of a report checks whether AI crawlers can reach a store. For that test only, we fetch the homepage, one collection and one product with each AI crawler’s published user agent string, and record whether the full page came back. These requests come from our servers, not from the AI companies, and we label the result as a user agent test. It uses three pages per crawler and nothing else.