AI crawlers / AI crawlers
ExaUser: what it does and what blocking it changes
What ExaUser is
ExaUser is the agent Exa uses to fetch a page live when a user, or an application built on the Exa API, asks for its current content. Exa says it is not used to crawl the web automatically; that is ExaSearchBot. It does not try to bypass logins, paywalls or CAPTCHAs, and submits no forms.
| Fact | ExaUser |
|---|---|
| Operator | Exa |
| Job | User-triggered fetch |
| robots.txt token | ExaUser |
| Follows robots.txt | Partial, vendor says robots.txt rules may not apply to user-initiated fetches |
| IP addresses | Verify tool |
| Vendor page read | 24 October 2026 |
What the vendor says
In the vendor's words (English original): “Exa uses ExaUser to fetch a page live when a user, or an application built on the Exa API, asks for its current content.”
Source: Exa: ExaUser, read on 24 October 2026.
What happens if you block ExaUser
Exa says that because fetches are initiated by a user, robots.txt rules may not apply to ExaUser. It also says ExaUser is not used to decide whether content appears in Exa search, and that search opt-outs and automatic crawling are managed with the ExaSearchBot token in robots.txt.
The robots.txt lines
To allow ExaUser:
User-agent: ExaUser
Allow: /To block ExaUser:
User-agent: ExaUser
Disallow: /Paste a block into the robots.txt at the root of your host, for example https://example.com/robots.txt. A group for one token does not cover the operator's other crawlers.
The user-agent string the vendor shows:
Mozilla/5.0 (compatible; ExaUser/1.0; +https://crawler.exa.ai/exauser)How to tell a real request from a fake one
The vendor says every request is signed with Web Bot Auth (RFC 9421) and publishes its keys at a public directory; it does not publish an IP list.
Compare with ExaSearchBot
ExaSearchBot: ExaSearchBot crawls automatically to build Exa's search index and follows robots.txt, while ExaUser fetches a page only when a user or application asks and the vendor says robots.txt may not apply.
Your own site
The free check in the box above tests eight crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended, Bingbot and Googlebot). ExaUser is not one of them: read your robots.txt for a ExaUser group and look for the user agent in your server log. The full list of crawlers is on the AI crawler hub.
Questions people ask
Does ExaUser follow robots.txt?
Not always. Partial, vendor says robots.txt rules may not apply to user-initiated fetches. A rule for ExaUser in robots.txt may therefore not stop it. To refuse it for certain, use a firewall or CDN rule that matches both the user agent and the published addresses.
How do I know ExaUser is really blocked?
robots.txt is only one layer: a CDN or firewall rule can also turn a crawler away, and a crawler can ignore the file. Check your server log for the user agent, and test the address from outside your network.
Sources
See what AI answers about your shop
The audit crawls your site up to your plan's page limit, shows which AI crawlers it lets in and gives the exact fix for each issue. From €9, 14-day money-back guarantee.
Audit my siteAudit from €9, paid once. 14-day money-back guarantee.