AI crawlers / AI crawlers

MistralAI-User, MistralAI-Index and MistralAI-Training explained

By the CitedBy editorial team · Reviewed on 2026-10-21

What MistralAI-User is

Mistral's crawler page lists three agents: MistralAI-User for user actions in Vibe, MistralAI-Index for indexing content for Mistral search, and MistralAI-Training, which crawls web content to help build datasets for training Mistral generative AI models. Webmasters can disallow each in robots.txt.

What the vendor says

In the vendor's words: “MistralAI-User is for user actions in Vibe. When users ask Vibe a question, it may visit a web page to help answer and include a link to the source in its response.”

Source: Mistral Docs: Mistral crawlers, read on 21 October 2026.

What happens if you block it

Disallowing MistralAI-User stops that agent opening your page for a Vibe user's question; it leaves the index and training agents alone, which have their own tokens. Mistral says the user agent is not used for crawling the web in any automatic fashion, nor to crawl content for generative AI training, and that content crawled by the index agent is not used for generative AI training of any kind.

The robots.txt lines

To allow it:

User-agent: MistralAI-User

Allow: /

To block it:

User-agent: MistralAI-User
Disallow: /

User-agent: MistralAI-Training
Disallow: /

The user-agent string the vendor shows:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-User/1.0; +https://docs.mistral.ai/robots)

How to tell a real request from a fake one

Mistral publishes separate address files for the agents. mistral.ai/mistralai-user-ips.json held 4 prefixes (creation time 2025-02-19) and the index file 2 prefixes (creation time 2026-04-19) on 21 October 2026. Two entries for an index crawler is a list a firewall rule can hold exactly.

Compare with ChatGPT-User

ChatGPT-User: The same job at OpenAI, with a different robots.txt promise.

Check your own site

The audit box above reads robots.txt for the crawlers in this set and sends one live request per crawler, so a rule that robots.txt allows but a firewall refuses shows up. The full list of crawlers is on the AI crawler hub.

Questions people ask

Which Mistral agent trains models?

Mistral's page says MistralAI-Training crawls web content to help build datasets for training its generative AI models. MistralAI-User and MistralAI-Index are described as not used for training.

Is Vibe a crawler?

Vibe is Mistral's product. MistralAI-User is the agent that may visit a page on behalf of a Vibe user.

Sources

• Mistral Docs: Mistral crawlers
• mistral.ai/mistralai-user-ips.json, read 21 October 2026