AI crawler access for otto.de
All 8 crawlers can read otto.de: robots.txt allows them and the server answered each test request.
| Crawler | Used for | robots.txt | Live request |
|---|---|---|---|
| GPTBot | Model training | Allowed · 5 path rules restrict this crawler | Answered 200 |
| OAI-SearchBot | Search and citations | Allowed · 5 path rules restrict this crawler | Answered 200 |
| ChatGPT-User | Page fetched for a person | Allowed · 5 path rules restrict this crawler | Answered 200 |
| PerplexityBot | Search and citations | Allowed · 5 path rules restrict this crawler | Answered 200 |
| ClaudeBot | Model training | Allowed · 4 path rules restrict this crawler | Answered 200 |
| Google-Extended | Model training | Allowed · 4 path rules restrict this crawler | Not requested (robots.txt switch only) |
| Bingbot | Search and citations | Allowed · 5 path rules restrict this crawler | Answered 200 |
| Googlebot | Search and citations | Allowed · 4 path rules restrict this crawler | Answered 200 |
The exact fix
Nothing to fix for crawler access.
Next: the rest of the site
This tool reads one page and two files. The free check crawls a sample of pages and lists everything that keeps search engines and assistants from reading otto.de, with the exact fix for each.
Who is who
ChatGPT, Perplexity, Gemini, Copilot and Claude answer buyers' questions with different crawlers. Search crawlers (OAI-SearchBot, PerplexityBot, Bingbot, Googlebot) decide whether you can be cited. ChatGPT-User fetches a page when a person asks about it. Training crawlers (GPTBot, ClaudeBot, Google-Extended) feed models, and blocking them is a choice that does not remove you from answers.
A note for EU sites
In the EU, a machine-readable reservation such as a robots.txt rule is how a rightsholder opts out of commercial text-and-data mining (Article 4 of Directive (EU) 2019/790). Blocking a training crawler is therefore a legitimate, documented choice. This is information, not legal advice.