AI crawlers / AI crawlers
GoogleOther: what it does and what blocking it changes
What GoogleOther is
GoogleOther is Google's generic crawler, used by various Google product teams to fetch publicly accessible content, for example in one-off crawls for internal research and development. Google lists it among its common crawlers, next to Googlebot, and also documents GoogleOther-Image and GoogleOther-Video as versions optimized for image and video URLs. The Google page does not say it is used for AI training.
| Fact | GoogleOther |
|---|---|
| Operator | |
| Job | Generic crawl for various Google product teams |
| robots.txt token | GoogleOther |
| Follows robots.txt | Yes. Google says common crawlers always obey robots.txt rules when crawling automatically. |
| IP addresses | 317 ranges (developers.google.com/static/crawling/ipranges/common-crawlers.json, file dated 2026-10-23) |
| Vendor page read | 24 October 2026 |
What the vendor says
In the vendor's words (English original): “GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.”
Source: Google Search Central: Google's common crawlers, read on 24 October 2026.
What happens if you block GoogleOther
Google says crawling preferences addressed to the GoogleOther user agent do not affect any specific product. Google documents GoogleOther-Image and GoogleOther-Video as separate robots.txt tokens, so a rule for GoogleOther is not stated to cover them, and the Googlebot token is a separate token again.
The robots.txt lines
To allow GoogleOther:
User-agent: GoogleOther
Allow: /To block GoogleOther:
User-agent: GoogleOther
Disallow: /Paste a block into the robots.txt at the root of your host, for example https://example.com/robots.txt. A group for one token does not cover the operator's other crawlers.
The user-agent string the vendor shows:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36How to tell a real request from a fake one
The file at developers.google.com/static/crawling/ipranges/common-crawlers.json held 317 ranges on 24 October 2026, file dated 2026-10-23.
Google says the common crawlers generally crawl from the ranges in common-crawlers.json, and that the reverse DNS name matches crawl-*-*-*-*.googlebot.com or geo-crawl-*-*-*-*.geo.googlebot.com.
Compare with Googlebot
Googlebot: Googlebot crawls for Google Search and related products, while GoogleOther is a generic crawler whose robots.txt rules do not affect any specific Google product.
Your own site
The free check in the box above tests eight crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended, Bingbot and Googlebot). GoogleOther is not one of them: read your robots.txt for a GoogleOther group and look for the user agent in your server log. The full list of crawlers is on the AI crawler hub.
Questions people ask
Does GoogleOther follow robots.txt?
Yes. Google says common crawlers always obey robots.txt rules when crawling automatically. Give GoogleOther its own User-agent group in robots.txt: the operator's other crawlers have their own settings.
How do I know GoogleOther is really blocked?
robots.txt is only one layer: a CDN or firewall rule can also turn a crawler away, and a crawler can ignore the file. Check your server log for the user agent, and test the address from outside your network.
Sources
- Google Search Central: Google's common crawlers
- Google: common-crawlers.json (read 2026-10-24, creationTime 2026-10-23)
- Google Search Central: Overview of Google crawlers and fetchers
See what AI answers about your shop
The audit crawls your site up to your plan's page limit, shows which AI crawlers it lets in and gives the exact fix for each issue. From €9, 14-day money-back guarantee.
Audit my siteAudit from €9, paid once. 14-day money-back guarantee.