Checks / AI crawler access
robots.txt blocks PerplexityBot
PerplexityBot builds Perplexity's index. Blocked, you are not cited there.
robots_perplexitybot · AI crawler access · Error
What to change
Remove the Disallow rule for PerplexityBot in robots.txt.
Example
Lines that allow PerplexityBot in robots.txt:
User-agent: PerplexityBot
Allow: /How common it is
On a sample, the study measured robots.txt blocking PerplexityBot on the homepage's site: 0 of 294 Show HN sites (0%), and 0 of 67 Lovable sites (0%).
Fetched 20 and 21 Oct 2026 as CitedBot, robots.txt obeyed, no JavaScript. Study: Launch crawl 2026-10 (English).
About this check
| Check ID | robots_perplexitybot |
|---|---|
| Report section | AI crawler access |
| Severity | Error |
| How it is decided | Measured: read directly from the page or the server's response. |
| Added | Before 19 Oct 2026 (versions 1 and 2), or a site-wide check with no version date yet |
| Status | Measured in every report |
| Report gives the code | Yes, one block for the site |
Settings by platform
For robots.txt, where the setting lives differs by platform. Our guides cover 13 of them: AI crawler settings by platform.
Related checks
- robots.txt blocks ChatGPT-User
- robots.txt blocks Claude-SearchBot
- robots.txt blocks Claude-User
- robots.txt blocks OAI-SearchBot
- robots.txt blocks Perplexity-User
- robots.txt blocks CCBot
See whether it applies to your site
Paste your address and we crawl up to 10 pages and show every issue we find, with its fix. No account, no card.
Every plan crawls up to its page limit and lists these checks with the fix: plans.
How an audit works: method