Checks / AI crawler access
No llms.txt (optional)
llms.txt is a short map of your site written for AI assistants. No major assistant is confirmed to read it, so treat this as optional.
llms_txt_missing · AI crawler access · Notice
What to change
Optional: publish /llms.txt. A draft built from your pages is in the fix box.
Example
Contents of /llms.txt:
# Example Shop
> One sentence on what the site offers and who it is for.
## Main pages
- [Products](https://example.com/products): the range
- [Shipping and returns](https://example.com/shipping): costs, delays and returnsHow common it is
On a sample, the study measured sites with no valid llms.txt: 192 of 292 Show HN sites (66%), and 53 of 65 Lovable sites (82%).
Fetched 20 and 21 Oct 2026 as CitedBot, robots.txt obeyed, no JavaScript. Study: Launch crawl 2026-10 (English).
Counted with the check's own rule, leaving out sites that refused the request (HTTP 403): CSV of the counts.
About this check
| Check ID | llms_txt_missing |
|---|---|
| Report section | AI crawler access |
| Severity | Notice |
| How it is decided | Measured: read directly from the page or the server's response. |
| Added | Before 19 Oct 2026 (versions 1 and 2), or a site-wide check with no version date yet |
| Status | Measured in every report |
| Report gives the code | Yes, one block for the site |
Related checks
- No llms-full.txt (optional)
- llms.txt points at pages that crawlers cannot use
- Site is not served over HTTPS
- Pages return an error status
- robots.txt blocks ChatGPT-User
- robots.txt blocks Claude-SearchBot
See whether it applies to your site
Paste your address and we crawl up to 10 pages and show every issue we find, with its fix. No account, no card.
Every plan crawls up to its page limit and lists these checks with the fix: plans.
How an audit works: method