How a CitedBy audit works

Enter a URL. CitedBy crawls the site, checks every page it finds and gives you a graded report with the exact fix for each issue. This page says what is checked, how the score is built and what we do not claim.

1. The crawl

We start from your URL, read robots.txt and the sitemap, and follow internal links to find pages and sub-pages. Each plan has a page cap; the single audit covers up to 500 pages. We identify ourselves with a CitedBy user agent and stay on your domain. Every request is made from the public internet, logged out, like any crawler would.

2. What is checked on every page

• Crawler access. robots.txt rules for GPTBot, ClaudeBot and PerplexityBot, the status code each page returns when fetched with those user agents, and signs of a firewall or CDN block.
• Readable HTML. The text a crawler receives without running JavaScript, compared with the size of the page. Pages that arrive empty are flagged.
• llms.txt and sitemap. Whether the files exist and parse. No major assistant is confirmed to read llms.txt, so it is marked optional and never drags the score down on its own.
• Structured data. JSON-LD for Organization, Product, FAQ and Article, checked for validity and for fields a buyer would ask about, such as price.
• Titles, headings and links. Missing or duplicate titles and descriptions, H1s, heading order, canonical tags, hreflang, broken internal links.
• Answer quality. Whether a page states what it is and answers a buyer's question near the top, and whether the site has FAQ, pricing, comparison and about pages that get quoted.

3. How the score is built

Each check passes or raises an issue per page. Issues are grouped by severity (Error, Warning, Notice) with the number of pages each one affects. Sections get a letter grade from A+ to F and the site gets one overall grade. Open any issue to see every affected URL with the value we measured on it (for example the word count or the status code) and the date of the crawl, so you can check any count against your own site.

Could not verify. If a page refuses our crawler (HTTP 401, 403 or 429), shows a firewall challenge, is blocked by your robots.txt or does not answer, we cannot tell what is on it. We list it as “could not verify” with the reason, skip its checks and leave it out of the score, so a firewall never costs you points. Allow our crawler and re-crawl to have those pages checked.

4. The fixes

Where a fix can be pasted, we give it: the robots.txt lines, a JSON-LD block, an llms.txt file, a rewritten answer block. Re-crawl after you change something and the score moves with it; scheduled re-crawls show the trend.

5. What we do not claim

• A good score does not guarantee that any assistant will name or link your site. It tells you that nothing on your side is stopping it.
• We do not report a "visibility percentage" from a sample of assistant answers. Anything we have not measured is shown as not measured, never as 0.
• Some checks are judgements about quality (is this page quotable). They are marked as such and carry less weight than the access and readability checks, which are exact.
• We do not use private or consumer chat interfaces of any vendor. Where a check calls a vendor service, it uses its official API.

Changes to the method

Last updated 18 October 2026. When a check is added or changed, this page changes with it. Who operates CitedBy and which CitedBy this is: About CitedBy. Questions about a result: hello@citedby.eu.