Data study · 14 Oct 2026
Launch crawl 2026-10: Show HN and Lovable sites read as a crawler reads them
We fetched 300 sites that launched on Show HN (15 to 20 Oct 2026) and 70 sites built with Lovable, as CitedBot, the crawler CitedBy sells. 49 of the 66 Lovable homepages we could read hold under 100 words of text in the HTML they send (74%); for the Show HN sites it is 47 of 293 (16%).
Fetched 2026-10-20 to 2026-10-21 · robots.txt obeyed · no JavaScript · homepage plus up to 9 pages · Download summary CSV
The numbers
| Share of sites that could be checked | Show HN (293 read) | Lovable (66 read) |
|---|---|---|
| Homepage under 100 words of text in the raw HTML | 16% (47/293) | 74% (49/66) |
| robots.txt blocks at least one of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, PerplexityBot | 4% (12/294) | 0% (0/67) |
| robots.txt blocks at least one search crawler (OAI-SearchBot, Claude-SearchBot, PerplexityBot) | 0% (0/294) | 0% (0/67) |
| Homepage turned away a live request from at least one of those five crawlers (firewall) | 11% (33/293) | 0% (0/66) |
| Homepage turned away a live request from a search crawler | 1% (3/293) | 0% (0/66) |
| sitemap.xml present and valid | 68% (198/293) | 30% (20/66) |
| llms.txt present | 34% (100/293) | 18% (12/66) |
| JSON-LD on the homepage | 47% (137/293) | 23% (15/66) |
Each cell is a share with its count in brackets; the denominator is the sites for which that check could be run, so it differs by row. Sites we could not read are left out, never counted as failures.
What the numbers say
- MeasuredMost Lovable homepages send almost no text in the HTML: 49 of 66 under 100 words, median 2 words. A crawler that does not run JavaScript reads an empty page; a browser shows more.
- MeasuredAmong Show HN sites, 33 of 293 (11%) turned away a request that carried the user-agent of at least one big AI crawler, while CitedBot got the page. Only 3 of 293 did so for a search crawler.
- Measuredrobots.txt is a smaller factor: 12 of 294 Show HN sites and 0 of 67 Lovable sites block one of the five crawlers there.
- MeasuredSitemaps: 68% of Show HN sites and 30% of Lovable sites. llms.txt: 34% and 18%; it is optional and there is no good evidence the large assistants read it.
- Not measuredWhether the sites are cited by any assistant, traffic, or anything a browser shows after JavaScript. Page-title uniqueness had too few sites with 3 or more titled pages (Show HN 198, Lovable 15) and is not reported.
Method and limits
- Show HN cohort: the 300 newest Show HN stories up to 20 Oct 2026 23:59 UTC that link to the homepage of a site on its own domain (not a code host, social network, app store or shared-hosting subdomain), one per registered domain. Stories dated 2026-10-15 to 2026-10-20. These are launches of every kind (tools, libraries, games, apps), not only SaaS.
- Lovable cohort: 70 sites. 31 are linked from Hacker News posts in the 12 months before 20 Oct 2026 that say the site was built with Lovable or sit on a Lovable domain; the other 39 are homepages on lovable.app in the Common Crawl index of September 2026, taken in the order of the SHA-256 of the host name. Nobody picked sites. Builders with too few sites to publish a share: Bolt (3 listed) and v0 (1 listed) count only in the CSV row V.
- What we fetched: CitedBot/1.0, robots.txt first and obeyed, no JavaScript, at most 1 request a second per site, the homepage plus up to 9 internal pages,
/sitemap.xml,/llms.txt, and the homepage once per crawler with that crawler's published user-agent string. The last test shows how a firewall treats the string; it is not the vendor's own crawler, which a site may verify by IP address, so a 403 here can differ from what the real crawler gets. - Could not check: 7 Show HN and 4 Lovable sites did not return an HTML homepage to CitedBot (robots.txt unreachable or erroring, rate limits, not HTML). They are left out of every denominator.
- Limits: Show HN sites are hours or days old and some are unfinished. Lovable sites on the owner's own domain that say nothing public about the builder are not reachable by our rules, and default-domain sites include unfinished projects. Counts describe 20 and 21 Oct 2026. Site names are not published.
- Checked: twenty rows are re-fetched by hand by a second person; more than 1 mismatch in 10 holds back any post that uses these numbers.
- Raw data: the aggregate table, one row per segment with every count and denominator: download.
Is your site readable to crawlers?
We crawl your site up to your plan's page limit and give you the exact fix for each issue. From €9, 14-day money-back guarantee.