15.6% of Top Websites Answer Made-Up Addresses With a 200 Success Code, and They Account for 96.9% of Empty 'Agent Card' Responses

Summary

Imagine a marketing manager whose free online checker reports that her site already has an llms.txt file and an AI agent card. She never created either. The checker saw a 200 code and stopped looking. Her server sends a normal page for any address, real or not.

This is often called a soft 404 or a catch-all route: instead of saying "not found," the server returns the homepage or an error page with a success code. HTTP Archive tests for it by requesting a /.well-known/ address that should never exist. AI files like llms.txt, agent cards and UCP profiles all live at fixed addresses, so this habit directly affects how AI agents and audit tools read a site.

It is common. 15.6% of top homepages answer the made-up address with a 200. Those sites produce most of the false 'found' signals for AI files in the crawl.

What we measured

We used the HTTP Archive October 2026 mobile crawl (httparchive.crawl.pages) for top-1M homepages. HTTP Archive's well-known metric requests a deliberately nonexistent /.well-known/ path; we flagged sites where it returned 200. We kept the 717,015 sites where that test recorded a status.

We then compared catch-all sites with all others on each AI file path HTTP Archive checks. A response counts as an empty hit when HTTP Archive marked the path as found but recorded no real content: no llms.txt text content type, no agent card fields, no catalog entries, no UCP profile, no keys, no security.txt contact.

About 1 in 6 top sites says 'found' for anything

Share of sites answering the made-up /.well-known/ address with 200, by popularity rank.

Rates sit between 12.8% and 17.4% in every bucket, so popular sites do it too. Even in the top 1,000, 91 sites do it.

Catch-all sites create most fake AI file hits

Empty 'found' responses on catch-all sites, by file. Each row is a share of the 112,117 catch-all sites.

Any tool or agent that trusts the status code will think these sites publish half a dozen AI files. In reality almost none do.

Catch-all sites are less likely to have real AI files

Real AI files on catch-all sites vs every other homepage.

'Other homepages' includes the 55,397 sites where the test recorded no status, so its rates are slightly conservative. The gap is large either way: sites that answer everything with a page are less likely to publish the real files.

What this means

Test your own site by typing a nonsense address, such as yoursite.com/.well-known/test-123. If you get a normal page instead of a 404, your server has a catch-all route. Ask your developer to return a real 404 for missing files. It makes AI audits honest and stops agents from reading your homepage as if it were an AI file. An AI SEO consultant will check this early, since a fake 200 can make every other AI file test look like a pass.

When a tool says your site already has llms.txt or an agent card, open the file and look. A success code alone proves nothing. A clean 404 is a small fix; to learn whether AI assistants can actually find and quote your business, book a free AI visibility audit with our team.

Limits

The made-up address is a single test path; some servers treat /.well-known/ differently from other folders.

HTTP Archive did not record the test result for 55,397 homepages, which are left out of the catch-all rate.

'Empty' is based on fields HTTP Archive extracts; we did not read raw response bodies.

Single crawl, October 2026.

Frequently asked questions

Related studies

• llms.txt and AI instruction filesllms.txt Now Outnumbers security.txt Nearly 5 to 1
• llms.txt and AI instruction filesllms.txt Adoption 2026: 14.3% of Top Sites Have One
• llms.txt and AI instruction filesllms.txt Grew From 1.1% to 14.3% of Top Sites in 15 Months
• llms.txt and AI instruction files46% of llms.txt Files Are Added Automatically by Platforms