Developers Are Split on llms.txt: 54% of Hacker News Comments That Take a Side Support It, 46% Doubt It

Summary

Picture a small business owner who hears from a web agency that they need an llms.txt file 'for AI search'. It sounds like robots.txt for chatbots. Is it worth paying for? Developers have been arguing about that exact question on Hacker News for two years.

llms.txt is a proposed plain-text file, placed at the root of a website, that gives AI tools a short map of the site's most useful content. It was proposed in September 2024. No major AI company has said its crawlers use it, but many documentation sites now publish one.

We read every Hacker News comment that mentions it. Opinion is close to an even split: 54.0% of comments that take a side support it and 46.0% doubt it. The support is mostly about something different from what agencies sell.

What we measured

We pulled every live comment and story title in the public Hacker News dataset on Google BigQuery that mentions llms.txt, llms-full.txt or close variants, from September 2024 through September 2026. That gave 391 comments and 111 stories.

Our team read all 391 comments and coded each as supportive, skeptical or no clear stance, then noted the main argument. Comments that only shared a link, asked a question or mentioned the file in passing were coded as no stance. Because this is every comment rather than a sample, there is no sampling error, though coding involves judgment.

Close to an even split among developers who take a side

Most mentions were neutral. People shared links to a project's llms.txt, asked whether a tool supports it, or compared it with other formats. Among those with a view, neither side dominates.

Skeptics outnumbered supporters in the first months after the proposal. Support rose in 2025 as documentation sites adopted the file, then the gap narrowed in 2026 as more people tested whether AI crawlers fetch it.

Supporters mostly use it for coding help, not search visibility

The two camps often talk past each other. Supporters mostly describe pasting a library's llms.txt into a coding assistant so it uses current documentation. Skeptics mostly ask whether AI search crawlers read the file on ordinary websites.

Several commenters who run sites or platforms reported checking their server logs and seeing few or no requests for llms.txt from major AI crawlers. Others pointed out that the file's own author has said it is meant for individual users' assistants rather than for crawlers. Both points can be true at once.

Lots of launches, little attention

The topic produces a steady stream of submissions, many of them tool launches, but few reach a wide audience.

Of the 111 stories, 29 were Show HN launches (26.1%), mostly generators, validators and directories. 91 scored 3 points or fewer (82.0%) and only 5 reached 10 points. The two stories that passed 100 points were the original proposal in September 2024 and a directory of llms.txt files in December 2024.

What this means

An llms.txt file is cheap to add and does no harm, but developers who have checked find little sign that AI search tools read it. Its clearest use, in these comments, is helping coding assistants read software documentation. For a typical business website, it should not be the centerpiece of an AI visibility plan, and nobody should charge much for it. Real SEO for AI search starts with the content on your pages, not a file at the root of your domain.

What AI assistants do read is your actual pages and what other sites say about you. Clear facts, consistent listings and pages that answer real customer questions matter more. Our free AI visibility audit checks those first, and if an AEO agency leads with llms.txt instead, ask why.

Limits

Stance and argument coding is our team's judgment. Some comments mix support and doubt and were coded by their main point.

Keyword matching can miss comments that discuss the file without naming it.

Hacker News commenters are mostly developers. Their view of llms.txt reflects technical use cases more than marketing ones.

We did not test whether AI crawlers fetch llms.txt files. Statements about crawler behavior are what commenters reported.

Dead and deleted comments carry no text in the dataset. Data runs through September 2026.

Frequently asked questions

Related studies

• Developer conversation on Hacker NewsAI SEO Draws 3x the Positive Developer Comments of SEO
• Developer conversation on Hacker News'Written by AI' Comments on Hacker News Up 4.6x Since 2023
• Developer conversation on Hacker News'Slop' Now Shows Up in 1 of Every 149 Hacker News Comments
• Developer conversation on Hacker News1 in 5 Popular Hacker News Stories Is Now About AI