Seven Open-Source Tools That Turn Websites Into AI-Ready Text Drew 1 of Every 417 GitHub Stars in 2026

Summary

Picture a bakery owner who asks ChatGPT for the best cake shop nearby, and the answer quotes another bakery's menu word for word. The assistant did not read that menu the way a person does. A tool fetched the page, stripped the clutter and passed plain text to the model. Tools like that can shape what an AI system sees on your site.

A small group of open-source projects does this job at scale. They crawl a page, remove navigation and ads, and return clean Markdown that a language model can read. Developers use them to build chatbots, search tools and AI agents.

We tracked seven of the best known. Together they drew 1 of every 417 stars on GitHub in 2026, up from 1 in 780 in 2024. Developers are betting heavily on this layer.

What we measured

We used GH Archive, the public log of GitHub events, in Google BigQuery. We followed seven repositories by their permanent repo ID, so renames are counted once. We counted stars (WatchEvents), forks, issue events and pull request events by year and by half year.

GH Archive recorded far fewer stars in 2026 than in earlier years. To compare fairly we divided the seven tools' stars by every star GH Archive recorded in the same period. We name tools only to report adoption counts. We make no claims about how well any of them work.

Their share of GitHub attention keeps climbing

We measured stars per million of all recorded GitHub stars, by year.

Raw stars rose 23.1% from 2024 to 2025. Measured as a share, attention grew 1.7 times. Only GPT Crawler existed for most of 2023, which is why that year is low.

Attention is concentrated in a few tools

Recorded stars and forks for each tool, January 2023 to October 20, 2026.

Three tools account for 198,255 of the 256,294 stars (77.4%). Earlier entrants that peaked in 2024 now draw little new attention, while the leaders keep adding stars.

Developers are building on these tools, not just bookmarking them

Forks are copies developers make to modify or self-host a project, a stronger signal than a star.

The fork share doubled in 2025 and held there in 2026. Issue and pull request events tell the same story: 5,517 issue events and 5,105 pull request events across the seven repos, most of them in 2024 and 2025.

What this means

When an AI assistant reads your website, one of these tools may have done the fetching. Tools like these are built to pull out a page's main text. Content locked in images, sliders, tabs that need clicks or text loaded late by scripts is generally harder for any text extractor to pick up. Plain, well-structured HTML gives every one of them the best shot at reading your page correctly.

Check what your key pages look like as plain text. If your prices, service areas or hours vanish, an AI tool may never see them. Seeing how AI systems currently read and describe your business, which a free AI visibility audit shows, means any LLM optimization starts from facts rather than guesses.

Limits

The tool list is our selection of widely used web-to-LLM extraction projects. Other tools exist and are not counted.

Stars and forks are those recorded by GH Archive. GH Archive captured far fewer star and fork events in 2026 and had collection gaps in October 2025 and August to October 2026, so 2026 raw counts understate real activity. We use per-million shares to compare years.

Stars show interest, not usage. Many of these tools are also offered as paid hosted services whose usage does not appear on GitHub.

We make no claims about the quality or internal behavior of any tool.

Frequently asked questions

Related studies

• The open-source tooling around AI searchGEO and AI SEO Repos on GitHub Hit 322 a Month in 2026
• The open-source tooling around AI searchOne List Holds 81% of AI Bot Blocklist Stars on GitHub
• The open-source tooling around AI searchllms.txt GitHub Projects: 83.1% Never Got a Star
• The open-source tooling around AI searchSEO Data MCP Servers on GitHub More Than Doubled in 2026