Downloads of AI-First Scraping Tools Ran at 61.7% of Their Weekday Pace on Weekends in 2026, Compared With 48.0% for Classic Python Scrapers

Summary

Picture a florist who checks her server logs on a Sunday and sees steady visits from tools she has never heard of. Nobody in an office pressed a button. Software set up weeks ago is running on its own schedule.

New AI-first scraping tools, built to feed web pages to language models, are installed from the Python Package Index like any other library. PyPI logs record which installer was used, whether the machine reported itself as a CI test runner, and the time of day, so we can see how much of this tool use looks like people at work and how much looks like machines.

In the Q3 2026 sample days, downloads of five AI-first scrapers slowed less on weekends: the daily weekend pace was 61.7% of the weekday pace, against 48.0% for classic scrapers. Three in four downloads came through uv, a fast installer common in automated builds, and almost none came from a web browser.

What we measured

We queried the public PyPI download log in Google BigQuery (bigquery-public-data.pypi.file_downloads), which records one row for every file downloaded from the Python Package Index. To keep the query affordable we sampled the first seven days of every month from January 2022 through September 2026, 399 days in total, and summed downloads by month, package, country, CI flag, installer and weekday versus weekend into one working table. A quarter in this study therefore means 21 sampled days (days 1 to 7 of each of its three months), not a full quarter. We removed downloads made by mirroring and proxy tools (bandersnatch, Nexus, Artifactory, devpi), which copy packages in bulk rather than install them. To control for the growth of PyPI itself we also pulled total daily download counts for the whole index from the table's partition metadata for the same sampled days.

We compared five AI-first scraping packages (crawl4ai, firecrawl-py, firecrawl, scrapegraphai, browser-use) with four classic Python scraping packages (scrapy, requests-html, selectolax, parsel). browser-use made up 82.3% of AI-first scraper downloads in Q3 2026, so the group's pattern is shaped mostly by its installs. Each sampled week has five weekdays and two weekend days, so a quarter has 15 weekdays and 6 weekend days; we compare average downloads per day. We report CI flags, installer names and weekend pace as signals of automation. No single signal proves a download came from a server.

AI-first scrapers slow down less on weekends

People install software mostly on workdays. Scheduled builds and auto-scaling servers do not care what day it is.

A year ago AI-first scrapers looked like classic scrapers, at about half the weekday pace on weekends. By Q3 2026 their weekend pace had moved to 61.7%, close to the 65.5% seen across all of PyPI, where automated traffic is heavy.

Three in four downloads come through an automation-friendly installer

The installer name shows how a package was fetched. pip is the classic tool a developer types at a terminal. uv is a newer, faster installer that is popular in containers, CI and tools that set up environments on the fly.

Developers also use uv by hand, so the installer alone does not prove a machine was behind a download. Taken with the weekend pace, the jump from 43.3% to 76.5% in a year suggests these tools are being installed more and more by scripts and pipelines.

Almost nobody downloads these tools by hand from a browser

A download from a web browser usually means a person clicked a file on pypi.org. In Q3 2026 that happened 17,791 times out of 35,909,995.

The CI flag tells a smaller story: 13.0% of AI-first scraper downloads were flagged as CI in Q3 2026, below the 46.0% for classic scrapers. Many automated environments, such as production containers or agent sandboxes, do not set a CI variable, so the flag undercounts automation.

What this means

AI tools that read websites are increasingly set up once and then left to run. For a business, that means visits from these tools come at any hour of any day, and a page that is slow, blocked or broken on a Sunday is broken for them too. Keep key pages fast and reachable all week, and check that your server or security plugin is not blocking well-behaved AI crawlers you actually want.

You cannot greet these visitors, but you can control what they find. Our free AI visibility audit shows how assistants describe your business and which crawlers reach your pages, and that is enough to plan your first round of LLM optimization.

Limits

A download is not a user. One developer, one Docker build or one CI job can download the same package many times, and a single busy pipeline can move a package's numbers sharply from month to month.

Figures cover the first seven days of each month only (21 days per quarter), so they are sample totals, not full-month or full-quarter totals. Growth rates compare like-for-like sample periods.

PyPI as a whole grew from 12.1 billion downloads in the Q3 2022 sample days to 115.0 billion in the Q3 2026 sample days (9.5x), so part of every package's growth is the index growing. We report share of all PyPI downloads where it matters.

PyPI-wide downloads in the September 2026 sample week were 38.1% lower than in the August 2026 sample week (28.7 billion vs 46.4 billion), so September figures for individual packages should be read with that drop in mind.

The CI flag is only populated in this dataset from March 2024, and it only catches environments that announce themselves as CI. Automated downloads that do not set a CI variable are counted as non-CI.

Country codes come from the downloading IP address. Cloud servers sit mostly in the United States, so country splits describe where machines run, not where developers live.

Weekend pace, installer and CI flag are indirect signals. A developer can use uv at a desk, and a server can use pip.

One package made up most of the AI-first group's downloads in Q3 2026, so the group's pattern largely reflects its installs.

Country data is not used in this study because download location mostly reflects cloud server regions.

Frequently asked questions

Related studies

• Downloads of the tools that read websites for AIAI Agent Framework Downloads Topped 108M in One Week
• Downloads of the tools that read websites for AIWeb-to-Text Tool Downloads Grew 71x Since 2022
• Downloads of the tools that read websites for AIMCP Downloads Now Exceed 10 Agent Frameworks Combined
• Downloads of the tools that read websites for AIAI Browser Agent Downloads Grew 54x in a Year