Your Website Is Majority Bot Traffic Now — Here's What Hosts Are Doing About It

Some time in 2024, the internet underwent a metamorphosis where bots — which were once just blockable nuisances — began generating more than 50% of all global web traffic.

To clarify, that’s not spam traffic; that’s all web traffic, period.

Now, those bots — crawlers, scrapers, AI agents, automated bullcrap — are making decisions as to whose sites show up in things like AI overviews and ChatGPT responses.

To be clear, this is not a cautionary tale as much as it is understanding what today’s evolved bots are doing. It’s mostly AI training, search indexers, LLM crawlers fetching answers for someone’s question. They’re not malicious; they’re more so doing a job.

But “not malicious” doesn’t mean “not a problem.” Today’s bots are notorious for repeatedly hammering endpoints, which skyrockets network and bandwidth bills, as well as skews user analytics among other KPIs.

To its credit, the hosting industry has moved fast with formal responses in the form of bot traffic controllers. But whether it actually helps resellers and agencies who are in the middle of bot management and their clients depends on who you’re hosting with.

The Numbers Got Wild

In 2026, according to HUMAN Security’s report, AI-driven bot is officially growing 8x faster than human traffic is. Agentic AI traffic specifically — that’s bots acting on behalf of users in real time — grew 7,851% YoY. Nope, not a typo.

Then there’s Cloudflare’s 2025 Year in Review, which took data from more than 81 million HTTP requests per second across 125 countries.

It found that AI bots (excluding Googlebot) averaged 4.2% of HTML requests across Cloudflare’s network last year, swinging from 2.4% in early April to 6.4% in late June. User-action AI bots — bots responding to actual user queries — grew more than 15x over the course of last year.

GPTBot alone grew 305% between May 2024 and May 2025. Makes you think about what this year’s data will look like.

What we do know is that 80% of today’s AI crawling activity is purely for model training. This means it generates zero referral traffic back to customers’ sites while stealing and passing along the information.

Cloudflare found that Anthropic has the highest crawl-to-refer ratios among major AI platforms, ranging from around 25,000:1 to 100,000:1. That means for every visitor Anthropic sent back, its crawlers made up to 100,000 requests against that site.

WP Engine also recently reported some scary numbers. In 2025 alone, it mitigated 75 billion bot requests. It also found that 76% of all traffic was completely unverifiable — meaning it could’ve come from anywhere (but probably bots).

“Automated traffic is becoming more difficult to manage as AI systems, bots, and human visitors increasingly blend together,” said WP Engine’s CTO, Ramadass Prabhakar. “Web teams need clearer insight into traffic behavior and the flexibility to respond quickly without adding operational complexity.”

To help its customers (and resellers) deal with the fast-rising volumes of unwanted bot traffic, WP Engine just launched Global Edge Security (GES), a Cloudflare-powered solution that adds customizable bot controls.

SiteGround recently did something similar, though from a different angle: They’re blocking AI training crawlers at the server level by default. This actually allows user-action crawlers through: A crawler indexing your site for ChatGPT’s search results might actually send you users, while a crawler scraping your content to train an LLM certainly will not.

Kinsta kind of went in the opposite direction entirely, committing to not billing customers for bot-generated bandwidth and not automatically blocking AI crawlers.

“From an infrastructure perspective, there’s no such thing as ‘just bot traffic.’ Every request is real work,” said Kinsta CTO Daniel Pataki in its recent report.

And almost everyone probably knows about Cloudflare’s big move last year (which arguably inspired others to follow). Last July, the CDN provider introduced pay-per-crawl, which lets site owners set a price (minimum $0.01 per request) for AI crawlers to access their content. They can also block or allow fully.

Cloudflare’s own VP of Product, Michael Tremonte, gets it.

“Not all automated activity behaves the same way, and not all of it is necessarily malicious,” he said. “The challenge is helping web teams distinguish bot traffic, and within that good from bad, without slowing performance or increasing operational overhead.”

There’s Also a Cost Problem

Has anyone noticed that security and analytics are the ones getting all the attention in this bot discussion? Only a handful are even pointing to the silent killer: the bill.

Hosting plans — bandwidth limits, compute allocations — are typically based on human traffic patterns. And that’s an issue since bots simply don’t behave like humans. For example: A human visitor loads a webpage, browses through a few product pages, maybe even converts. An AI crawler repeatedly hits endpoints over and over.

In fact, Kinsta’s analysis of 10 billion requests across its own infra found patterns where a single misbehaving bot managed to trigger enough traffic to justify its own mitigation rule. It’s as Kinsta said: It’s “not just quantity. It is repetition, loops, and waste.”

Speaking of Kinsta, it alongside WP Engine is one of the managed providers to take action. It announced a billing model update that eliminates charges for bandwidth consumed by bots and scrapers. WPE also updated its billing language to exclude suspected bot traffic from billable metrics.

Both are rare acts of kindness that go a long way for hosts and their users.

What Can Resellers and Agencies Actually Do?

If you’re managing client sites — whether as a WPE reseller, a Kinsta agency partner, or running your own stack — the bot traffic problem is already your problem, even if you don’t yet realize it.

Contentsquare’s 2026 benchmarks also reported that 59% of sites saw traffic decline in 2025 from 2024 while cost per visit rose 9% YoY. For hosting providers who haven’t filtered bots yet, your economics may be off by potentially a third. Ouch.

Start with visibility. Before you can make any decisions about blocking or allowing, you need to understand what’s actually hitting your clients’ sites.

Most managed WordPress platforms now surface bot traffic data in dashboards — use it. You can also see breakdowns between the bot types (Google, Bing, etc.), unverified bots, and human traffic to see a full breakdown.

This naturally segues into fixing your analytics, too. Whether you’re reporting traffic numbers to clients or they’re doing it on their own, look for bot exclusion in whatever stack you’re using, like GA4.

Start with blocking by bot type if you don’t want to make a sweeping decision. User-action agents are different categories than the LLM/AI training crawlers. Blocking GPTBot while allowing OAI-SearchBot lets you deny OpenAI’s bots from using your content for free training while still staying visible on ChatGPT’s search results.

And for resellers/agencies, WPE’s GES is a perfect example of the most direct play. Amsive’s director of web development, Mark Davoli, puts it well.

“[It] is the ability to quickly see unwanted bot activity actually happening across a site and take action without adding technical overhead,” he said. “For agencies managing multiple client environments, that means less time reacting to traffic and security issues and more time improving client performance, protecting revenue, and driving growth.”