Your Analytics Dashboard Is Missing Part of the Internet

I have a slightly uncomfortable question for anyone reporting on website performance every month:

Do you actually know who is using your website?

Because in 2026, the answer isn’t as obvious as it used to be.

We normally open Google Analytics and see:

Users.

Sessions.

Traffic sources.

Landing pages.

Conversions.

And we treat that as our website audience.

But increasingly, there is another audience accessing websites constantly that traditional marketing dashboards are barely showing you.

AI agents.

AI search crawlers.

Training bots.

Retrieval systems.

Search bots.

Automated tools acting on behalf of actual people.

And the really important part is that they’re not all doing the same thing.

More than half of web traffic can now be non-human — but don’t panic

You may have seen headlines saying:

“Most internet traffic is now bots.”

That sounds dramatic.

And technically, recent Cloudflare data supports the general direction: the company reported that more than 50% of the traffic it observes is now non-human. 

But this is exactly where marketing stats get mangled.

It does not mean 50% of the people visiting your homepage are secretly Claude wearing a trench coat.

Cloudflare is measuring HTTP requests travelling across its network.

A crawler can request thousands of pages.

A human might visit three.

So request volume and human sessions are not remotely the same metric.

Still, the trend matters.

Because websites increasingly have machines consuming their content before, instead of, or on behalf of humans.

And most businesses aren’t measuring that layer at all.

GA4 is not wrong. It is showing you a different part of reality.

Google Analytics automatically excludes traffic from known bots and spiders.

You can’t switch that exclusion off.

You also can’t open a report showing exactly how many known bots were removed. 

That’s generally a good thing.

Nobody wants their monthly report saying:

“Amazing news! Traffic increased 320%!”

only to discover an automated crawler requested /services/ 18,000 times.

But it creates an interesting situation.

Your website could be receiving requests from AI systems that are reading your pages, retrieving your content and potentially using it to answer someone’s question…

while your analytics dashboard shows nothing happened.

That means GA4 can be completely accurate for what it is designed to measure and still not represent every meaningful interaction happening with your website.

We need to stop calling everything “AI traffic”

There are a few completely different things being lumped into that phrase.

A training crawler might access your content to help build or improve a model.

A search or retrieval crawler may access pages so an AI product can understand what exists and surface information later.

A user-triggered agent may fetch your website because somebody has actively asked an assistant to research something.

Then there is an actual human referral, where someone sees your business inside ChatGPT, Gemini, Perplexity or another AI experience and clicks through to your website.

Commercially, those are very different behaviours.

Cloudflare’s June 2026 data illustrates that difference nicely: 52% of crawler requests were classified as AI training, while more than 36% came from mixed-use crawlers spanning activities including search, agents and training. 

So simply saying:

“ChatGPT crawled our website 800 times.”

doesn’t tell me whether that’s good.

I want to know why.

An AI crawler isn’t automatically a lead

This is where I think AEO reporting could get very silly very quickly.

We’ve already had years of SEO reports celebrating:

Impressions.

Rankings.

Keyword counts.

Traffic.

without asking whether any of those things made the business money.

Let’s please not create the AI version.

“ClaudeBot visited us 4,000 times this month!”

Okay.

Did that lead to your company being recommended?

Did humans click through?

Did branded search increase?

Did anyone enquire?

Did the crawler successfully access commercially important pages?

Was it simply downloading thousands of articles for training?

Those are very different outcomes.

AI crawl activity should be considered a signal, not automatically a KPI.

But ignoring agent activity completely isn’t smart either

The other extreme is:

“They’re bots. Who cares?”

I don’t think that’s going to age well.

AI assistants are increasingly acting between users and websites.

Someone might ask:

“Find me three UK web agencies that specialise in rebuilding websites after bad agency experiences.”

An AI system may research agency websites, compare information and present three options.

That person might never conduct the fifteen traditional Google searches we used to see.

The AI has done part of the browsing.

Your eventual visitor might arrive much later, perhaps by typing the company name directly into Google.

Your reporting could then say:

Source: Organic Search

when their actual discovery journey started inside an AI assistant.

Attribution was messy before AI.

It is about to get messier.

Your website now has at least four measurement layers

This is the framework I’d use going forward.

Measurement layer

What it tells you

Example

Human behaviour

What actual visitors do on the site

GA4, Webflow Analyze, Framer Analytics, heatmaps

Search visibility

How people discover you through traditional search

Search Console, rankings, impressions

Machine access

What crawlers and AI agents are requesting

Server logs, CDN/bot analytics

Business outcomes

Whether any of this creates commercial value

CRM, forms, calls, proposals, revenue

The mistake is expecting one tool to answer all four.

Google Analytics is brilliant for analysing human website behaviour.

Search Console is much better for understanding Google’s organic search layer.

Server or edge infrastructure can show bots that never behave like a normal browser session.

Your CRM tells you whether any of those visitors actually became opportunities.

The interesting insight appears when you connect those layers.

Your server logs suddenly matter to marketing

Server logs used to be one of those things marketers happily left developers to deal with.

And, to be fair, most of the time you probably still should.

They’re not exactly a thrilling Monday-morning read.

But they contain something increasingly important:

Requests that marketing analytics never see.

Cloudflare launched its Attribution Business Insights dashboard in July specifically around this problem, showing businesses bot-versus-human activity, AI bot traffic and crawl-to-referral ratios. 

That tells you where analytics is heading.

The question is shifting from:

“How many people visited?”

to:

“Who or what accessed this content, why, and what value came back?”

That is much more useful.

WordPress, Webflow and Framer businesses will approach this differently

This isn’t really a platform war.

The measurement options simply differ depending on your infrastructure.

On a WordPress site, you may have access to hosting logs, security tooling, CDN-level analytics or Cloudflare data alongside GA4.

On managed platforms such as Webflow or Framer, your normal workflow may rely more heavily on their platform analytics plus any external analytics or edge infrastructure you’ve configured.

Webflow itself has been investing heavily in bringing analytics closer to website management. Its current Analyze product exposes visitor behaviour such as traffic sources, top pages, engagement and click behaviour directly inside Webflow. 

That’s useful.

But the key distinction remains:

Human behaviour analytics and machine-access analytics answer different questions.

You shouldn’t replace one with the other.

You need to know which one you’re looking at.

This also changes how we report SEO

Imagine organic website sessions drop 12%.

Traditionally, everyone panics.

SEO is falling.

Agency meeting.

Emergency spreadsheet.

But what else happened?

Did branded searches increase?

Did AI referrals increase?

Did conversion rate improve?

Did qualified enquiries remain stable?

Did your pages start being fetched more frequently by search/retrieval agents?

Did your average customer arrive more informed?

Did revenue change?

Traffic still matters.

But interpreting it without the wider journey is getting increasingly dangerous.

One of the biggest mistakes businesses could make in the AI-search era is optimising solely for:

more sessions.

I’d much rather see:

12,000 sessions → 6 qualified enquiries

become:

8,500 sessions → 14 qualified enquiries.

Marketing dashboards tend to emotionally reward the bigger number.

Businesses should reward the useful one.

Don’t immediately block every AI bot either

Once businesses realise AI systems are crawling their websites, the natural reaction is sometimes:

Block them.

And there are absolutely legitimate reasons to restrict particular crawlers.

Cloudflare itself has introduced much more granular controls because the old relationship between crawlers and publishers has changed dramatically. It now allows website owners to distinguish and control different types of AI access rather than treating everything as one category. 

But whether you should allow a crawler depends on what it’s doing.

There is a difference between:

An AI system training on your content.

A search crawler helping your business become discoverable.

An agent retrieving your pricing because a potential customer specifically asked about your services.

Blanket rules can accidentally remove useful visibility.

The right question isn’t:

“Should we block AI?”

It’s:

“Which automated visitors provide value to this business, and which don’t?”

That’s a much better governance decision.

Your robots.txt file has become a business decision

I never thought I’d write that sentence in a marketing blog.

Yet here we are.

Historically, robots.txt was mostly something SEO people and developers worried about.

Now crawler policies can influence:

Search discovery.

AI retrieval.

Model training.

Content usage.

Infrastructure load.

Potential agent access.

That means decisions around crawler access shouldn’t quietly be made by whoever happens to manage the hosting account.

Marketing needs input.

SEO needs input.

Development needs input.

And for organisations with sensitive content, legal or compliance may need input too.

Because you’re not just deciding whether a robot can index /about-us/.

You’re deciding how external systems are allowed to consume your company’s information.

Ask your agency what your analytics actually measure

This is probably my biggest practical takeaway.

If you receive a monthly website report from an agency, ask them:

What exactly is included in these numbers?

If they say:

“Website traffic.”

Ask again.

Is that GA4?

Server traffic?

Cloudflare traffic?

Search Console clicks?

Sessions?

Users?

Requests?

Are bots excluded?

Are internal employees excluded?

What counts as a conversion?

How are AI referrals categorised?

Which traffic sources become “direct”?

You should be able to understand your report without becoming a data scientist.

And your agency should be able to explain the limits of its data.

A dashboard isn’t useful because it has twelve graphs.

It’s useful because you can make a decision from it.

Don’t create another vanity dashboard

We don’t need a new chart containing:

AI BOT HITS ↑ 72% 🎉

with absolutely no explanation.

If I were building website reporting for a business now, I would separate the information into three questions.

Are people finding us?

Search visibility, referrals, brand discovery and relevant traffic.

Are people doing what we want when they arrive?

Engagement, journeys, conversions and qualified actions.

Are machines successfully accessing the information we want them to access?

Crawler activity, agent access, errors and important pages being retrieved.

Then I’d bring all of that back to:

Did it help the business?

Leads.

Sales.

Applications.

Registrations.

Bookings.

Revenue.

Whatever the website actually exists to create.

Do a 30-minute analytics reality check

Open your latest website report.

Pick the five metrics everyone talks about most.

For each one, ask:

What decision would we make if this number increased?

Then:

What decision would we make if it decreased?

If nobody can answer either question…

I’d question why it’s on the report.

Then check whether your reporting covers human behaviour, organic search visibility and actual commercial outcomes.

Finally, ask whoever manages the website whether anyone is monitoring server/CDN-level bot activity.

You don’t necessarily need a giant AI-agent dashboard tomorrow.

You just need to understand which parts of your website activity are visible and which aren’t.

That’s the first step.

The web isn’t becoming less measurable. Measurement is becoming more layered.

I don’t think GA4 is dead.

I don’t think traditional SEO metrics are dead.

And I definitely don’t think businesses need to abandon website traffic as a metric.

We just need to stop pretending one analytics dashboard represents the entire internet.

The browser used to sit directly between a person and a website.

Increasingly, there can be another layer in the middle:

person → AI → website.

Sometimes the AI sends the person onwards.

Sometimes it retrieves the answer itself.

Sometimes it simply learns from the content.

Those behaviours should not all be measured the same way.

So next time somebody opens the monthly report and says:

“Website traffic is down.”

My first question would be:

“Which traffic?”

Because in 2026, that’s no longer a pedantic question.

It might be the difference between understanding your website and completely misreading it.