See which crawlers actually arrive
Separate training crawlers, search indexers, and live user-fetch agents, with volume and trend per crawler.
Measure the input side
Agent Analytics
Most GEO measurement only covers the output side — what engines say. Agent Analytics covers the input side by reading your existing CDN access logs: which AI crawlers arrived, what they requested, whether they got a 200, and which visits came from a human following an AI citation.
Connect CDN access logs to see which AI crawlers reach your site, which pages they read, whether they were served or blocked, and how much human traffic AI answers refer back.
What this helps your team do
Find content AI cannot read
Surface real content pages that returned 4xx or 5xx to a crawler — a fixable cause of absence rather than a ranking mystery.
Close the crawl-to-answer gap
Compare pages crawled heavily against pages that never surface in tracked answers, so structure problems become visible.
How it works
STEP 01
Route existing CDN logs
Send access logs from CloudFront, Cloudflare, Fastly, Akamai, or Google Cloud CDN to the ingest endpoint. No application code changes.
STEP 02
Verify crawler identity
Validate source IPs against published bot ranges and confirm reverse DNS, so spoofed requests are counted separately from verified crawls.
STEP 03
Read pages and referrals
Review which content pages were served or blocked, with static assets and probes filtered out, and track human visits referred from AI platforms.
Evidence available
- Identified crawler, verified or unverified
- Request volume per crawler with trends
- Content pages accessed, served, or blocked
- Purpose classification: training, indexing, or live-user fetch
- AI-referred human traffic by referer and UTM attribution tier
Interpretation limits
- Agent Analytics needs a connected CDN log source; without one there is nothing to read.
- Google Search and AI Overview referrals share one referer and cannot be separated in logs, so they are reported blended.
- Unverified requests claiming a crawler identity are reported separately and should not be read as confirmed crawler activity.
Product sources
Frequently asked questions
Does Agent Analytics require changes to my site?
No. It ingests access logs your CDN already produces, routed to the ingest endpoint via the provider's existing log delivery feature.
How do you know a crawler is really who it claims to be?
Source IPs are cross-referenced against published bot ranges and reverse DNS is checked. Requests that fail verification still count toward volume but are reported separately from verified crawl analytics.
Can you separate AI Overview referrals from normal Google Search?
No. Both arrive with the same referer, so they are reported together as a blended Google figure rather than split on a distinction the logs do not carry.