GA4 vs Server Logs: Tracking AI Traffic Honestly
GA4 captures clicks from AI tools as referral traffic. Server logs capture every HTTP request, including the AI bot crawls that GA4 will never see. If you only check one, you miss most of the picture.
GA4 will show you chatgpt.com / referral and perplexity.ai / referral sessions — real human visits from AI answers that included a link to your site. Server logs will show you GPTBot, ClaudeBot, PerplexityBot, and other AI crawlers fetching your pages directly, often before any citation appears in an AI answer. Both signals matter and they tell you different things.
This post covers what each method captures, what each method misses, and how to use the two together to understand AI activity on your site. For the broader tracking framework — manual query audits, share of voice, and what cannot be tracked yet — see How to Track Whether AI Is Citing You.
Quick Navigation
- What GA4 Captures and What It Misses
- Why Server Logs See Things GA4 Cannot
- The Major AI Bot User Agents
- Where to Get Your Server Logs
- How to Cross-Reference GA4 and Server Logs
- What Each Signal Means in Practice
- Frequently Asked Questions
What GA4 Captures and What It Misses
GA4 captures real human visits from AI tools as referral traffic, but it cannot see AI bot crawls, citations without click-throughs, or visits where the AI tool strips the referrer.
In GA4, AI referral traffic appears under Reports > Acquisition > Traffic acquisition with source/medium values like:
chatgpt.com / referralperplexity.ai / referralclaude.ai / referralgemini.google.com / referralcopilot.microsoft.com / referral
Each of those sessions represents a person who saw your site cited in an AI answer, clicked the link, and landed on your page with cookies and JavaScript enabled. That is a high-value signal — actual humans who chose to visit you after an AI recommendation.
What GA4 does not capture:
AI bot crawls. Bots do not run JavaScript, so the GA4 measurement script never fires. GPTBot can hit your page 1,000 times and GA4 will show zero.
Zero-click citations. When an AI quotes your page in its answer and the user gets what they need without clicking, GA4 sees nothing. The citation happened, but the visit did not.
Privacy-stripped referrals. Some AI tools or user configurations strip the Referer header. When that happens, GA4 sees the visit as (direct) / (none) even though it actually came from an AI tool.
Bot-only visits. A small number of "user" visits in GA4 may actually be sophisticated bots. GA4's bot filtering catches the obvious ones, but not all of them.
The GA4 number is a floor on AI activity — useful, but always an undercount.
Why Server Logs See Things GA4 Cannot
Server logs record every HTTP request to your site, regardless of whether the visitor runs JavaScript. AI bot crawls show up in logs even when GA4 shows nothing.
A server log entry includes the timestamp, IP address, requested URL, HTTP status code, response size, referer (if any), and user agent. The user agent is what identifies an AI bot. When OpenAI's training crawler hits your site, the user agent string contains "GPTBot" — your log file shows that explicitly.
This matters for AI search tracking for three reasons.
Bot crawl frequency tells you whether AI knows you exist. If GPTBot hits your site daily and PerplexityBot weekly, those AI systems have indexed you and are keeping their information current. If neither has visited in months, you may be invisible to those platforms.
Crawl patterns reveal what AI thinks is important. If a bot fetches one page repeatedly and ignores the rest, that page is probably what AI returns when relevant queries are asked. If a bot crawls everything once a year and never returns, your content is in the index but not actively refreshed.
Discrepancies between bot activity and human referrals expose the citation gap. A site with high GPTBot crawl volume but zero chatgpt.com / referral sessions is being read by AI but not cited in AI answers. That is the structural problem the Brass-SEO AI Citability audit is designed to solve.
The Major AI Bot User Agents
Major AI vendors publish their bot user agent strings in their official documentation. The list changes as vendors add and rename bots, so check vendor docs for current names — but the common ones as of early 2026 are listed below.
| Vendor | Bot user agent | Purpose |
|---|---|---|
| OpenAI | GPTBot |
Training crawler |
| OpenAI | ChatGPT-User |
User-initiated browsing |
| OpenAI | OAI-SearchBot |
Search-related fetches |
| Anthropic | ClaudeBot |
Training crawler |
| Anthropic | Claude-User |
User-initiated browsing |
| Anthropic | Claude-SearchBot |
Search-related fetches |
| Perplexity | PerplexityBot |
Training crawler |
| Perplexity | Perplexity-User |
User-initiated browsing |
Google-Extended |
AI training opt-out signal (not a separate fetch) | |
| Common Crawl | CCBot |
Public crawl used by many AI training pipelines |
| ByteDance | Bytespider |
TikTok / Doubao training |
Google-Extended is different from the others: it is a robots.txt token, not a distinct crawler. Google's regular Googlebot performs the fetch, and the Google-Extended token tells Google whether to use the content for AI training and AI features. You will not see Google-Extended as a user agent in your logs.
A few practical notes. AI bot user agents change. Vendors add new bots, rename old ones, and split single bots into multiple roles (training vs. browsing vs. search). The list above reflects published documentation as of early 2026 and should be verified against current vendor docs before you build automated parsing. The Brass-SEO crawler also has its own identity — Brass-SEO/1.0 (+https://brass-seo.com/bot) — visible in customer logs when we audit a page.
Where to Get Your Server Logs
How you access server logs depends on your hosting platform, and not every platform exposes them at the level you need for AI bot analysis.
Vercel provides runtime logs through the dashboard and via CLI. Logs include the full request including user agent. For long-term retention and analysis, route logs to a log drain service like Axiom, Datadog, or Logtail.
Cloudflare logs every request through Cloudflare Workers Analytics or Logpush. AI bot fingerprinting is one of Cloudflare's stronger features — they identify bots by IP range and user agent and surface AI bot traffic as a separate category in the dashboard.
Traditional hosting (cPanel, VPS, dedicated) typically writes Apache or Nginx access logs to a known location on the server. You can grep them directly. AWStats and GoAccess parse these logs into reports.
WordPress hosts often expose access logs through cPanel or a plugin. Some managed hosts (Kinsta, WP Engine, Pressable) provide log access through their dashboards.
If your hosting does not expose logs at all, you have two options. First, switch to a host that does — log access is increasingly standard. Second, use a CDN that logs centrally (Cloudflare's free tier exposes some bot data even on the free plan).
For analysis tools, GoAccess is the simplest option for one-off audits. Axiom and Datadog handle ongoing monitoring with alerting. Brass-SEO does not currently parse customer server logs — its tracking comes from GSC and GA4 — so the server-log work is something you set up alongside your Brass-SEO subscription, not inside it.
How to Cross-Reference GA4 and Server Logs
Combining GA4 and server logs for the same date range gives you the full picture: bot activity from logs, real human visits from GA4, and the gap between the two.
A useful monthly workflow:
Step 1: Get the bot crawl count. From server logs, count how many times each major AI bot user agent fetched your site over the last 30 days. This is your raw "AI is reading you" signal.
Step 2: Get the human referral count. From GA4, count chatgpt.com / referral, perplexity.ai / referral, claude.ai / referral, and gemini.google.com / referral sessions over the same 30 days. This is your "AI is sending you traffic" signal.
Step 3: Compute the ratio. A site with 10,000 GPTBot fetches and zero ChatGPT referral sessions has a citation gap — AI is reading the content but not citing it in user-facing answers. A site with 500 GPTBot fetches and 100 ChatGPT referral sessions has high citation conversion.
Step 4: Identify the popular pages from bot logs. Which URLs are AI bots fetching most? Those are likely the pages AI considers most relevant for its training or live retrieval. Compare to your top organic pages in GSC — overlap is expected, but mismatches are interesting.
Step 5: Cross-check with manual query audits. Take the most-fetched URLs and run their target queries through ChatGPT and Perplexity. Are those pages cited in the answers? If not, the structural gap is what to fix.
This is not a perfect causal model — bot crawls do not guarantee citations, and citations do not require recent crawls (training data updates lag the live web). But the cross-reference reveals patterns that neither signal would show alone.
What Each Signal Means in Practice
Server logs and GA4 measure different parts of the AI traffic pipeline. Treat them as complementary, not redundant.
| Signal | What it means | What to do with it |
|---|---|---|
| High AI bot crawl, zero AI referrals | AI is reading your content but not citing it | Audit citability — answer capsules, original data, expert quotes |
| Moderate AI bot crawl, growing AI referrals | Citations are working; structure is paying off | Identify the cited pages, replicate the pattern across more content |
| Low AI bot crawl, low AI referrals | AI may not have indexed you | Check robots.txt, ensure no AI bots are blocked; check for noindex |
| Low AI bot crawl, high AI referrals | Citations come from training data, not live retrieval | Continue current strategy; expect slower iteration cycle |
For sites just starting to track AI traffic, the simplest setup is the GA4 referral check (5 minutes weekly) plus a quarterly server log review (20 minutes per quarter). The full cross-referencing workflow takes more setup but gives meaningfully more insight once it is in place.
For the broader tracking framework — manual query audits, share of voice, third-party tracking tools — see How to Track Whether AI Is Citing You. For the structural fixes that close the citation gap when bots crawl but do not cite, see the Generative Engine Optimization guide.
For the GA4-specific setup that captures Google AI Mode traffic, see Tracking AI Mode Traffic in GA4: A Setup Guide.
Frequently Asked Questions
Why does GA4 show fewer AI sessions than I expect?
GA4 only sees humans who clicked through with JavaScript enabled and the referrer intact. Bot crawls, zero-click citations, and privacy-stripped referrals do not appear. The GA4 number is always a floor, not a complete count.
Should I block AI bots in robots.txt?
That is a content rights decision, not a technical one. If you want AI systems to cite your site, do not block them. If you do not want your content used in AI training, block GPTBot, ClaudeBot, PerplexityBot, and CCBot in robots.txt. Note that blocking a training bot does not prevent live retrieval bots from fetching your site to answer user queries.
Are AI bot crawls counted in my Google Analytics user count?
No. Bots do not run JavaScript, so the GA4 measurement script does not fire. Your GA4 user counts reflect humans only — assuming GA4's bot filtering caught the obvious bot traffic.
What if my host does not give me access to server logs?
Use a CDN or proxy that does. Cloudflare's free tier exposes some bot identification, even without paid Logpush. Vercel and similar platforms provide runtime logs through their dashboards. If none of those are available, your last option is to switch hosts — log access is standard at any host that takes performance and security seriously.
Do AI bots respect robots.txt?
Major vendors publicly state that their bots respect robots.txt directives. Independent verification of compliance varies by vendor. If you block a bot in robots.txt and still see it in your logs, file a complaint with the vendor and consider blocking by IP at the network level.
How often should I review server logs for AI bot activity?
Quarterly is sufficient for most small business sites. Monthly if you are actively iterating on GEO content and want to see how bot crawl patterns change. Daily monitoring is overkill unless you are running a large content operation or seeing aggressive crawl behavior that affects performance.
Does Brass-SEO show me my server log data?
No. Brass-SEO's data sources are Google Search Console (search performance), Google Analytics 4 (traffic and engagement), and live page crawls (for the AI Citability audit). Server log analysis is a separate workflow outside the Brass-SEO product.