Why single-fetch scanners are structurally blind to AI-agent attacks

By Youssef Boukachabine · 2026-05-09 · ~12 min read

Burp Suite, OWASP ZAP, Snyk, and every other web vulnerability scanner share a hidden architectural assumption: they fetch each URL exactly once, as a generic crawler. That assumption broke the moment AI agents started reading websites on behalf of users — and the gap between what traditional scanners see and what AI agents see is now the threat surface that nobody is monitoring.

This post explains the architectural gap concretely, walks through three attack classes that only become visible through multi-agent fetching, and frames why AI-agent web security is a category that requires different infrastructure, not just different signatures bolted onto existing scanners.

What is single-fetch scanning?

Single-fetch scanning is the architecture every traditional web vulnerability scanner uses today: for each URL in scope, the scanner makes one HTTP request, parses the response, and runs detection rules against the response body. The request typically uses a generic crawler user-agent (often a customized identifier the scanner brand controls) and follows a single render path.

This works fine for the threat model these scanners were built around: an attacker injecting JavaScript that targets a human visitor in a browser. The browser will fetch the page once, render it, and the human will see what's rendered. If the scanner's single fetch reveals a malicious payload, the human's single fetch will too.

The architectural assumption baked into this design: the response any visitor receives is functionally equivalent to the response a scanner receives. Same URL, same response, same threat surface.

For human-targeted attacks, that assumption holds. For AI-agent-targeted attacks, it breaks completely.

How do AI agents fetch differently?

When ChatGPT browses a webpage to summarize it for a user, the request lands on the target server with the user-agent ChatGPT-User or OAI-SearchBot. When Claude does the same, it identifies as ClaudeBot or Claude-Web. Perplexity sends PerplexityBot. Microsoft Copilot identifies as part of Bingbot with extended fingerprinting. Google's Gemini extension sends Google-Extended. Each has documented its bot identity for site operators to honor or block.

This sounds like a minor implementation detail. It is not. It is the entry point for an entirely new attack class.

The moment a server can distinguish an AI-agent request from a human-browser request, the server can serve different content to each. From the scanner's single fetch, the server can serve clean content. From an AI agent's fetch, the server can serve a payload designed specifically to manipulate that agent. The traditional scanner, fetching once with its generic crawler identity, never sees the payload at all.

This is not a theoretical concern. It is the operational reality of how every major AI agent currently fetches the web in 2026.

What three attack classes does single-fetch scanning miss?

1. User-agent cloaking against AI agents

A server detects an AI-agent user-agent and serves a different response than what a human or scanner would see. The malicious response contains prompt-injection content; the human-facing response is clean.

# Server pseudocode
if request.headers["User-Agent"].lower() in {
    "chatgpt-user", "oai-searchbot",
    "claudebot", "claude-web",
    "perplexitybot",
    "google-extended"
}:
    return inject_malicious_content(legitimate_page)
else:
    return legitimate_page

From a single-fetch scanner's perspective, nothing is wrong. The site looks identical to what humans see. From an AI agent's perspective, the site is now a hostile content surface designed to manipulate the agent's output to the user. Detection is impossible without making at least two fetches with two different identities and comparing them.

This pattern is structurally identical to historical SEO cloaking (where sites served different content to Googlebot vs. browsers). The difference: SEO cloaking targeted a search index; AI-agent cloaking targets the actual answer a user receives.

2. Dynamic injection conditional on agent-detection JavaScript

A server returns identical HTML to every fetcher, but the HTML contains JavaScript that detects the runtime environment and injects payload only when an AI agent's renderer is detected. Single-fetch scanners that do not execute JavaScript miss this entirely. Single-fetch scanners that do execute JavaScript still miss it because they execute as their own generic identity, not as the AI agent the payload is targeted at.

The detection logic can be subtle: missing browser APIs (no WebGL context, no navigator.plugins), specific viewport dimensions associated with headless browsers, missing user-interaction events, or fingerprinting against the AI agent's known render harness. When the conditions match, the script injects content into the DOM — content that the AI agent then includes in its summary.

This attack class is currently undetectable by any scanner that doesn't render the page as the actual target AI agent and capture the post-script DOM.

3. Agent-specific payload tuning

The same site can serve subtly different payloads to ChatGPT vs. Claude vs. Perplexity, calibrated to each model's specific tokenization, system-prompt structure, and behavioral tendencies. A payload that works against ChatGPT may be ignored by Claude; a payload tuned to bypass Claude's safety alignment may be over-aggressive for Perplexity and trigger its refusal patterns.

An attacker who has tuned their payload library against multiple agents can serve each agent its custom payload, maximizing the success rate of manipulation. A scanner fetching as a single identity sees only one payload variant — if any. The full attack surface is only visible when comparing what each AI agent receives, side-by-side.

Why this matters

Each of these three attack classes exploits the same architectural assumption: that one fetch reveals what every fetcher will see. The attack vector exists because attackers can rely on scanners not making the second, third, fourth, fifth fetch under different agent identities. The solution is not better signatures applied to a single fetch — it is a different scanning architecture entirely.

What does multi-agent fetching actually do?

Multi-agent fetching is the architectural alternative: for each URL in scope, the scanner makes multiple requests, each impersonating a different AI agent's documented user-agent and request fingerprint, and compares the responses for divergence.

The detection signal is the diff itself. If the response to ChatGPT-User differs from the response to ClaudeBot in non-trivial ways — different DOM nodes, different alt-text, different HTML comments, different SVG content, different inline scripts — that divergence is the suspicious pattern. Legitimate adaptive serving (mobile detection, language headers, geolocation) accounts for some divergence; the rest is signal.

Three capabilities make this work:

Parallel fetches across canonical AI-agent identities. ChatGPT, Claude, Perplexity, Copilot, Googlebot, Google-Extended, Applebot-Extended — the documented bot list grows every quarter. Coverage matters; missing one means missing the cloaking specifically tuned to that one.
DOM-aware diff, not raw-HTML diff. Whitespace and tracking-pixel differences are noise. The diff has to operate on parsed DOM, comparing semantically meaningful nodes — visible text, alt-text, computed CSS visibility, comments, embedded SVG, inline scripts, metadata.
Signature library for known prompt-injection phrasings. “Ignore previous instructions” and its variants. Adversarial alt-text patterns. SVG <text> elements with non-rendering styles. The library evolves weekly because attackers evolve weekly. This is the part that has to be maintained continuously, not shipped once.

None of these capabilities exists in any single-fetch scanner today. Building them in is not a small refactor; it is a different scanning architecture.

Why hasn't traditional security tooling caught up?

Three structural reasons:

The threat model assumes the wrong target. XSS, SQL injection, CSRF — the OWASP Top 10 that scanners optimize against — all assume the attacker is trying to compromise a human user via their browser. None of them describe an attacker manipulating an AI agent that reads the DOM as input to a model. The scanner's detection rules are built for the wrong threat model. Adding a sixth detection rule for “Ignore previous instructions” payloads doesn't change the architecture; it just adds a substring search to the wrong scanner.

The OWASP LLM Top 10 (LLM01:2025: Indirect Prompt Injection) names the threat but doesn't ship tooling. The OWASP framework documents the threat class explicitly. The implementation gap — what infrastructure detects this in production — remains the responsibility of the security tooling industry. To date, that industry has not shipped multi-agent diff scanning at the depth the threat requires. Vendors are starting to add “AI-aware” modes to existing single-fetch products; structurally, that's the wrong fix.

Single-fetch is computationally cheap; multi-agent is not. A single-fetch scanner can audit a 10,000-page site for the cost of 10,000 HTTP requests. A multi-agent scanner with 5-7 agent identities makes 50,000-70,000 requests for the same coverage, plus the DOM-diff and analysis work on top. The cost is real, and incumbent vendors with single-fetch infrastructure have a financial reason to call multi-agent “overkill” rather than rebuild their architecture.

What detection actually requires architecturally

If you accept the framing that AI-agent web security is a category — not a feature added to existing scanners — the architectural requirements clarify:

Capability	Single-fetch scanner	Multi-agent scanner
Fetches per URL	1	5-7+ (one per AI agent identity)
Detection signal	Pattern match in response	Pattern match + cross-agent diff
Required render	Optional (some don't render)	Mandatory (must capture post-script DOM per agent)
Update cadence for signatures	Quarterly	Weekly (adversaries iterate fast)
Detects user-agent cloaking	No	Yes (the diff itself is the signal)
Detects dynamic agent-conditional injection	No	Yes (post-render DOM diff)
Detects agent-tuned payload variants	No	Yes (per-agent payload comparison)
Cost per scan	1× baseline	5-7× baseline

The cost multiplier is real, but it's also the price of seeing the threat surface that exists. A 1×-cost scanner that misses the threat is not 7× cheaper; it's free, in the sense that an unused product is free.

What should defenders do today?

Three concrete moves, ranked by leverage:

Add an explicit entry to your threat model document for “AI-agent-targeted injection.” Most security teams in 2026 have not done this. The OWASP LLM Top 10 categorization (LLM01:2025) gives you the standard reference. Until this is on the threat model, no tooling decision against it is meaningful.
Audit your site as multiple user-agents manually. Use curl with -A "ChatGPT-User", then -A "ClaudeBot", then -A "PerplexityBot", against your homepage and three highest-traffic pages. Diff the responses. Any non-trivial divergence is either intentional (geolocation, CDN) or unintentional (injection, cloaking, third-party widget compromise). Investigate every divergence.
Add multi-agent scanning to your CI/CD pre-deploy pipeline. Whether you build the multi-agent diff yourself or use a tool, the integration point is the same: every deploy fetches the changed pages as N AI agents in parallel and fails the deploy on suspicious divergence. Most threats arrive via third-party content or compromised dependencies; the scan window is the deploy gate.

The category framing

This is the broader point: AI-agent web security is a category, not a feature. The category exists because AI agents read web content as input to models, and that creates a threat surface that no human-targeted security tooling was designed to monitor. The category requires different scanning architecture, different detection signatures, different update cadences, and different threat-model entries from anything in the OWASP Top 10 (the human-targeted one) or any traditional web vulnerability scanner.

Single-fetch scanners are not bad; they're optimized for a different threat model. Adding AI-agent detection to them is structurally the wrong move — not because it can't catch some payloads (it can), but because it leaves the architectural gap (cloaking, dynamic injection, per-agent tuning) wide open. The right move is purpose-built infrastructure that fetches as the actual target agents and diffs the results.

That infrastructure is what EverHarden is. The free first scan fetches your site as five AI agents in parallel, diffs the responses, and surfaces divergence. If your site is clean, you get a clean report. If it isn't, the report shows you exactly what each agent sees that the others don't — which is the threat surface that single-fetch scanners cannot reach.

Run a free first scan →