Source: https://aiexorcist.com/threshold
Last-verified: 2026-10-26

Summary: This mirrors the human page: what the AI Exorcist Threshold Test checks, how it works, and its limits.

Threshold Test (can AI assistants open a website?)

This mirrors the human page: what the AI Exorcist Threshold Test checks, how it works, and its limits.

When someone pastes a link into ChatGPT, Claude or Perplexity, the assistant sends its own fetcher to read the page. Firewalls, CDN bot protection and security plugins often turn those fetchers away, and nobody is told: the person just gets a vague or wrong answer. The Threshold Test checks whether each assistant gets in, page by page, and what to change if it doesn't.

What it tests

These are user-triggered fetchers: they read a page because a person asked about it. They are different from search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) and training crawlers (GPTBot, ClaudeBot), which the AIO audit's robots.txt check covers. A site can allow one kind and block another.

How it works

1. Probe as each assistant. We request a sample of key pages (homepage, pricing, product, docs and a few others) with each fetcher's exact user agent, plus an ordinary request with our own user agent as a control. Requests are spaced out and carry an X-AIExorcist-Probe header so site owners can find them in their logs.
2. Ask the assistant itself. For ChatGPT and Claude, we ask the assistant through OpenAI's and Anthropic's official APIs to read one key page (pricing, or the homepage if there is no pricing page). In our tests those requests came from the same fetcher identity and vendor network as the ChatGPT and Claude apps. We check the page title and heading the assistant reports against what is on the page. When this check gives a clear pass or fail, it decides the result for that page. Perplexity's API does not read a given page live, so Perplexity gets step 1 only.
3. Classify the cause and give the fix. Each failure gets a cause (firewall block, bot challenge, rate limit, login wall, redirect loop, timeout, empty or JavaScript-only page, robots.txt disallow) and short instructions for the detected setup: Cloudflare, WordPress security plugins, Vercel, Netlify, Nginx and others.

Results

Each page and assistant gets one of three results:

• Got in — the assistant's fetcher received the page.
• Turned away — it was blocked, challenged, sent elsewhere or given an empty page, while our control request got through (or robots.txt disallows a fetcher that follows it).
• Can't tell — our control request was turned away too, so the block is not specific to AI assistants, or the check could not finish. For ChatGPT and Claude, the API check settles these when it can.

Limits

• A pass covers the sampled pages, not every URL.
• Some firewalls decide by where a request comes from, not only by its name. Step 1 catches rules based on the fetcher's name. When our probe server is turned away too, we report "Can't tell" rather than guess. The API check covers one key page, not every sampled page.
• A failure is strong evidence, not proof, that every request from that assistant is blocked.
• We never try to get past a CAPTCHA or bot challenge. If we hit one, that is the finding.
• We only test sites whose owners ask us to. Results go in a private report and are never published in the Registry.
• The Threshold Test is reported separately and does not change the AIO score or the Ghost Test.

Pricing

• One-off Threshold Test: free for companies that have claimed their AI Exorcist profile; otherwise priced on request. Includes a private report and a free re-test after settings change.
• Threshold Watch: $30 / year. The test re-runs every month, and the owner gets an email when an assistant that used to get in starts getting turned away.
• Neither needs the AI Exorcist Mirror.

Request either at /threshold. Full pricing: /pricing.

Full page: /threshold · Audit methodology: /methodology