AIO score
Strong74 / 100
AI visibility report
webcrawlerapi.com · Last audited May 3, 2026
AIO score
Strong74 / 100
Ghost Test
No gaps0 / 1
key questions AI crawlers couldn't answer
Summary
webcrawlerapi.com is in good shape for AI crawlers (AIO 74/100). The most room to improve is Renderability & JS (6/20). AI crawlers can find a description of what the company does in the HTML your server sends.
We ask an AI model each question twice: once using only the HTML a crawler like GPTBot receives, and once using the page as visitors see it after JavaScript runs. A question is flagged when only the visitor version can answer it.
Homepage only: some of the pages the targeted questions need were not found, so we asked one broad question about the homepage instead.
Page: https://webcrawlerapi.com/
Both versions could answer: AI crawlers get the same information visitors do.
01 WebcrawlerAPI is a web crawler API that allows you to extract data from all pages of a website with a single request. It makes your life better by providing clean markdown that can be used directly in AI support agents, handling complexities like JavaScript, CAPTCHAs, and proxies automatically. You can keep your knowledge base fresh with its Feeds feature, and you only pay for what you use without needing a subscription.
01 This is the WebCrawler API, which allows you to turn documents and websites into clean markdown for AI agents. It makes your life better by providing a reliable way to extract useful content from web pages, removing unnecessary clutter, and formatting it for easy use in AI applications. It also offers features like smart caching for faster access, change detection for keeping content updated, and no-code integrations for ease of use.
Each sampled page, scored on what an AI crawler receives before any JavaScript runs.
homepage · 0% of visible text in server HTML
pricing · 0% of visible text in server HTML
docs · 0% of visible text in server HTML
docs · 0% of visible text in server HTML
product · 0% of visible text in server HTML
security · 0% of visible text in server HTML
blog · 0% of visible text in server HTML
blog · 0% of visible text in server HTML
No crawl warnings: every sampled page loaded.
This report shows what AI crawlers read. It doesn’t test whether ChatGPT, Claude and Perplexity get past the site’s firewall when a person asks them about a page. If this is your company, the Threshold Test checks each assistant against your key pages and tells you what to change. It’s free once you’ve claimed this profile.
The Ghost Test checks whether an AI crawler can answer the questions buyers ask, such as pricing and main features. We ask an AI model each question twice about the page most likely to answer it: once using only the HTML the server sends (what crawlers like GPTBot read, since they don't run JavaScript), and once using the page as visitors see it. A question is flagged as a gap when only the visitor version can answer it.
No. A gap means the information exists on the site but AI crawlers can't read it without running JavaScript. That raises the risk that an AI assistant guesses or leaves the fact out, but it is not a record of a specific wrong answer.
The AIO score adds up four pillars: technical & crawlability (25 points), renderability (20), structure (30) and metadata (25). Each is scored on the HTML AI crawlers receive. The methodology page lists every check.
Coverage describes how many of the site's important pages the audit managed to sample; certainty describes how consistent the evidence was. Lower values mean the report is directional and worth re-running.