Free tool
Can AI crawlers read your page?
Check robots.txt rules for ChatGPT, Claude and Perplexity crawlers, plus the indexing basics that decide whether a page can be cited at all. Free, no sign-in, no AI calls.
What it checks
- Search crawlers: Googlebot and Bingbot (also behind AI Overviews, AI Mode and Copilot).
- AI answer crawlers: OAI-SearchBot and ChatGPT-User (ChatGPT), Claude-SearchBot and Claude-User (Claude), PerplexityBot.
- Training-only bots: GPTBot, ClaudeBot and Google-Extended, reported separately because blocking them doesn’t remove you from AI search.
- Indexing basics: status code, noindex, canonical, sitemap, title and description, and text in the server HTML.
Not sure which bots to allow? Read AI crawlers and robots.txt.
Questions
What does the AI crawler checker test?
It fetches your page and robots.txt once, then checks: HTTP status and redirects, meta robots and X-Robots-Tag noindex, the canonical URL, robots.txt rules for Googlebot, Bingbot, OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, Claude-User and the training bots GPTBot, ClaudeBot and Google-Extended, whether the page is in your sitemap, and how much text is in the server HTML.
Is blocking GPTBot bad for AI visibility?
Not for ChatGPT search, according to OpenAI: GPTBot is used for training, while ChatGPT search uses OAI-SearchBot. The checker reports blocked training bots separately and doesn’t count them as blockers.
Why does it say my page passed, but assistants still don’t cite me?
Crawl access is necessary, not sufficient. CDN bot protection can also block crawlers regardless of robots.txt, and the page still has to answer the buyer’s question better than the pages assistants already cite.
Do you store the pages I check?
The free checker doesn’t save results. It reads only the public URL you enter and refuses private network addresses.