What is the purpose of this tool?
This tool helps you verify if your website allows or blocks AI web crawlers (like ChatGPT, Claude, and Gemini) via robots.txt, and validates your sitemap structure.
Which AI bots does it check?
We currently check for GPTBot (OpenAI), CCBot (Common Crawl), Google-Extended (Gemini/Bard), AnthropicAI (Claude), FacebookBot, and Applebot-Extended.
Why should I block AI bots?
You might want to block AI bots to prevent your content from being used to train AI models without your permission, or to save server bandwidth.
Why should I allow AI bots?
Allowing AI bots helps your content appear in AI-generated answers (like ChatGPT search or Google AI Overviews), potentially driving new traffic sources.
How do I block GPTBot?
Add 'User-agent: GPTBot' followed by 'Disallow: /' to your robots.txt file.
What is a sitemap validator?
It checks if your XML sitemap follows the standard protocol, ensuring search engines can discover and index your pages correctly.
What errors does the sitemap checker find?
It detects invalid XML syntax, incorrect root elements (must be urlset or sitemapindex), and empty sitemaps.
Can I validate a sitemap index file?
Yes, the tool supports both standard sitemaps (<urlset>) and sitemap index files (<sitemapindex>).
Does this tool save my data?
No, all validation happens in your browser. Your robots.txt and sitemap content is not sent to any server.
Is this tool free to use?
Yes, this tool is 100% free and unlimited for all users.