Skip to content

Free tool · llms.txt and robots.txt validator

Do AI crawlers understand your website?

Enter a domain. We check llms.txt against the specification and the robots.txt rules for AI crawlers such as GPTBot or PerplexityBot.

No sign-up · 3× per day without signing in

How it works

  1. 1We load llms.txt and robots.txtFrom the root of your domain.
  2. 2We check the structureTitle, description, sections and links per llmstxt.org.
  3. 3We check the AI crawlersWhich of them robots.txt lets in and which it blocks.
A tool shows one moment. Pozorio measures the same questions repeatedly on several AI platforms and shows the trend.Free audit

Frequently asked questions

Questions about the tool

Didn't find your answer? Write to info@pozorio.sk — we usually reply within 24 hours.

What exactly does this validator check?
Three things: 1) that your llms.txt exists and is syntactically correct according to the llmstxt.org spec, 2) whether the extended llms-full.txt version is present, 3) whether robots.txt allows access to AI crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, PerplexityBot, Bytespider, CCBot) and what the rules say for the Google-Extended and Applebot-Extended tokens. Those two are not crawlers — they are only robots.txt tokens Google and Apple use to control how content is used for their AI models.
What is Google-Extended and how does it relate to Googlebot?
Google-Extended is neither a crawler nor a user agent — you will never see it in your logs. It is just a token for robots.txt rules. Your site is always fetched by Googlebot; the Google-Extended rule only decides whether Google may use that content to train and ground Gemini models (the Gemini app, Vertex AI). It has no effect on Google Search or AI Overviews — those are governed by Googlebot, so you must not block Googlebot if you want to appear in AI Overviews. Applebot-Extended works the same way: Applebot fetches the site and the token only decides on training Apple’s AI models.
A score of 50 — is that enough?
For most small e-shops, yes, but if you want to target AI traffic seriously, aim above 80. The biggest difference comes from: a valid llms.txt (45 pts) + unblocked AI crawlers (25 pts) + a Sitemap declaration in robots.txt (10 pts).
Do you cache results?
Yes, for 24 hours per domain. If you make changes in the meantime, the old result stays until the next day. For an immediate revalidation, write to info@pozorio.sk.
Is it safe? Will you download my website?
We only fetch 3 public files: /llms.txt, /llms-full.txt and /robots.txt — each with an 8 s timeout and a 1 MB cap. No authenticated pages, no forms, no POST requests. Our user agent: Pozorio-Validator/1.0.