LLMs.txt and llms-full.txt: the new standard for AI-first websites
TL;DR
llms.txt is a proposed standard from September 2024 by Jeremy Howard (fast.ai) that defines a structured overview of a website's content specifically for LLM agents. As of spring 2026 it is supported by OpenAI, Anthropic, Perplexity and several smaller AI crawlers. For online stores it is the simplest way to tell AI models "this is what we offer, take it seriously". Implementation takes 30 minutes.
Why a new standard at all?
Until now, three files mattered to crawlers:
robots.txt— what may and may not be crawledsitemap.xml— a list of URLsschema.orgJSON-LD — structured data per page
The problem: LLMs don't read HTML the way a browser does. During grounding they pull only ~8–32 KB of text per query. If your homepage contains 2 MB of JS rendering, the AI may never reach the content at all.
llms.txt solves exactly this: a plain-text markdown file with a prioritised list of what the AI should know about your site, including links to detailed content in a simple format.
Anatomy of an llms.txt file
Stored at https://yourdomain.com/llms.txt. Markdown format:
# Pozorio
> SaaS for monitoring the visibility of online stores in AI search engines
> (ChatGPT, Gemini, Perplexity, Google AI Overviews).
## What we offer
- Weekly monitoring for Slovak and Czech online stores
- Weekly action plan with concrete recommendations
- Multi-channel alerts (email, Slack, Google Chat, webhooks)
## Plans
- Monitor — €39/mo, 1 brand, 50 queries
- Optimize — €99/mo, 3 brands, 150 queries
- Dominate — €299/mo, 10 brands, daily monitoring + API
## Key pages
- [Homepage](https://www.pozorio.sk/): Free audit at no cost
- [Documentation](https://www.pozorio.sk/docs): How the audit and monitoring work
- [Roadmap](https://www.pozorio.sk/roadmap): What we're building
- [Pricing](https://www.pozorio.sk/cennik): Plan comparison
## Optional
- [Changelog](https://www.pozorio.sk/changelog): New features
- [Status](https://www.pozorio.sk/status): Health dashboard
Key elements
# Name— your brand name> Blockquote— one sentence on what you do (the AI uses it as a TL;DR)## Sections— logical sections with short links## Optional— optional content the AI doesn't always have to load
llms-full.txt — full context for agents
The second file, llms-full.txt, contains the complete text of all pages as a single markdown file. It is used when an agent needs real context to generate an answer without further HTTP requests.
For an online store, that means:
# Your Online Store — Full Reference
## About us
We are a Slovak online store founded in 2020…
## Product categories
### Running shoes
We stock Nike, Adidas, Asics, New Balance…
[View all 247 products](https://yourshop.com/running-shoes)
### Cycling gear
…
## Delivery and returns
Delivery within 2–5 working days. Free returns within 30 days…
## Contact
Email: info@yourshop.com
Phone: +421 2 1234 5678
Size: typically 50–500 KB. The agent pulls the whole thing on its first query and then caches it.
Why AI browsers want it
Claude Computer Use, ChatGPT Operator and Perplexity Sonar will grow in 2026 as "AI shopping assistants". When a user says "find me the cheapest running shoes under €100", the agent visits 10–20 online stores and makes a decision.
If your site:
- Renders via JS without SSR → the agent sees an empty page
- Requires a login to show prices → the agent gives up
- Has 30 trackers + a cookie banner → the agent gets stuck
...then you lose to a competitor that has an llms-full.txt with clean context and an llms.txt with direct links.
Practical implementation steps
Step 1: Generate llms.txt
For a static site — by hand, 30 minutes. For WordPress / Shopify / WooCommerce:
- WordPress: the "LLMs.txt Generator" plugin (available in the repository since 2025)
- Shopify: the "AI Search Ready" app generates it automatically
- Next.js: add a route handler
app/llms.txt/route.ts:
export async function GET() {
const body = `# ${process.env.BRAND_NAME}
> ${process.env.BRAND_DESCRIPTION}
## Categories
- [Running shoes](https://yourshop.com/running-shoes)
- [Cycling](https://yourshop.com/cycling)
## Documentation
- [Delivery](https://yourshop.com/delivery)
- [Returns](https://yourshop.com/returns)
`;
return new Response(body, {
headers: { 'Content-Type': 'text/plain; charset=utf-8' },
});
}
Step 2: Generate llms-full.txt
This is not something you want to do by hand. A typical approach:
- A script walks the entire sitemap.xml
- For each URL it extracts the text with
@mozilla/readability - It concatenates everything into a single markdown document
- It uploads it to
/llms-full.txt
Run it as a weekly cron job. Total size target: up to 5 MB.
Step 3: Confirm in robots.txt
User-agent: OAI-SearchBot
Allow: /llms.txt
Allow: /llms-full.txt
User-agent: Claude-SearchBot
Allow: /llms.txt
Allow: /llms-full.txt
Step 4: Test
curl https://yourshop.com/llms.txt
curl https://yourshop.com/llms-full.txt
The content must be plain text/markdown, Content-Type: text/plain; charset=utf-8, HTTP 200.
Why you should care now
- Early adopter advantage: as of spring 2026, fewer than 2 % of the top 100 Slovak online stores have an
llms.txt. Whoever adds it now has a clear path to AI citations. - Low effort, high upside: 30 minutes of work, resulting in a 10–20 % increase in AI mention rate according to our clients' A/B tests.
- A future requirement: OpenAI is proposing that from 2027
llms.txtbe mandatory for appearing in the ChatGPT Search "Top Brands" panel. Better to be among the early adopters.
Sources and further reading
- llmstxt.org — the official specification
- Jeremy Howard — Why llms.txt (YouTube)
- Anthropic Claude docs: llms.txt support
Pozorio automatically checks whether your site has an llms.txt and flags it as a recommendation. Run a free audit — you'll see the result within 2 minutes.
Try our free tools
Don't wait for an audit — our generator and validator give you an answer in 30 seconds. No sign-up needed.
- Generate an llms.txt for your online store — enter your domain; we fetch your homepage, navigation and sitemap and return a ready-made llms.txt in Slovak, Czech, English or German.
- Validate an existing llms.txt + robots.txt AI bot policy — checks AI crawlers (GPTBot, ClaudeBot, PerplexityBot…) and the Google-Extended and Applebot-Extended robots.txt tokens, and returns actionable recommendations.
For specific platforms we also have step-by-step install guides: Shopify, WooCommerce, Shoptet, WordPress, Webflow, Wix, Framer, Upgates.