Skip to content

llms.txt

A proposed standard — a text file in the website root that briefly tells AI models what the site offers and where to find key content.

llms.txt is a proposed web standard (llmstxt.org): a markdown file placed at /llms.txt that gives large language models a concise, curated overview of a website — what the site offers, what its key sections are and where to find important content without wading through HTML templates.

Structure of llms.txt:

  • A heading with the website name and a one-sentence description
  • Sections with links to key pages (products, categories, FAQ, contact)
  • Optionally /llms-full.txt with extended content

Why have one: AI crawlers process huge numbers of pages on a limited token budget. llms.txt gives them a shortcut to the essence of your website — it increases the chance that the model understands what you sell, correctly and quickly. For online stores it is useful to add information about shipping, returns and unique selling points — exactly the things customers ask AI about.

Relationship to robots.txt: llms.txt neither blocks nor allows anything — it is a content map. AI crawler access is controlled by robots.txt (GPTBot, ClaudeBot, PerplexityBot); the Google-Extended token in it also determines whether Google may use the content for Gemini.

Does AI mention your store?The free audit shows you in about a minute.Start an audit