What is llms.txt?
llms.txt is a proposed standard: a Markdown file in the root directory of a website that gives language models a compact overview of the most important content.
Definition
The file lives at /llms.txt and is written in Markdown. It contains the name of the website, a short summary and links to the most important pages, organized into sections, each with an explanatory sentence. The proposal dates from 2024 and is aimed at AI systems that read a website at runtime.
How it differs from robots.txt and a sitemap
robots.txt governs what crawlers are allowed to fetch. The XML sitemap lists all URLs as completely as possible. llms.txt does neither: it is a curated content overview readable by humans and machines alike – deliberately short, so that it fits into a language model's limited context window.
Structure
- An H1 heading with the name of the website or brand
- A blockquote with a summary in one or two sentences
- Optionally, further paragraphs with background
- H2 sections with link lists in the format “[Title](URL): description”
- An “Optional” section for content that can be dropped when context is tight
A realistic view of the benefits
llms.txt is not a binding standard, and the major AI providers have not uniformly committed to supporting it. The effort is small, however, and the file forces you to name your core content clearly. As a complement to crawlable content and structured data it makes sense; as a standalone measure it does not.
Creating an llms.txt with KI.SO
KI.SO's llms.txt generator produces a ready-made draft from your brand data and your page structure, which you copy or download. Publish the file in the root directory of your website – KI.SO itself provides an example at kiso.pro/llms.txt.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal with no standards body behind it. Adoption is growing; support from AI providers is inconsistent.
Can I use llms.txt to block AI crawlers?
No. Access rules belong in robots.txt. llms.txt is a content overview, not a control file.
How long should an llms.txt be?
As short as possible: the 10 to 30 most important pages, each with one descriptive sentence. Detailed content belongs on the linked pages.
Related topics
- Site audit & llms.txt: Check how well search engines and AI models can read your website: multi-page site audit, llms.txt generator and Organization markup built from your data.
- GEO: Generative Engine Optimization (GEO) explained simply: definition, how it differs from SEO, the most important tactics and how to measure GEO success.
- LLMO: LLMO (Large Language Model Optimization) explained: how language models build knowledge about brands, training knowledge vs. web search, and what you can do.