What is llms.txt?

llms.txt: definition and idea

Jeremy Howard of Answer.AI proposed llms.txt in September 2024. The idea: websites are designed for people and contain navigation, scripts, and ads that get in the way when AI systems read them. A lean text file at your-domain.com/llms.txt is meant to show which pages really matter.

That makes llms.txt a kind of table of contents for machines, similar to a sitemap but designed for readability by language models. It is often confused with robots.txt; llms.txt does not fulfill that file’s job of allowing or forbidding crawler access.

How an llms.txt is structured

The proposed format is simple Markdown: a heading with the website’s name is followed by a short summary and sections with links and short descriptions of the most important pages. Optionally there is a variant called llms-full.txt, which bundles entire content in a single file. The components at a glance:

• Title (H1) with the name of the company or project
• A short summary in a blockquote
• Sections with links to key pages, each with an explanatory sentence
• An optional section for less important content
• Served at /llms.txt in the root directory of the domain

Real-world example: a software vendor’s docu­menta­tion

A software vendor publishes extensive documentation for its interface, and developers use AI assistants to generate sample code. An llms.txt pointing to the most important documentation pages can help such tools find the right sources. That is where it makes the most sense, because developers can pass the file to their tools deliberately.

For the website of a tradesperson or service provider, by contrast, the benefit is hardly proven: here, content, structure, and trust signals decide visibility. There, classic measures such as clear copy, structured data, and good internal linking achieve more than an additional file.

Benefits, limitations, and common mistakes

The effort is small, and the file forces you to name your most important content. The benefit for visibility, however, is not proven: Google representatives said in 2025 and 2026 that they know of no AI system that evaluates llms.txt. As things currently stand, there is no publicly known commitment from major AI providers to take the file into account. Typical mistakes:

• Expecting llms.txt to improve ranking or mentions in AI answers
• File contents that differ from the website or are outdated
• Important information that appears only in llms.txt and not on the website
• Blocking AI crawlers at the same time via robots.txt or a firewall

What companies should look out for

Treat llms.txt as an optional extra, not a lever. More important are well-structured, accessible content, a clean sitemap, and open access for crawlers. If you create the file, keep it current and consistent with the website. Nano Concepts assesses whether llms.txt is worthwhile for your project and, if needed, generates it automatically from the content management system.