What Is llms.txt? Format, Example and Who Actually Reads It

llms.txt is one of the most discussed files in AI search advice, and one of the most misunderstood. It is often described as a sitemap for ChatGPT or a switch for AI citations. In reality it is a short Markdown file, proposed in 2024 and revised in 2026, that gives AI agents a curated map of a site's most useful content.

Whether you should publish one depends on what your site offers and who will read the file. Google Search has documented that it ignores llms.txt, and none of the AI crawler documentation we reviewed says the crawlers read it. At the same time, documentation platforms generate the file automatically and AI labs publish one for their own developer docs.

This guide explains the proposal and its format with a complete example, separates what vendors actually document from what is assumed, and shows how to decide, write and validate a file. Every vendor statement is dated October 2026.

What llms.txt is

The llms.txt proposal, written by Jeremy Howard and first published on September 3, 2024, suggests adding a Markdown file named llms.txt to a website "to provide information to help agents use a website". Version 2, modified on August 10, 2026, keeps the idea and spells out where files live and how agents should find and use them.

The reasoning is practical. HTML pages wrap their information in navigation, scripts and layout that are hard to turn back into clean text, and context windows are still too small for most websites in their entirety. A short, curated file with links to clean Markdown lets an agent fetch only what it needs. The proposal expects the file to be used mainly on demand, when an agent needs information while helping a user, rather than for training.

Three ideas in version 2

• Placement. The file can sit at the root, /llms.txt, or at any path such as /docs/llms.txt, and it covers the pages under that path. When several files apply, agents should use the most specific one.
• Markdown versions of pages. Pages that agents might need can offer a clean Markdown copy at the same URL with .md appended (page.html.md) or with the extension replaced (page.md).
• Discovery. A page can point to its Markdown version with rel="alternate" type="text/markdown" and to the llms.txt file that covers it with rel="describedby", either in HTML link elements or in an HTTP Link header.

Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"

The proposal is open for community input through a public GitHub repository. No standards body has adopted it, which is why validators, including ours, separate what the proposal requires from quality suggestions.

The llms.txt format, section by section

A file that follows the proposal contains these sections, in this order. Only the first heading is mandatory.

# Example Analytics

> Example Analytics is a web analytics service for small online stores. This file lists the documentation and policies an agent needs to answer questions about setup, pricing and data handling.

Prices are in US dollars. Features are available on every plan unless a page says otherwise.

## Docs

- [Quick start](https://example.com/docs/quick-start.md): Install the tracking snippet and verify the first event
- [Event reference](https://example.com/docs/events.md): Every built-in event with its parameters
- [API reference](https://example.com/docs/api.md): REST endpoints, authentication and rate limits

## Policies

- [Pricing](https://example.com/pricing.md): Plans, usage limits and what happens when a limit is reached
- [Data retention](https://example.com/legal/data-retention.md): How long raw events and reports are stored

## Optional

- [Changelog](https://example.com/changelog.md): Release notes for every version

llms.txt, robots.txt and sitemap.xml compared

The three files sit next to each other at the root of many sites, but they do different jobs. The proposal itself says llms.txt is meant to coexist with them, not replace them.

According to the proposal, a sitemap is no substitute because it usually lacks LLM-readable versions of pages, does not include helpful external URLs and covers far more content than fits in a context window. For how sitemaps should be built, see our XML sitemap best practices.

What about llms-full.txt?

You will often see a file named llms-full.txt next to llms.txt. The version 2 proposal does not define it. It is a convention used by some documentation platforms. Mintlify, for example, documents an automatically hosted llms-full.txt that combines the entire documentation site into one file, with each page's title, source URL, description and full Markdown content.

A single file can be convenient for loading a whole documentation set at once. In our view it also works against the proposal's own design, in which the index stays small enough to fit in context and detail is fetched only when needed. For a typical company website, a full-content dump rarely adds anything the pages do not already offer.

Who publishes llms.txt, and who documents reading it

Publishing a file and having systems read it are different questions. Most confusion about llms.txt comes from treating the first as evidence of the second.

Who publishes one

The proposal says thousands of sites publish an llms.txt file and lists platforms that generate one automatically, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix. On October 31, 2026, we requested three files from AI companies' developer documentation: developers.openai.com/llms.txt, ai.google.dev/gemini-api/docs/llms.txt and docs.claude.com/llms.txt. All three answered HTTP 200 with a Markdown H1 (# OpenAI Developers, # Gemini API and # Anthropic Developer Documentation). The crawler documentation pages of OpenAI, Perplexity and Meta also point readers to an llms.txt index of their own docs.

That shows the format is used for developer documentation. It is not a statement that these companies' crawlers or assistants read llms.txt files on other sites.

Who documents reading it (as of October 2026)

Does your site need one?

Because no major search or AI product documents using the file, the decision is a cost and benefit judgment. This is Backlnk's editorial view, based on the documentation above.

Whatever you decide, the basics stay the same. Google documents that a page must be indexed and eligible to show a snippet before it can appear as a link in AI Overviews or AI Mode, and an llms.txt file does not change that. If an important page is missing from search, start with why a page is not indexed.

How to write and validate an llms.txt file

1. Choose the scope. Use /llms.txt for the whole site, or a path such as /docs/llms.txt if only one section is meant for agents.

2. Write the H1 and the summary. Put the site or product name in the H1, then one or two factual sentences in a blockquote that tell an agent what the site is and what the file covers.

3. Add details only if they help. Short paragraphs or lists can explain how to read the linked files. Do not use headings in this part.

4. Pick the links. Choose pages that answer real questions, such as documentation, pricing and policies. Link Markdown versions where they exist and never invent .md URLs.

5. Write factual notes. Follow each link with a colon and a short description of what the page covers. The proposal asks for concise, clear language, so leave out marketing claims.

6. Serve it correctly. The URL should return HTTP 200 with the Markdown text, not an HTML page from a catch-all route.

7. Validate and test. Check the structure and every link, then follow the proposal's advice: give an agent only your llms.txt and ask it questions about your content.

8. Maintain it. Update or regenerate the file whenever linked pages move or change.

• One H1 with the site or project name, before anything else
• A blockquote summary directly after the H1
• Every list item starts with a [name](url) link
• Every link answers HTTP 200 without redirecting
• No login, checkout or private URLs
• Notes describe content instead of making claims

Check it with Backlnk's llms.txt Generator & Validator

The llms.txt tool has two modes. Validate fetches /llms.txt from your site, or checks text you paste before publishing, and reports format errors against the proposal separately from quality observations. It flags HTML served instead of Markdown, a missing or repeated H1, sections out of order and list items without proper link syntax, then checks up to 60 listed links for errors and redirects. Quality notes, such as links without descriptions, promotional wording or a very large file, are labeled as Backlnk heuristics. Every report also states that a valid file says nothing about which tools use it.

Generate reads your homepage, robots.txt and sitemap, then up to 25 pages, and drafts a file using only pages that returned HTTP 200 during the run. Descriptions come from each page's own title, meta description or first paragraph, it never creates .md URLs that do not exist, and nothing is published to your site.

As a reference, we validated the proposal's own file, llmstxt.org/llms.txt, on October 31, 2026. It returned HTTP 200 as text/plain, 637 characters on 10 lines, with one H1 ("llms.txt"), a blockquote summary and one H2 section, "Docs", listing three links. All three answered HTTP 200, all pointed to Markdown files and every link had notes, so the report showed no format errors. A small, focused file like this is exactly what the proposal describes.

The free full website audit runs every Backlnk tool, including llms.txt validation and a draft, and sends the results as one PDF report by email.

Next step

Decide whether your site has content that agents genuinely need, such as documentation, pricing or policies. If it does, draft a file with the llms.txt Generator & Validator, edit the descriptions by hand and validate it before you upload it. If it does not, spend the time on crawlability and clear pages, which Google documents as the foundation for its AI features.