The useful boundary
llms.txt is a publisher-controlled text file proposed as a concise map of important material for language-model use. It can help a site present selected documentation or verified pages in a simple format. It is not a universal indexing standard and it does not prove that a particular assistant fetched, trusted or cited the file.
If the file would not help a human reviewer locate the site's best factual pages, it probably will not become useful merely because it is written for machines.
What belongs in the file
- A clear site or organization name.
- A short factual description consistent with the home and About pages.
- Links to canonical service, methodology, evidence and documentation pages.
- Optional notes that explain the scope of a linked section.
- Only public facts and URLs the publisher intends to maintain.
What it cannot replace
| System | Primary job | Why llms.txt is different |
|---|---|---|
| robots.txt | Controls supported crawler access patterns. | A content map is not an access-control rule. |
| XML sitemap | Lists canonical URLs for search discovery. | A curated summary need not contain every indexable URL. |
| Canonical link | Signals the preferred version of a page. | A directory does not resolve duplicate URLs. |
| Structured data | Describes page entities in a formal vocabulary. | Markdown links do not create an equivalent entity graph. |
| Visible content | Communicates the complete claim to people. | Hidden machine-only claims remain untrustworthy. |
A publication rule
Every factual statement in llms.txt should be supportable on a public canonical page. Do not add confidential client results, invented credentials, private pricing or claims that legal and editorial owners have not approved. The file should point to evidence, not become a shortcut around it.
Maintenance protocol
- Generate the file from the same canonical route inventory used by the site.
- Review it when services, case studies or documentation change.
- Return a plain-text response with a stable public URL.
- Check every linked page for status, canonical and indexability.
- Keep a change note so tests can distinguish content changes from platform variability.
When to deprioritize it
If primary pages return errors, depend on client-side rendering for their main answer, contradict their schema or lack a clear provider identity, fix those issues first. Experimental discoverability files are an enhancement after the underlying website is coherent.
Reference
Where to go next
Use the Nexus AI Visibility Framework to place this topic inside a complete measurement system, or review the AI Visibility Audit scope.
13 Jul 2026 — Expanded with an independent structure, examples, implementation guidance and primary references.