llms.txt

llms.txt is a proposed web standard: a plain Markdown file placed at a website's root (/llms.txt) that gives AI systems a curated guide to the site's most important content. Proposed by Jeremy Howard of Answer.AI in September 2024, it typically contains a short site description followed by annotated links to key pages, often paired with Markdown versions of those pages (or a fuller llms-full.txt) so language models can ingest clean text without parsing navigation, scripts, and boilerplate HTML.

Why it matters

AI systems work under context constraints — they can't read your whole site, and HTML is noisy to parse. llms.txt is an inexpensive way to tell them what matters: which pages define your product, your pricing, your documentation, your best explainers. Adoption grew steadily through 2025–2026, particularly among developer-tool companies, documentation sites, and GEO-conscious marketing teams. The honest caveat: major AI providers have not formally committed to consuming llms.txt the way search engines honor robots.txt, and Google has publicly downplayed it for its systems. It's best understood as a low-cost, low-risk hedge with growing informal usage — some AI tools and agents do fetch it — rather than a proven ranking lever.

How it's used

Marketers implement it in an afternoon: create a Markdown file with an H1 (site name), a blockquote summary, and H2-grouped link lists ("Docs," "Guides," "Pricing") with one-line descriptions per link. Prioritize cornerstone content — the 10–30 pages you'd want an AI to read before describing your company. Many CMSs and frameworks (Mintlify, Docusaurus plugins, WordPress plugins) now generate it automatically. Keep it updated alongside your sitemap, and don't confuse it with robots.txt: robots.txt controls what crawlers may access; llms.txt suggests what AI systems should read.

Related terms

GEO, AEO, context window. See also the GEO content checklist.