llms.txt: What It Is — and Does It Actually Work?
Every few weeks someone declares llms.txt the future of the web, and someone else declares it dead. We build a validator and a generator for the thing, so people ask us which it is. Here's the honest answer, current as of October 2026.
The spec in 60 seconds
llms.txt is a file at your site root — https://yoursite.com/llms.txt — written in ordinary Markdown. It has a heading with your site's name, a blockquote with a one-line summary, and a list of your key pages:
# Example.com > Invoicing software for freelancers. ## Pages - [Pricing](/pricing): Plans and limits - [Docs](/docs): API reference - [Changelog](/changelog): Weekly releases
The idea: when a model or crawler wants to know what a site is, instead of guessing from navigation menus and meta tags, it reads one curated file. Like robots.txt, but describing content instead of permissions. It is a community-proposed specification, not an official web standard.
The controversy in one paragraph
Search engines never needed sites to describe themselves — that's what ranking algorithms are for. Skeptics argue the same applies to AI engines: models already learn what sites are from training data and retrieval, and a self-declared file can say anything. Supporters counter that a machine-readable summary lowers the cost of understanding small sites that algorithms under-serve. Both are right, which is why adoption is uneven.
Which engines use it today
Honestly: it varies, and it changes. Some AI crawlers fetch it opportunistically; some index it if present; several ignore it entirely, and no major engine has made it a formal, documented dependency. [We re-verify this quarterly — last review Oct 2026.] Anyone who tells you llms.txt definitively gets you cited is overselling. Anyone who tells you it's pointless is overselling too.
When llms.txt is worth it
- Your site is small or new. Less training data exists about you; a curated self-description helps retrieval systems that actually read it.
- Your best pages aren't your homepage. Docs, comparisons, calculators — llms.txt points engines at the pages worth quoting.
- You want a forcing function. Writing "what this site is, in one line, plus the five pages that matter" is a genuinely useful exercise, and this is where it lands.
When it isn't
- Your robots.txt blocks the AI crawlers anyway — fix that first; a description nobody fetches is decoration.
- Your homepage renders nothing without JavaScript — engines reading raw HTML see an empty shell with or without llms.txt.
- You expect it to substitute for being mentionable. llms.txt describes you; it doesn't vouch for you.
Try it on your own site
Our generator builds a valid file in about a minute, and the checker validates any site's file — structure, links, and how it fits with everything else. Or run the full AI visibility check to see all four factors in one report.
llms.txt is one lever of seven. The full playbook is in how to get cited by ChatGPT: 7 fixes that work.
Whether or not engines adopt it, a valid llms.txt costs you ten minutes and forces you to write down what your site is about — that alone is worth it.