Tell AI models what your site is for. Then check they can reach it.

llms.txt is a plain markdown file placed at the root of a domain, at /llms.txt, that gives large language models a curated guide to a site. It contains a title, a one-line summary, and grouped lists of links to the pages the site owner considers most worth reading, each with a short note. The idea, proposed by Jeremy Howard of Answer.AI in 2024, is that a model working from a hand-written map produces better answers than one working from whatever it managed to crawl.

robots.txt is a permission file: it tells crawlers which paths they may and may not fetch, and it has been an agreed convention since 1994. llms.txt is a content file: it grants no permissions and blocks nothing, it simply points a model at the pages you most want understood. The two are complementary, and a site can sensibly have both, since one controls access and the other controls emphasis.

There is no guarantee that any given model does. llms.txt is a community proposal rather than a ratified standard, and no major AI provider has publicly committed to consuming it, so adoption is worth checking rather than assuming. The pragmatic case for publishing one is that it takes about an hour, it cannot harm anything, and it is the only artefact where you get to state in your own words what your site is for.

Upload it so that it resolves at the root of your domain, exactly at https://yourdomain.com/llms.txt, served as plain text. It must not sit in a subfolder, because the whole convention depends on the file being findable at a predictable address. If your site is built with a static generator or a framework, place it in whichever directory is published verbatim, commonly public or static.

llms-full.txt is a companion file that inlines the actual content of your key pages rather than merely linking to them. It suits documentation sites where a model benefits from reading the whole text in one fetch, and it can become very large. Start with llms.txt, since it is short, easy to maintain and describes the same structure.

Short enough that a person would read it, which in practice means a handful of sections and perhaps ten to thirty links. The purpose is curation, so listing every URL on the site recreates the sitemap and defeats the point. Put the pages that explain what you do and how you do it in the main sections, and anything merely nice to have under the Optional heading.