What Is llms.txt, and Should Your Site Have One?

llms.txt

A proposed Markdown file placed at a website root that gives language models a concise summary of the site and a curated list of links to its most useful content.

Also known as: LLMs.txt, llms txt file

The proposal site, llmstxt.org, credits Jeremy Howard as author, first published on September 3, 2024, with a second version dated August 10, 2026 [s1]. The idea is simple: websites are built for browsers, full of navigation, scripts and layout, while a model working inside a limited context window benefits from a short, clean index that says what the site is and where the substance lives.

The proposal also suggests that sites offer Markdown versions of individual pages, reachable by appending .md to the page URL, so a tool can fetch clean text instead of HTML [s1]. According to its authors, the file has mainly been useful at inference time, when a tool reads it while answering, rather than for training [s1].

The same Ahrefs study found that 28% of the domains it checked published a valid file, that among files which did get traffic most requests came from bots, and that only about a fifth of those requests came from AI systems, with the bulk coming from SEO auditors and technology profilers [s2]. It also found no AI bot requests for llms.txt on domains where the file did not exist, which suggests AI tools fetch it when directed to rather than discovering it on their own [s2].

Where the file does get used is visible if you read developer documentation. Some documentation sites now open their pages with a note telling AI tools to fetch the site's llms.txt for an index of every page. The Model Context Protocol documentation does this, for example. In that setting, where a coding agent is working through a product's docs to write code, a curated index is genuinely useful.

Google's guidance on generative AI features says you don't need to create new machine readable files, AI text files or markup to appear in AI Overviews or AI Mode [s3], and its optimization guide lists llms.txt among the myths, stating that Google Search doesn't use such files [s4]. For Google, the file neither helps nor hurts.

For other answer engines, the practical evidence points the same way: their crawlers mostly do not request the file. What does affect whether a page is retrieved and quoted is covered in the guides on getting cited by answer engines and on answer engine optimization basics: crawlable HTML, specific and original content, and clear dates.

Oppira publishes one at oppira.com/llms.txt. It describes the three modules, the plans and the feature pages in plain Markdown. It is there because it costs almost nothing to maintain and gives any tool pointed at the site an accurate summary, not because it is expected to change how often the site is cited.

The name invites confusion with robots.txt, but the two do different jobs. robots.txt tells crawlers what they may fetch. llms.txt suggests what is worth reading. If you want to control AI crawler access, that is a robots.txt decision, covered in the guide on whether to allow AI crawlers.

Nor does it replace your XML sitemap, which search engines use for discovery. And nothing in it can make a page more likely to be cited. If a page is not crawlable, not specific or not current, listing it in llms.txt does not change that.