# llms.txt - Definition, Format and What Actually Reads It > What llms.txt is, the exact format, how it differs from llms-full.txt and llms.jsonl, and the honest answer on which AI systems read it. Source: https://angeo.dev/llms-txt-definition/ Updated: 2026-10-03 Canonical definition # llms.txt - Definition, Format and What Actually Reads It **llms.txt is a Markdown file served at the root of a domain that gives AI systems a curated map of a site's most important content.** It is a proposal by Jeremy Howard, published in September 2024, not a standard from any AI vendor. It tells a model which pages matter and what each one is for - the job robots.txt does for permission and sitemap.xml does for coverage, but for *meaning*. **In one sentence** A plain-Markdown table of contents at `/llms.txt`, written for language models rather than crawlers, listing the pages that best explain what a site is and does. ## What the file contains The proposal defines a small, fixed structure. Everything below the H1 and blockquote is optional, but the order is not free-form: ``` # Site name > One-sentence summary of what this site is. Optional free paragraph of context. ## Section name - [Page title](https://example.com/page): One-line description of the page. ## Optional - [Lower-priority link](https://example.com/x): Skipped when context is short. ``` The `## Optional` section is the only heading with defined semantics: a client under context pressure drops it first. Every other section name is yours to choose. ## llms.txt, llms-full.txt and llms.jsonl | File | Contains | Use when | | `llms.txt` | Links plus one-line descriptions | Always. This is the entry point. | | `llms-full.txt` | The full text of those pages, inlined | The site is small enough to fit a context window. | | `llms.jsonl` | One self-contained JSON object per line - product, category or page | Ecommerce. A catalogue is data, not prose. | For a store, the distinction matters more than it looks. `llms.txt` describes the shop; `llms.jsonl` describes the inventory. A model answering "does this shop sell waterproof jackets in size L" needs the second, and streaming line-delimited JSON is what lets it read 20,000 products without loading a 40 MB array. ## Who reads it - and the honest answer **No AI vendor has committed to reading llms.txt** OpenAI, Anthropic, Google and Perplexity have not published a statement that they consume the file. Anyone claiming a measured ranking effect is overstating what is publicly known. What *is* true: the file costs almost nothing to serve, is trivially fetchable by any agent given the URL, and several developer-facing platforms publish one. The defensible reason to publish it is narrower and still real: it is the only artefact where a merchant states, in their own words, what their store is - instead of leaving a model to infer it from theme markup and faceted-navigation URLs. ## How common is it on Magento? We scanned live Magento 2 storefronts drawn from the Tranco top-200,000 list (list ID `ZJGPG`, frame fixed 8 August 2026). Of the 837 confirmed Magento stores, 762 were reachable in the run of 9 August 2026: | Signal | Stores | Share of 762 | | Serves `llms.txt` | 85 | 11% | | Emits JSON-LD `Product` | 73 | 10% | | Blocks at least one AI crawler | 110 | 14% | Method and limits: a store counts as serving the file only if `/llms.txt` returned HTTP 200 with a non-empty body. 23 stores hit rate limits during the run and are counted as not serving it, so 11% is a floor, not a point estimate. ## Common mistakes ### Publishing it once and never regenerating A catalogue changes; a static file does not. A stale `llms.txt` is worse than none, because it states prices and availability that are wrong with the authority of a first-party document. ### Letting meta-description artefacts leak in Most generators build the one-line descriptions from meta descriptions. Whatever is broken there - a truncated first character, an editor note, a character count - is republished verbatim in the file a model quotes. ### Listing every URL It is a curated map, not a sitemap. A 4,000-line `llms.txt` defeats the purpose, which is to tell a model what to read *first*. ## Generating it on Magento 2 Magento has no native support. [angeo/module-llms-txt](https://angeo.dev/modules/llms-txt/) generates all three files from the catalogue - multi-store, Page Builder aware, atomic writes, MIT-licensed: ``` composer require angeo/module-llms-txt bin/magento module:enable Angeo_LlmsTxt bin/magento setup:upgrade bin/magento angeo:llms:generate bin/magento angeo:llms:validate ``` ## Questions Is llms.txt a standard? No. It is a public proposal from September 2024 with no standards body behind it and no ratified specification. Treat it as a convention that costs little to follow, not as a requirement. Does llms.txt replace robots.txt or sitemap.xml? No - the three answer different questions. robots.txt says who may crawl, sitemap.xml says which URLs exist, llms.txt says which pages explain the site and what each one covers. Serve all three. Will llms.txt make ChatGPT recommend my store? No, and any vendor promising that is overselling. No AI vendor has confirmed it reads the file. What a merchant controls is crawler access, structured data, a conforming product feed and a verifiable profile - llms.txt is a cheap addition to that set, not a substitute for it. Where does the file have to live? At the domain root: `https://example.com/llms.txt`, served as `text/plain` or `text/markdown` with HTTP 200. A subdirectory copy is not discoverable. What is the difference between llms.txt and llms.jsonl? llms.txt is prose links for humans-as-read-by-models; llms.jsonl is one JSON object per line, one per product, built for catalogue ingestion. A store publishing only llms.txt has described its shop but not its inventory. ## Related - [Module: llms.txt generator for Magento 2](https://angeo.dev/modules/llms-txt/) - [What llms.jsonl is and why ecommerce needs it](https://angeo.dev/what-is-llms-jsonl-and-why-ecommerce-needs-it/) - [Magento AEO - definition](https://angeo.dev/magento-aeo-definition/) - [Magento 2 AEO Guide 2026](https://angeo.dev/magento-2-aeo-guide/) Verified 28 August 2026 against the llms.txt proposal and our own scan data. Store figures: 762 reachable Magento 2 storefronts, scan of 9 August 2026, Tranco list ID ZJGPG. Disclosure: we publish the open-source module referenced above. ## More on this topic - [Markdown Mirror - Definition](https://angeo.dev/markdown-mirror-definition/) - [llms.txt v2: What the Spec Says, and What the Evidence Shows](https://angeo.dev/llms-txt-v2-spec-and-evidence/) - [How to generate llms.txt for Magento 2 in 5 minutes](https://angeo.dev/how-to-generate-llms-txt-for-magento-2-in-5-minutes/) - [llms.txt for Magento 2: 6 Modules Compared (2026)](https://angeo.dev/llms-txt-magento-2-free-vs-paid-module-comparison/) ## Sources - [llms.txt proposal](https://llmstxt.org/) - [llms.txt change notes (v2, August 2026)](https://llmstxt.org/changes.html) - [llms.txt repository on GitHub](https://github.com/AnswerDotAI/llms-txt) Links checked 23 September 2026.