- llms.txt is a plain Markdown file at yoursite.com/llms.txt that lists a site's key pages with one-line notes, so an AI tool can find the right page fast. It is a 2024 proposal, not a standard.
- Google says Search, including AI Overviews, doesn't use it, and an Ahrefs study found 97% of llms.txt files got zero requests in May 2026.
- Making one takes minutes and costs nothing, but robots.txt, readable pages and complete product data do far more for AI search. Shopify now generates the file for every store and points it to /agents.md.
llms.txt is a plain Markdown file at the root of a website, at yoursite.com/llms.txt, that lists the site's most useful pages with a one-line note on each, so an AI tool can find the right page without reading the whole site. Jeremy Howard of Answer.AI proposed it on September 3, 2024 at llmstxt.org. It is a proposal, not a standard. Google says its Search, including AI Overviews, doesn't use it, and server-log studies show AI crawlers rarely request it. This article covers what goes in the file, our own llms.txt and robots.txt as examples, what the data says about who reads it, and what to do on Shopify.
What goes in an llms.txt file?
The proposal sets out sections in this order:
- An H1 with the site's name. This is the only required part.
- A blockquote with a short summary: the facts a reader needs to make sense of the rest.
- Optional detail in paragraphs or lists, with no headings.
- H2 sections with lists of links, each written as
[name](url), optionally followed by a colon and a note about the page.
The proposal also suggests a clean Markdown copy of each page at the same address with .md added, so a tool can read the text without the page's code. It says the file is meant for use when someone asks an AI a question, not for training models.
What does a real llms.txt look like?
Here is the top of ours, at bloom10x.com/llms.txt, as of November 10, 2026, cut down to a few links per section:
# BLOOM10X > BLOOM10X is a growth agency for ecommerce brands on Shopify doing $1M+ a year. It runs creative, paid ads, conversion and retention as one plan, measured on Shopify sales. ## How we work - [The 10X Engine: growth marketing for Shopify](https://bloom10x.com/10x-engine): The 10X Engine runs creative, paid ads, your Shopify store and email and SMS as one plan, measured on store sales. What each stage does and the proof. - [Shopify SEO and AI search visibility](https://bloom10x.com/services/seo): Store SEO for Shopify brands: collections, product pages, structured data and content, so the store is found in Google and cited in AI answers. ## Proof - [Wholesale Fashion Trends: 84% more purchases](https://bloom10x.com/case-studies/wholesale-fashion-trends): A wholesale and retail fashion brand on Shopify Plus: purchases up 84% on 5% more Meta spend, store sales up 45%. Jun to Sep 2026 vs 2025. ## Free tools and guides - [ROAS calculator with break-even ROAS](https://bloom10x.com/tools/roas-calculator): Free ROAS calculator for Shopify brands. Enter order value, margin, fees and ad spend to get break-even ROAS, cost per purchase and MER.
What each part does:
- The H1 and blockquote say who we are and who we are for in two lines. If an AI tool reads nothing else, it gets the fit right: ecommerce, Shopify, $1M+ a year.
- The H2 groups (How we work, Proof, Company, Free tools and guides, Blog) sort pages by the question they answer, not by the site menu.
- Each line is the page's title and meta description. Our site build writes the file from the same title and description each page already has, so it updates itself when a page is added or reworded and can't drift out of date.
- Proof lines keep their numbers and dates. A summary tool that quotes "84% more purchases" also gets "Jun to Sep 2026 vs 2025".
What is the difference between llms.txt and robots.txt?
| robots.txt | llms.txt | |
|---|---|---|
| Job | Tells crawlers what they may fetch | Points AI tools to the pages worth reading |
| Status | A long-standing web standard | A 2024 proposal |
| Format | Plain-text rules per crawler | Markdown: title, summary, link lists |
| Who reads it | Every major search and AI crawler | Rarely fetched (see the data below) |
| What it changes | Whether a crawler may fetch your pages at all | Nothing is required to happen |
Robots.txt is the one that matters. Here is part of ours:
User-agent: * Allow: / Disallow: /variations/ User-agent: OAI-SearchBot Allow: / Disallow: /variations/ User-agent: ClaudeBot Allow: / Disallow: /variations/ User-agent: Google-Extended Allow: / Disallow: /variations/ Sitemap: https://bloom10x.com/sitemap.xml
The * group covers every crawler. The named groups (ours also lists GPTBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Bingbot) make the choice explicit for each AI company's bots, so a later edit to the * rules can't shut them out by accident. /variations/ holds draft page versions we don't want in any index. We allow the training crawlers too; a brand that wants to opt out of training can block GPTBot, ClaudeBot and Google-Extended and still allow the search crawlers. Our GEO vs SEO guide has a table of what each crawler does.
Does Google use llms.txt?
No. Google's guide to its generative AI features, last updated July 10, 2026, lists llms.txt under "what you don't need to do". It says site owners don't need new machine-readable files, AI text files or Markdown to appear in Google Search or its AI features, because "Google Search itself doesn't use them." Google may crawl the file like any other text file; that doesn't mean it treats it as special.
That covers AI Overviews and AI Mode. It says nothing about ChatGPT, Claude or Perplexity, which run their own crawlers.
Do AI crawlers read llms.txt?
Rarely. The largest study so far is Ahrefs' analysis, published June 15, 2026:
| Finding | Number |
|---|---|
| Domains with traffic in May 2026 | 137,210 |
| Share publishing an llms.txt file | 28%, about 38,000 domains |
| llms.txt files with zero requests that month | 97% |
| Requests from SEO audit tools | 21.7% |
| Requests from AI training crawlers (mostly GPTBot) | 5.3% |
| Requests from AI assistants such as ChatGPT-User | 2.5% |
| Requests from AI search bots such as OAI-SearchBot and PerplexityBot | 1.1% |
Most of the traffic to llms.txt files came from tools checking whether the file exists, not from AI answer engines reading it. Claude Code, a coding assistant, was among the AI agents fetching the files.
That points to where llms.txt is used today: developer documentation, read by coding tools. OpenAI and Perplexity both publish an llms.txt for their own developer docs. Their crawler documentation (OpenAI, Perplexity), like Anthropic's, explains robots.txt controls and does not mention reading llms.txt from other sites.
Should a Shopify store have an llms.txt?
On Shopify the question is mostly settled for you. In May 2026 Shopify began serving /llms.txt for stores itself and now redirects it to a file at /agents.md. A Shopify staff reply in the Shopify developer forum on May 20, 2026 said /agents.md is now the main discovery file for AI and agent experiences, and that the way to customize it is a templates/agents.md.liquid file in your theme. Stores that served their own llms.txt through an app, redirect or proxy report that it no longer shows.
What to do:
- Open yourstore.com/llms.txt and see where it lands and what it says.
- Check the facts in it: store name, what you sell, shipping regions, returns. Wrong facts are worse than none.
- Customize only if something is wrong or missing, through
templates/agents.md.liquid, or ask your developer.
On other platforms, making one takes about ten minutes:
- Write an H1 with your brand name and a one-line blockquote: what you sell, to whom, from where.
- Add H2 groups such as Products, Guides, Policies, About.
- Under each, list your best pages as
[Page title](full URL): one-line description. Reuse the meta descriptions. - Save it as llms.txt in the site's root folder and open yoursite.com/llms.txt to check it loads.
Keep it to pages you want quoted. Leave out cart, account, search and filtered collection URLs.
What matters more than llms.txt for AI search?
The things AI crawlers do fetch: your pages, and your product data.
- Robots.txt and bot settings that let OAI-SearchBot, Claude-SearchBot and PerplexityBot in. Check your CDN or firewall too.
- Pages that work without JavaScript, so product details, reviews and FAQs are in the HTML.
- Answer-first pages with specific, sourced facts on collections, products and guides.
- Complete product data in Shopify, Merchant Center and anywhere else your products are listed.
We set this up for Shopify brands as part of AI search optimization and Shopify SEO.
Frequently asked questions
What is llms.txt in simple terms? A Markdown file at yoursite.com/llms.txt that lists your key pages with short notes, so AI tools can find the right page quickly. It was proposed in September 2024.
Does llms.txt help SEO? Not in Google. Google says Search and its AI features don't use llms.txt, so it is not a ranking or AI Overviews lever.
Is llms.txt the same as robots.txt? No. Robots.txt tells crawlers what they may fetch and is widely followed. llms.txt only suggests pages to read, and most AI crawlers don't request it.
Does ChatGPT read llms.txt? Rarely. In Ahrefs' May 2026 data, AI search bots including OpenAI's OAI-SearchBot made about 1% of requests to llms.txt files, and 97% of files got none.
Does Shopify support llms.txt? Yes. Since May 2026 Shopify serves /llms.txt for every store and redirects it to /agents.md, which you can customize with a theme template.
Michael Bui founded and runs BLOOM10X, a growth agency for ecommerce brands on Shopify. About BLOOM10X
