llms.txt: What It Is and Whether It Matters Yet
What is llms.txt? llms.txt is a proposed plain-text file placed at yoursite.com/llms.txt that gives AI systems a curated, markdown-formatted map of your most important content. It was proposed in 2024 as a way to help language models find and read the pages that matter, without parsing your full site.
The pitch sounds sensible. Websites are built for browsers, full of navigation, scripts and clutter that models waste context on. A clean index file solves that. The problem is that a standard only matters if the systems you care about read it, and here the record is unusually clear.
What Google actually said
Google does not support llms.txt. Gary Illyes confirmed in July 2025 that Google does not use the file and is not planning to. John Mueller compared it to the keywords meta tag, which is the least flattering comparison a Google engineer can reach for. When people noticed llms.txt files on some Google-owned properties, Mueller clarified that the Search team neither uses nor endorses them. Google’s 2026 site-owner guidance for AI features goes further and lists llms.txt by name in a mythbusting section of tactics that do not help.
That is about as unambiguous as Google ever gets. For AI Overviews and Gemini visibility, llms.txt does nothing. The work that does matter for those systems is covered in How AI Overviews Choose Which Sites to Cite.
Who ships one anyway, and why
Why do sites still adopt llms.txt? Adoption continues because the file costs almost nothing to create and some AI tools outside Google’s ecosystem do read it, particularly coding assistants and agent frameworks that fetch documentation. For developer-facing sites, it functions as a machine-readable docs index rather than an SEO play.
The numbers tell a split story. SE Ranking’s study of 300,000 domains found a 10.13% adoption rate, roughly one site in ten. Documentation-heavy developer tools adopted it widely because IDE agents actually consume it. Marketing sites adopted it mostly out of fear of missing out.
No published study has shown that having an llms.txt file measurably improves citation odds in ChatGPT, Claude, Gemini or Perplexity answer surfaces. Treat any vendor claiming otherwise as selling something.
My honest recommendation
If your site is documentation or a developer tool: ship one. The cost is an hour, coding agents read it, and your users benefit when their tools answer questions from your docs correctly.
If your site is a normal business or content site: skip it, or ship one only after everything that demonstrably works is done. Clean crawlable HTML, fast pages, definition-first content, accurate schema. That list is the technical SEO guide, and it has a queue of higher-return work on it. A text file cannot compensate for pages a model finds unquotable.
What you should not do is block AI crawlers in robots.txt while shipping an llms.txt to invite them in. I have audited sites doing exactly that. Decide what your AI access policy is, then make robots.txt, your CDN rules and llms.txt agree with each other.
Common questions
Does llms.txt affect Google rankings?
No. Google has stated it does not use the file for Search or AI features. It neither helps nor hurts rankings.
Is llms.txt the same as robots.txt?
No, they are opposites in spirit. robots.txt tells crawlers what not to access and major crawlers respect it. llms.txt suggests what AI systems should read, and adoption by AI companies is partial and informal.
What should go in an llms.txt file if I make one?
A short site description, then a markdown list of your most important pages with one-line summaries each. Keep it under a few hundred lines, link canonical URLs only, and update it when your key pages change.
