Do you need llms.txt? What actually helps AI find your website
llms.txt is often sold as the key to AI visibility. Here is what it is, what Google and the data say, and what actually helps AI assistants find and quote your website.
For most business websites, no. llms.txt is a proposed file that lists your key pages for AI tools, but Google says its search does not use it, no major AI company has confirmed using it in their answers, and one large study found 97% of llms.txt files received no requests at all. It is harmless and cheap to add, but crawlable pages, clear answers and a fast site do far more.
- llms.txt is a proposed standard (2024), not something search engines or AI assistants are required to read.
- Google says you do not need llms.txt or other AI-specific files to appear in its search or AI features.
- Ahrefs found 97% of llms.txt files got zero requests in May 2026.
- Adding one does no harm; just do not expect it to change your AI visibility.
- Access for crawlers, answer-first pages, freshness and reputation are what actually move the needle.
What is llms.txt?
llms.txt is a plain-text file placed at the root of a website, at yoursite.com/llms.txt, that lists and briefly describes the site's most important pages for large language models (LLMs). It was proposed in 2024 as a voluntary standard, inspired by robots.txt and sitemaps, to give AI tools a short, clean map of a site.
Since then, many SEO tools and plugins have added an "llms.txt generator", and it is often sold as a way to appear in ChatGPT or Google's AI answers.
Do you need llms.txt to appear in AI answers?
For most business websites, no. Three things stand out:
- Google says it is not needed. Its guide to AI features lists llms.txt and other machine-readable AI files under what you do not need to do, and in June 2026 Google added that its search does not use them.
- The files are rarely read. Ahrefs found that 97% of llms.txt files received zero requests in May 2026.
- No major AI company has confirmed using it in the answers it shows users. An Optimizely review of the available studies concluded that AI search crawlers almost never fetch it.
llms.txt vs robots.txt: what is the difference?
| robots.txt | llms.txt | |
|---|---|---|
| Purpose | Tells crawlers what they may and may not access | Suggests which pages matter, for AI tools |
| Status | Long-established, followed by major crawlers | Voluntary proposal from 2024 |
| Effect on AI answers | Real: blocking a crawler removes your pages from its answers | No confirmed effect |
| Worth checking? | Yes, always | Optional |
robots.txt is the file that actually matters for AI visibility. If it blocks OpenAI's search crawler (OAI-SearchBot), PerplexityBot or others, your pages cannot appear in their answers. OpenAI documents its crawlers here.
Is there any reason to add llms.txt?
A few:
- It costs almost nothing and does no harm.
- Developer documentation sites use it so coding assistants can find the right pages.
- It may matter later if AI companies adopt it; adoption by websites keeps growing, even if use by AI tools has not followed yet.
If your platform adds one automatically, keep it accurate. Just do not count it as your AI strategy.
Get a free website audit: speed, SEO, AEO and mobile, with the fixes that matter most.
What actually helps AI find and quote your website?
- Open access. robots.txt and your firewall allow the crawlers you want; key pages load as text, not only after scripts run.
- Indexing. Your site is verified in Google Search Console and Bing Webmaster Tools, with an up-to-date XML sitemap.
- Answer-first pages. Each key page answers one real buyer question in its first lines, under a question-style heading.
- Clear structure. Tables, numbered steps and FAQs that are easy to extract.
- Freshness. Updated facts and visible "Updated" dates on important pages.
- Credibility. Named authors, case studies with real results and consistent mentions of your business on other sites.
- Speed and accessibility. A fast, readable site for people is also an easy site for crawlers.
The full method is in what AEO is and how to get cited by ChatGPT and AI Overviews.
What do we do on the websites we build?
We focus on what has evidence behind it: crawlable, fast pages, answer-first content, structured data for rich results, a clean sitemap and robots.txt that welcomes the AI crawlers our clients want. We can add an llms.txt file too, as a low-cost extra, but we do not sell it as a way to win AI answers.
Questions people ask
What is llms.txt?
A plain-text file at the root of a website (yoursite.com/llms.txt) that lists and describes the site's most important pages for large language models. It was proposed in 2024 as a voluntary standard.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may and may not access, and major crawlers follow it. llms.txt only suggests what to read, and there is no evidence the major AI tools rely on it.
Does Google use llms.txt?
Google says its search, including AI Overviews and AI Mode, does not use llms.txt and that you do not need it to appear there.
Should I remove my llms.txt file?
There is no need. It does no harm. Just keep it accurate if you have one, and spend your effort on the pages themselves.
Sources
- The llms.txt proposal (llmstxt.org)
- Optimizing your website for generative AI features on Google Search (Google Search Central)
- llms.txt adoption rises 8.8x but 97% of files get zero AI requests (PPC Land, Ahrefs data)
- Google officially kills the llms.txt SEO hype (TechWyse, Jun 2026)
- Does llms.txt work? The honest answer, with receipts (Optimizely)
- Overview of OpenAI crawlers (OpenAI)
For 10 years Vlad has led the design and build of websites, apps and internal tools for companies across industries. He writes about what keeps software projects on time and how businesses get found in search and AI answers.
More about us
