LLMs.txt is a proposed plain-text, markdown style file placed at a site root (example.com/llms.txt). The aim is to give large-language-model crawlers a short, curated map of priority pages, usage terms, and preferred attribution, similar in spirit to robots.txt and XML sitemaps. The idea gained attention in late 2024 through open-source and SEO circles, including work by Jeremy Howard.
What SE Ranking measured
SE Ranking crawled roughly 300,000 domains and checked two things: whether a domain had an llms.txt file, and how often domains were cited across answers from major LLM products. Their dataset shows:
- Overall adoption: 10.13% of domains had llms.txt, meaning close to nine in ten sites do not use the format.
- Adoption by traffic tier:
- Low traffic (0–100 monthly visits): 9.88%
- Mid traffic (1,001–5,000 visits): 10.54%
- High traffic (100,001+ visits): 8.27%
Uptake is flat across site sizes, with the largest sites slightly less likely to add the file.
Does llms.txt relate to AI citations?
To test impact on citations, SE Ranking ran:
- Spearman correlation tests between “has llms.txt” and citation counts
- An XGBoost regression model predicting citation frequency, plus SHAP feature-importance analysis
Result: no meaningful correlation surfaced in the stats, and the machine-learning model improved when the llms.txt variable was removed. In model terms, llms.txt adds noise rather than signal, so presence of the file does not explain who earns citations at scale.
Why the null result makes sense right now
Public guidance from major platforms lines up with the data.
- Google’s AI Overviews and AI Mode documentation says existing Search signals remain the basis for inclusion, with no extra technical requirements beyond normal indexing and quality practices. No reference to llms.txt appears in those guides.
- OpenAI’s crawler docs focus on robots.txt controls and recommend allowing OAI-SearchBot for crawling and surfacing, with no promise llms.txt drives ranking or citations.
Some server logs show GPTBot or other bots fetching llms.txt on occasion, yet retrieval alone is not enough to shift citation behaviour across hundreds of thousands of sites.
What to do next
- Treat llms.txt as an optional hygiene step, not a growth lever. Adding the file is quick and low risk, but current evidence shows zero short-term upside for citations.
- Focus on factors already tied to AI visibility: strong topical pages, clear entity signals, internal linking, and externally earned mentions and links. These remain part of Google and LLM retrieval pipelines today.
- If adding llms.txt, keep the file simple. Point to a small set of primary pages, include short topic summaries, and avoid dumping full site maps.
Bottom line
Across 300k domains, llms.txt usage sits around 10% and shows no measurable relation to citation frequency. Add one if future-proofing matters to your workflow, but do not sell the file internally as a proven driver of AI traffic.