Back to SEO Glossary
AI & GEO

What Is llms.txt?

llms.txt is a proposed web standard (similar to robots.txt) that provides AI crawlers with a curated shortlist of a site's most important pages and their descriptions. It gives LLMs explicit guidance on which content to prioritize when learning about a website.

Why llms.txt matters for SEO

llms.txt has been adopted by companies like Anthropic, Stripe, Zapier, and Cloudflare. However, Google's John Mueller compared it to the keywords meta tag in April 2025, and in May 2026 called its usefulness 'purely speculative for now', noting that none of the AI systems use it. OpenAI hasn't confirmed support. It's a forward-looking signal, not yet a proven ranking factor.

Related terms

Robots.txtllms.txt is conceptually similar to robots.txt but specifically for LLMsAI Crawlerllms.txt is designed to guide AI crawlers to your best contentMCPllms.txt describes your site in a static file; MCP answers an agent's queries live

Learn more

What Google's John Mueller has said about llms.txt and AI search

Check llms.txt on your site

llms.txt GeneratorBuild a valid llms.txt from your own pages: the H1, blockquote summary and H2 link sections the proposal specifies, checked for relative URLs and duplicates.Robots.txt GeneratorBuild a correct robots.txt from presets for WordPress, ecommerce, and AI crawlers. Decide which bots to allow, add your sitemap, copy the file.

While llms.txt is still emerging, the technical SEO fundamentals it builds on are proven. CrawlRaven audits everything AI crawlers need to find and cite your content.

Try CrawlRaven Free