The Complete Guide to llms.txt: Making Your Website AI-Discoverable

What is llms.txt?

The llms.txt file is a proposed standard for providing AI-friendly documentation about your website or product. Functioning similarly to robots.txt for search engine crawlers, llms.txt tells AI language models what your site is about and where to find key information. It uses a simple Markdown format that is easy for both humans and machines to read.

As AI assistants become the primary way people discover and interact with software products, having a well-structured llms.txt file ensures your product is accurately represented in AI-generated responses. Without it, AI models may hallucinate incorrect information, miss key features, or fail to guide users to the right documentation. Major tech companies including Stripe, Vercel, Anthropic, Cloudflare, Cursor, Mintlify, and Supabase have already adopted this standard to improve their AI discoverability.

Benefits

AI-powered search engines and coding assistants are becoming mainstream. When a user asks an AI about your product, the AI needs an authoritative source to answer accurately. An llms.txt file provides this curated context, reducing hallucinations and improving the accuracy of AI recommendations.

While llms.txt does not directly affect traditional search engine rankings, it is a critical investment for AI-era SEO and discoverability. Companies adopting this standard early are positioning themselves to be accurately cited in AI-generated search results and coding assistant suggestions. This helps developers find API docs faster, ensures SaaS founders get accurate feature descriptions, and allows technical writers to make their documentation the authoritative source for AI models.

Use Cases

The llms.txt file is designed for various types of websites and products. It is especially useful for developer tools, SaaS platforms, and documentation-heavy sites. You can use it to help AI coding assistants understand your API docs, SDKs, and code examples. SaaS founders can use it to ensure AI search engines accurately represent their product features and pricing. Technical writers can use it to guide AI models to the correct tutorials and guides.

There are three common patterns for structuring your file. The Catalog Pattern groups links by product or feature area. The Workflow Pattern organizes links around user workflows like setup and troubleshooting. The Index Plus Export Pattern maintains a slim version for quick lookups and a comprehensive version for deep ingestion. You can choose the pattern that best fits your needs.

Pricing

The llms.txt specification is an open standard, and tools to generate compliant files are widely available and free to use. There is no cost associated with creating or using an llms.txt file.

Vibes

Major tech companies including Stripe, Vercel, Anthropic, Cloudflare, Cursor, Mintlify, and Supabase have already adopted this standard. This adoption signals that the industry recognizes the importance of AI-friendly documentation. Users and developers are increasingly relying on AI assistants to discover and use software, making accurate representation in these tools a priority for many businesses.

Additional Information

The llms.txt file should be placed in the root directory of your domain so it is accessible at yourdomain.com/llms.txt. For Next.js projects, place it in the public folder. If you choose to create a comprehensive version, you can name it llms-full.txt and place it in the same root directory. It is recommended to update your file whenever you make major documentation changes, add features, or restructure your site. Some companies automate this by generating the file from their documentation system during the build process.