What is llms.txt? How it works and examples

You might be seeing llms.txt pop up more lately. It’s a new standard that’s quickly gaining traction that makes web content easier for large language models (LLMs) to understand. The idea was first proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI, as a way to structure content in plain text for AI tools. Adoption remained niche until November, when Mintlify rolled out support for /llms.txt across all docs sites it hosts. Practically overnight, thousands of docs sites—including Anthropic and Cursor—began supporting llms.txt. Now, with more LLMs reading the web and users expecting better AI-generated answers, llms.txt is becoming a simple but essential way to control how your content shows up in these AI tools. Let’s break down what it is, why it matters, and how to implement it. What is llms.txt? llms.txt is a plain Markdown file served at /llms.txt on your website. It provides a lightweight summary of your most important content, structured in a way that’s easy for LLMs to read—without the clutter of HTML, Javascript, or advertisements. Unlike robots.txt (which tells crawlers what to avoid) or sitemap.xml (which lists URLs without context), llms.txt is meant to expose content with structure and meaning. Here’s what an llms.txt file can include: An H1 title (this is the only required section for the file) A short summary of the site, usually in a blockquote Details about how the site is structured or how to interpret provided files H2 sections, each with Markdown-formatted lists of important links An“Optional” section to flag lower-priority resources that can be skipped if needed An example llms.txt file: # Example Product Docs > Learn how to get started, use the API, and explore tutorials. ## Guides - [Getting Started](): Intro guide - [Install](): Setup steps ## Reference - [API](): Endpoint list and usage Aside from /llms.txt serving as the outline of your site, the standard also proposes providing more web content in markdown in general. llms-full.txt: Compile all of your site’s text into a single markdown file, to make it easier to paste a single URL to load context into an AI tool. This file structure was developed by Mintlify in collaboration with customer Anthropic, and subsequently included as part of the official llms.txt proposal. .md: Provide a markdown version of a page by appending .md to the original URL. Why should you incorporate llms.txt? As AI-generated answers become a more common way for people to discover, evaluate, and interact with products, making your content LLM-friendly is no longer optional. Parsing unstructured HTML is slow and error-prone for models. If your content isn’t easy to understand, the results in tools like AI chat interfaces, coding assistants, or embedded help widgets will be lower quality. llms.txt helps: Surface key content quickly: Guides, references, tutorials—whatever matters most. Reduce model error: Structured links are easier to parse and reason about. Improve user experience: When someone asks an AI “How do I get started with X?”—you want the answer to come from you. LLM traffic is still early, but it’s projected to jump from 0.25% of search in 2024 to 10% by the end of 2025. Adding llms.txt now ensures you're in control of how your content shows up as AI continues to reshape how people explore the web. llms.txt example use cases Teams are already putting llms.txt into practice to make their content easier for AI tools to understand and use. You can browse hundreds of live examples at llmstxt.site and directory.llmstxt.cloud. Here are top examples for llms.txt: Documentation: Companies like Pinecone and Windsurf use llms.txt (via Mintlify) for their developer documentation, making their API references, SDKs, and tutorials discoverable and ingestible by AI. This helps AI applications generate more accurate, contextual answers about how to use your product, grounded in your up-to-date documentation. Website structure: Companies like Svelte.dev or Rainbowkit use llms.txt as a file structure for important links in their site. Website messaging: Companies like Wordlift or Tiptap add llms.txt to their marketing site with both link structure and additional context about their messaging, to guide AI applications in how to interpret and position their product. How to get started If you’re using a documentation platform like Mintlify, /llms.txt, /llms-full.txt, and .md versions of your docs are generated automatically. If you're setting it up manually: Create the file in Markdown Start with a # heading (site title) Add a short blockquote with a summary Include sections (##) for key content areas Under each, list important links using standard Markdown bullets and links Optionally include an “Optional” section for lower-priority pages Host it at /llms.txt Place it in the root directory of your site, just like robots.txt Or in a relevant subpath like

Apr 3, 2025 - 03:51

What is llms.txt? How it works and examples

You might be seeing llms.txt pop up more lately. It’s a new standard that’s quickly gaining traction that makes web content easier for large language models (LLMs) to understand.

The idea was first proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI, as a way to structure content in plain text for AI tools. Adoption remained niche until November, when Mintlify rolled out support for /llms.txt across all docs sites it hosts. Practically overnight, thousands of docs sites—including Anthropic and Cursor—began supporting llms.txt.

Now, with more LLMs reading the web and users expecting better AI-generated answers, llms.txt is becoming a simple but essential way to control how your content shows up in these AI tools.

Let’s break down what it is, why it matters, and how to implement it.

What is llms.txt?

llms.txt is a plain Markdown file served at /llms.txt on your website. It provides a lightweight summary of your most important content, structured in a way that’s easy for LLMs to read—without the clutter of HTML, Javascript, or advertisements.

Unlike robots.txt (which tells crawlers what to avoid) or sitemap.xml (which lists URLs without context), llms.txt is meant to expose content with structure and meaning.

Here’s what an llms.txt file can include:

An H1 title (this is the only required section for the file)
A short summary of the site, usually in a blockquote
Details about how the site is structured or how to interpret provided files
H2 sections, each with Markdown-formatted lists of important links
An“Optional” section to flag lower-priority resources that can be skipped if needed

An example llms.txt file:

# Example Product Docs  

> Learn how to get started, use the API, and explore tutorials.  

## Guides  
- [Getting Started](): Intro guide  
- [Install](): Setup steps  

## Reference  
- [API](): Endpoint list and usage

Aside from /llms.txt serving as the outline of your site, the standard also proposes providing more web content in markdown in general.

llms-full.txt: Compile all of your site’s text into a single markdown file, to make it easier to paste a single URL to load context into an AI tool. This file structure was developed by Mintlify in collaboration with customer Anthropic, and subsequently included as part of the official llms.txt proposal.
.md: Provide a markdown version of a page by appending .md to the original URL.

Why should you incorporate llms.txt?

As AI-generated answers become a more common way for people to discover, evaluate, and interact with products, making your content LLM-friendly is no longer optional.

Parsing unstructured HTML is slow and error-prone for models. If your content isn’t easy to understand, the results in tools like AI chat interfaces, coding assistants, or embedded help widgets will be lower quality.

llms.txt helps:

Surface key content quickly: Guides, references, tutorials—whatever matters most.
Reduce model error: Structured links are easier to parse and reason about.
Improve user experience: When someone asks an AI “How do I get started with X?”—you want the answer to come from you.

LLM traffic is still early, but it’s projected to jump from 0.25% of search in 2024 to 10% by the end of 2025.

Adding llms.txt now ensures you're in control of how your content shows up as AI continues to reshape how people explore the web.

llms.txt example use cases

Teams are already putting llms.txt into practice to make their content easier for AI tools to understand and use.

You can browse hundreds of live examples at llmstxt.site and directory.llmstxt.cloud.

Here are top examples for llms.txt:

Documentation: Companies like Pinecone and Windsurf use llms.txt (via Mintlify) for their developer documentation, making their API references, SDKs, and tutorials discoverable and ingestible by AI. This helps AI applications generate more accurate, contextual answers about how to use your product, grounded in your up-to-date documentation.
Website structure: Companies like Svelte.dev or Rainbowkit use llms.txt as a file structure for important links in their site.
Website messaging: Companies like Wordlift or Tiptap add llms.txt to their marketing site with both link structure and additional context about their messaging, to guide AI applications in how to interpret and position their product.

How to get started

If you’re using a documentation platform like Mintlify, /llms.txt, /llms-full.txt, and .md versions of your docs are generated automatically.

If you're setting it up manually:

Create the file in Markdown
Start with a # heading (site title)
Add a short blockquote with a summary
Include sections (##) for key content areas
Under each, list important links using standard Markdown bullets and links
Optionally include an “Optional” section for lower-priority pages
Host it at /llms.txt
Place it in the root directory of your site, just like robots.txt
Or in a relevant subpath like /docs/llms.txt if needed
Make sure the raw text is accessible at that URL

You also don’t need to start from scratch. Tools like llmstxt.new let you generate a draft just by prepending a URL.

For more details or to follow updates to the spec, visit llmstxt.org.

What’s next

llms.txt is part of a broader push to make web content more accessible by AI tools.

Looking ahead, efforts like Model Context Protocol (MCP) are exploring how LLMs can move from reading content to directly interacting with products. Platforms like Mintlify now generate both llms.txt and MCP servers automatically—turning documentation into an interface layer between your product and AI tools.

If you’re interested in learning how to make sure your documentation is prepared for tomorrow, get in touch.