Files
mintlify__docs/ai/llmstxt.mdx
Ethan Palm 15cf3ba057 llms.txt/llms-full.txt for authed vs partially authed sites (#5992)
* docs: fix llms.txt auth behavior for partially-authenticated sites

Both llms.txt and llms-full.txt are publicly accessible when a
site has any public pages, reflecting public content only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update llmstxt.mdx

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 15:39:28 -07:00

80 lines
4.9 KiB
Plaintext

---
title: "llms.txt"
description: "Automatically generate llms.txt and llms-full.txt files so AI tools like ChatGPT and Claude can index and understand your documentation."
keywords: ["llms.txt", "LLM indexing", "AEO", "GEO", "content discovery"]
boost: 3
---
import { PreviewButton } from "/snippets/previewbutton.jsx"
The [llms.txt file](https://llmstxt.org) is an industry standard that helps LLMs index content more efficiently, similar to how a sitemap helps search engines. AI tools can use this file to understand your documentation structure and find content relevant to user queries.
Mintlify automatically hosts an `llms.txt` file at the root of your project that lists all available pages in your documentation. This file is always up to date and requires zero maintenance. You can optionally add a custom `llms.txt` file to the root of your project.
<Note>
Authentication affects `llms.txt` and `llms-full.txt` differently depending on your site configuration:
- **Fully authenticated sites**: Both files require authentication. AI tools that cannot authenticate cannot access them.
- **Partially authenticated sites**: Both files are publicly accessible, but only list public pages. Pages restricted to user groups are excluded.
For more information on how authentication affects AI features, see [Feature availability](/deploy/authentication-setup#feature-availability).
</Note>
View your `llms.txt` by appending `/llms.txt` to your documentation site's URL. Mintlify also hosts the file at `/.well-known/llms.txt` for compatibility with tools that follow the `.well-known` convention.
<PreviewButton href="https://mintlify.com/docs/llms.txt">Open the llms.txt for this site.</PreviewButton>
Mintlify adds HTTP headers to every page response so AI tools can discover your `llms.txt` files without prior knowledge of their location:
- `Link: </llms.txt>; rel="llms-txt", </llms-full.txt>; rel="llms-full-txt"`: Follows the standard HTTP `Link` header format for resource discovery.
- `X-Llms-Txt: /llms.txt`: A convenience header for tools that check for `llms.txt` support.
## llms.txt structure
An `llms.txt` file is a plain Markdown file that contains:
- **Site title** as an H1 heading.
- **Site description** as a blockquote summary below the title, sourced from the `description` field in your `docs.json` configuration.
- **Structured content sections** with links and a description of each page in your documentation.
- **API specification links** to your OpenAPI and AsyncAPI specs, if your documentation includes them.
The `llms.txt` file lists your pages alphabetically in the order they appear in your repository, starting from the root directory. Page links in the `llms.txt` file include a `.md` extension so AI tools can fetch the Markdown version of each page directly.
Each page's description comes from the `description` field in its frontmatter. Descriptions truncate at 300 characters and the first line break. For API reference pages, the description also includes the specification information from the `openapi` or `api` frontmatter field. Pages without a `description` field appear in the `llms.txt` file without a description.
This structured approach allows LLMs to efficiently process your documentation at a high level and locate relevant content for user queries, improving the accuracy and speed of AI-assisted documentation searches.
```mdx Example llms.txt
# Site title
> A brief description of the documentation site.
## Docs
- [API](https://example.com/docs/api.md): Endpoint list and usage
- [Install](https://example.com/docs/install.md): Setup steps
- [Getting started](https://example.com/docs/start.md): Intro guide
## OpenAPI Specs
- [openapi](https://example.com/docs/openapi.json)
## AsyncAPI Specs
- [asyncapi](https://example.com/docs/asyncapi.yaml)
```
## llms-full.txt
The `llms-full.txt` file combines your entire documentation site into a single file as context for AI tools and LLM indexing.
Mintlify automatically hosts an `llms-full.txt` file at the root of your project. View your `llms-full.txt` by appending `/llms-full.txt` to your documentation site's URL. Mintlify also hosts the file at `/.well-known/llms-full.txt` for compatibility with tools that follow the `.well-known` convention.
<PreviewButton href="https://mintlify.com/docs/llms-full.txt">Open the llms-full.txt for this site.</PreviewButton>
## Custom files
To add a custom `llms.txt` or `llms-full.txt` file, create an `llms.txt` or `llms-full.txt` file at the root of your project. Adding a custom file overrides the automatically generated file of the same name. If you delete a custom file, Mintlify restores the automatically generated file.
Your custom `llms.txt` or `llms-full.txt` file must have a site title as an H1 heading. Other content is optional. See [Format](https://llmstxt.org/#format) in the `llms.txt` specification for more information on optional sections and best practices.