What Is llms.txt, and Do You Need One?

What is llms.txt?

llms.txt is a plain-text Markdown file at the root of a website, at /llms.txt, that gives AI tools a short summary of the site and links to its most important pages. Jeremy Howard proposed it in 2024 as an easy entry point for language models and AI agents (llms.txt proposal).

It is a community proposal, not an official standard from a standards body or a search engine.

Do you need an llms.txt file?

You don’t need one, but it is cheap and harmless to add. Google says its AI Overviews and AI Mode don’t require any AI-specific text files (Google Search Central). OpenAI’s and Perplexity’s crawler documentation focuses on robots.txt access rather than llms.txt.

Our honest take: add one if it takes you 30 minutes, keep it accurate, and spend the rest of your effort on crawlable, answer-first content. A great llms.txt file won’t rescue thin pages.

What goes in an llms.txt file?

The proposal defines a simple Markdown structure. Only the first item is required.

1. An H1 with your site or company name (required).
2. A blockquote summary: one or two sentences on who you are and what you do.
3. Optional detail: short paragraphs or lists of key facts, with no headings.
4. H2 sections of links: each a Markdown link to an important page, optionally followed by a colon and a short note.
5. An “Optional” section: secondary links an AI tool can skip if it needs less context.

The proposal also suggests offering Markdown versions of key pages, at the same URL with .md added (llms.txt proposal).

Example

# Example Co

> Example Co makes endpoint security software for mid-size companies.

Key facts:
- Founded in 2019, based in San Jose, California
- Pricing starts at $8 per device per month

## Product

- [Product overview](https://www.example.com/product/): Features and how it works
- [Pricing](https://www.example.com/pricing/): Plans and what each includes

## Docs

- [Getting started](https://www.example.com/docs/start/): Install and setup

## Optional

- [Blog](https://www.example.com/blog/)

How is llms.txt different from robots.txt and sitemap.xml?

robots.txt controls access, sitemap.xml lists every URL, and llms.txt is a short, curated guide for AI tools. They do different jobs, so having one doesn’t replace the others.

If you only fix one, fix robots.txt. Make sure AI search crawlers are allowed.

How do you add llms.txt to your site?

1. Write it. Follow the structure above: name, one-line summary, key facts, and links to your 10 to 30 most important pages.
2. Upload it to your site root, so it loads at yourdomain.com/llms.txt. On WordPress, some SEO plugins can generate it, or upload it by SFTP.
3. Check it loads as plain text in a browser.
4. Keep it current. Update it when you add a service, change pricing, or publish a cornerstone article.

Frequently asked questions

Does llms.txt improve rankings?

There’s no evidence it does. Google says no AI-specific files are needed to appear in AI Overviews.

Can llms.txt block AI crawlers?

No. Use robots.txt to control crawler access. llms.txt only describes your site.

Is there any downside to adding one?

Very little, as long as it stays accurate. An outdated file with old prices or services could confuse the tools that do read it.

What is llms-full.txt?

Some sites publish a second file containing the full text of key pages in one place. It is optional and not part of the core proposal.

---

Want help with the AI crawler setup that actually matters? Our technical SEO service covers robots.txt, llms.txt, and server-rendered content, or start with a free AI visibility audit.

About Silicon Valley SEO

Silicon Valley SEO is a Bay Area consultancy that helps tech companies rank in Google and get cited by AI assistants, using the CITED Method. Questions? Email support@siliconvalleyseo.ai.

Sources

• The /llms.txt file, proposal by Jeremy Howard
• AI features and your website, Google Search Central