llms.txt: What It Is, What It Isn't, and Whether You Need One

Somewhere in your analytics you spotted a hit on /llms.txt — or more likely, someone in an AI-SEO thread told you that you need one. Five minutes of Googling later you've found a proposal site, a few hundred hot takes, and zero agreement.

Here's the sober version, based on what we see auditing real sites: llms.txt is a cheap, honest signal — not a magic ranking lever, and not a substitute for fixing access problems. Let's separate what it does from what people wish it did.

What llms.txt actually is

llms.txt is a plain markdown file at your site root (so it resolves at yoursite.com/llms.txt). The format proposed at llmstxt.org is simple:

# Your Company

> One paragraph: what you do, who for.

## Products
- [Main product](https://yoursite.com/product): what it does, in five words
- [Pricing](https://yoursite.com/pricing): plans and limits

## Docs
- [Quickstart](https://docs.yoursite.com/quickstart)

An H1 with your name, a blockquote summary, then H2 sections linking to your best pages with one-line descriptions. That's the whole spec. It exists because AI systems often land on a site with limited context and no JavaScript — a curated map beats letting them guess.

Three myths worth killing

Myth 1: "AI engines rank you by it"

No major engine has confirmed using llms.txt as a ranking input, and none is obligated to fetch it at all. Treat any guarantee language in tooling marketing accordingly. It's an orientation aid for systems that look — nothing more, and honestly priced at that.

Myth 2: "It replaces robots.txt"

These files do opposite jobs. robots.txt says "who may enter" and is widely enforced. llms.txt says "here's a map once you're inside." If your robots.txt blocks GPTBot or ClaudeBot, your beautiful llms.txt is a map handed to nobody. Access first, always.

Myth 3: "It compensates for a JavaScript-only site"

The linked pages still have to be readable without JavaScript. If your homepage is a client-rendered shell, an AI system that follows your llms.txt link arrives at another blank wall. We regularly measure sites where a bot gets under 10% of the content a browser gets — no text file fixes that.

So who should bother?

• Yes, do it now if your site is crawlable and server-rendered. Ten minutes, zero risk, and it can only help an AI that's already reading you summarize you correctly.
• Do it after the basics if you have robots blocks, a leftover noindex, or a JS wall. Those are the reasons you're invisible; llms.txt is polish.
• Skip the anxiety if a vendor is selling you an expensive "llms.txt optimizer." This is a markdown file you can write yourself from the template below.

The 10-minute version

1. Write the file — H1 name, blockquote summary, 3–5 sections of your best links with one-line descriptions. Under ~50 links total; curation is the point.
2. Upload it to your site root so yoursite.com/llms.txt resolves.
3. Verify it returns HTTP 200 (curl it, or just open the URL).
4. Add a Sitemap: line to robots.txt while you're in there — same spirit, better support.

Your two-minute self-check

• Open yoursite.com/llms.txt in a browser. 404? You don't have one (that's fine — see above for whether you need one).
• Open yoursite.com/robots.txt. Any Disallow lines naming GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot — or a bare Disallow: /? That's the actual fire.
• View your homepage source (Ctrl+U). Can you find your main headline and product description in the raw HTML? If not, JS wall — that's fire too.

Frequently asked

Does llms.txt improve my AI search rankings?

Not directly. No major engine has confirmed using it as a ranking input, and none is required to read it. Think of it as a cheap, honest orientation aid — useful precisely because almost nobody bothers to lie in it yet.

Is llms.txt a replacement for robots.txt?

No. robots.txt controls access and is enforced; llms.txt is optional information with no enforcement. Blocked bots never get far enough to read your map.

How long does it take to add?

About ten minutes including verification: write the markdown, upload to your root, confirm the URL returns 200.