The Purpose of /llms.txt
The proposed /llms.txt standard provides AI models with a token-efficient plain text Markdown summary of a web property. Instead of forcing a search spider to scrape 50 KB of JavaScript and CSS (wasting 10,000+ LLM tokens), a clean manifest delivers verified brand facts and pricing tiers in under 500 tokens.
1. What is the SPA Soft-404 Trap?
In modern frameworks (Next.js App Router, Remix, Vite), unconfigured routes are intercepted by client-side routers. Instead of returning an HTTP 404, the server returns HTTP 200 OK with an HTML page root.
Content-Type: text/html; charset=utf-8
<!DOCTYPE html><html><head><script src="/main.js"></script>...
When AI parsers encounter HTML tags and scripts instead of clean Markdown, the parsing pipeline aborts with a format error, discarding your brand facts.
2. How to Serve a Pure Markdown /llms.txt
Step 1: Place the File in /public/llms.txt
Ensure the file is served statically from your root public assets directory:
> Deterministic AEO and AI visibility audit platform.
## Core Tools
- [Crawler Checker](https://citedo.dev/tools/crawlability): Test 12 AI bots.
- [GEO Score](https://citedo.dev/tools/geo-score): 28-point score.
Step 2: Enforce Clean Content-Type in Next.js
Configure next.config.mjs to explicitly send text/markdown:
export default {
async headers() {
return [{
source: '/llms.txt',
headers: [{ key: 'Content-Type', value: 'text/markdown; charset=utf-8' }]
}];
}
};