1. Core Site Information Real-time update
2. Key Resource Links
Provide clean Markdown links with concise 1-sentence explanations. LLMs use these URLs to fetch deep context.
Deploy this file to the root of your domain: https://yourdomain.com/llms.txt
Why Do AI Search Engines Need /llms.txt?
Standard websites are full of navigation menus, promotional banners, CSS styles, and tracking scripts. When an LLM (like ChatGPT Search, Claude, or Perplexity) browses the web to answer a user question, parsing complex HTML wastes context window tokens and increases the risk of hallucinations.
The /llms.txt standard proposes a clean, structured Markdown index that acts as a curated table of contents. It gives AI crawlers the exact high-signal summaries and direct links needed to cite your brand accurately.
Concise Overview Index
Contains your H1 project name, single-paragraph summary blockquote, and curated links to essential documentation, pricing, and guides with one-sentence descriptions.
Complete Knowledge Base
Contains all markdown documentation concatenated into a single cohesive document, allowing LLMs with large context windows (200k+ tokens) to ingest your entire product specs in one request.
How to Deploy llms.txt to Your Stack
Next.js (App Router)
Place the generated file in your /public/llms.txt folder. Next.js automatically serves files from public/ at the root path.
Cloudflare Pages & Workers
Put llms.txt in your build output directory (dist/ or build/), or route it via a custom Worker with Content-Type: text/plain; charset=utf-8.
WordPress / Shopify / Webflow
Upload to your root media/assets directory or configure a 301/302 rewrite rule in your CDN (e.g. Cloudflare Page Rules: /llms.txt -> your-hosted-file.txt).
Step 2: Verify If AI Crawlers Can Access Your Site
Creating an llms.txt is only half the battle. If your robots.txt blocks GPTBot or your JavaScript framework prevents text extraction, AI models still won't cite you.