How do I know if GPTBot is blocked?

Run a live probe with GPTBot user-agent or use CiteDo's free GPTBot Checker. If you get 403 or HTML byte ratio <0.2 vs browser, you're blocking.

Does allowing GPTBot affect SEO or train OpenAI on my content?

Allowing GPTBot for search (OAI-SearchBot/ChatGPT-User) is for retrieval, not training. Block CCBot/Bytespider separately if you want to block training.

Where is robots.txt on Shopify / Next.js / WordPress?

Shopify: templates/robots.txt.liquid. Next.js: public/robots.txt or app/robots.ts. WordPress: Yoast/RankMath → Tools → File editor or public_html/robots.txt.

Crawlability · 5 min fix

GPTBot Blocked? How to Allow AI Crawlers in robots.txt (2026 Fix)

Q: Does allowing GPTBot affect SEO or train OpenAI on my content?

Allowing GPTBot for search (OAI-SearchBot/ChatGPT-User) is for retrieval, not training. Block CCBot/Bytespider separately if you want to block training.

Q: Where is robots.txt on Shopify / Next.js / WordPress?

Shopify: templates/robots.txt.liquid. Next.js: public/robots.txt or app/robots.ts. WordPress: Yoast/RankMath → Tools → File editor or public_html/robots.txt.

Your site is online for browsers but returns 403 for GPTBot. ChatGPT then never cites you — not because your content is bad, but because one line in robots.txt locks the door. Evidence this week: a crawl study found 97% of llms.txt files get zero AI-crawler requests — because the crawler never gets past robots.txt.

→ Free live check (30s)

Test now: GPTBot Access Checker — simulates GPTBot, OAI-SearchBot, PerplexityBot and shows byte ratio vs browser.

Why GPTBot gets blocked (3 causes)

robots.txt explicit Disallow — Default WordPress/Shopify templates or Cloudflare “Bot Fight Mode” add User-agent: GPTBot Disallow: /.
WAF / Bot protection — Cloudflare, Akamai or Wordfence returns 403/challenge to AI UAs, byte ratio <0.2.
JS wall — SPA that returns empty shell to bots (see blank wall test).

How to check in 30 seconds

# curl with GPTBot UA; compare byte size vs browser

curl -A "GPTBot/1.0" -I https://yoursite.com/

curl -A "Mozilla/5.0" -I https://yoursite.com/

# CiteDo does this automatically and shows WAF-untrusted badge when blocked

Or use 12-bot Crawlability Audit for all 12 AI bots at once.

The 5-minute fix (allow search, block training)

Allow AI search crawlers for citations, keep training scrapers blocked if you wish:

# Allow AI search (for citations)
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Google-Extended
Allow: /

# Block training scrapers (optional)
User-agent: CCBot
Disallow: /

User-agent: Bytespider
Disallow: /

Shopify (Liquid)

Create templates/robots.txt.liquid:

# In templates/robots.txt.liquid, add after default groups:
# CiteDo fix
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /

Next.js (App Router)

// app/robots.ts
export default function robots(){ return { rules:[{userAgent:'GPTBot',allow:'/'},{userAgent:'PerplexityBot',allow:'/'},{userAgent:'CCBot',disallow:'/'}], sitemap:'https://yoursite.com/sitemap.xml'}}

WordPress (Yoast/RankMath)

SEO → Tools → File editor → robots.txt → paste the block above. If using Cloudflare, turn off Bot Fight Mode or add Skip rule for GPTBot.

How to verify

Re-run curl -A GPTBot — expect 200 and byte ratio ~1.0
Re-run GPTBot Checker — should show Allowed ✓
Request indexing in GSC (URL Inspection) — GSC uses Googlebot, but AI crawlers follow the same Allow

What to do next

Allowing the crawler is step 1. Step 2 is giving it something to cite: a clean llms.txt + GEO audit. Or run the full 30s free check and get a human-reviewed action list.

Hotspot refs: Reddit 97% zero-request study · B2B SaaS AI findability list · GSC citedo.dev 0→3 clicks opportunity. Generated via content_brief.py --kw "gptbot blocked".