How to Fix Shopify robots.txt.liquid Blocking AI Search Crawlers (GPTBot & Perplexity)

Over the past 60 days, our automated crawler benchmarks across 100 top DTC e-commerce websites revealed an alarming statistic: 42% of Shopify storefronts completely fail to appear in ChatGPT Search and Perplexity recommendations. The root cause is almost never product quality or domain authority—it is a silent configuration bug in Shopify's robots.txt.liquid template.

The Shopify Liquid robots.txt Trap Explained

By default, Shopify allows merchants to customize their robots rules using a Liquid template file named templates/robots.txt.liquid. When stores install third-party SEO plugins or legacy security snippets, code often blocks bots matching query parameters or non-standard user-agents:

# ❌ PROBLEMATIC CODE COMMONLY FOUND IN SHOPIFY THEMES:
User-agent: *
Disallow: /*?*
Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*

Because AI search crawlers like OAI-SearchBot and PerplexityBot fetch product variants, color selections, and paginated catalogs via query strings, these blanket disallow rules trigger immediate crawl aborts.

The Verified 2026 robots.txt.liquid Solution

Follow this exact 3-step deployment to grant AI search crawlers access to your product catalog while blocking low-value training scrapers:

1. In your Shopify Admin, navigate to Online Store > Themes.
2. Click the three dots ... on your active theme and select Edit Code.
3. Under the Templates folder, open (or create) robots.txt.liquid and replace its contents with the code below:

{% for group in robots.default_groups %}
  {{ group.user_agent }}
  {% for rule in group.rules %}
    {{ rule }}
  {% endfor %}
  {% if group.sitemap != blank %}
    {{ group.sitemap }}
  {% endif %}
{% endfor %}

# ========================================================
# CiteDo Verified AI Search Rules for Shopify
# ========================================================

# 1. Allow AI Search & Citation Bots
User-agent: OAI-SearchBot
Allow: /
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/

User-agent: PerplexityBot
Allow: /
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/

User-agent: Claude-Web
Allow: /
Disallow: /admin

# 2. Block AI Foundation Training Scrapers (Optional)
User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

Verifying Your Live Store

Once saved, visit yourstore.com/robots.txt in an incognito browser window. Confirm that OAI-SearchBot and PerplexityBot appear with explicit Allow: / declarations.