🔒 SEO Vault

Shopify Product Pages Blocked by robots.txt and Not Indexing? How to Fix It

Your product or collection pages dropped out of Google and Search Console flags them "Blocked by robots.txt" — almost always after an SEO app or a theme edit quietly changed your robots.txt.liquid. The catch: Shopify keeps no version history of that file and never logs which app touched your crawl rules, so there's no built-in way to see what changed or roll it back.

Step by step

  1. Confirm the block and find the exact rule. Open yourstore.com/robots.txt in a browser and scan for any Disallow line covering /products or /collections. Then run Google Search Console > URL Inspection on a blocked page — it names the precise Disallow rule stopping the crawler. Shopify's default robots.txt never blocks product or collection pages, so any such rule was added by an app or by hand.
  2. Trace what added it. Go to Online Store > Themes > (...) > Edit code and open templates/robots.txt.liquid — it isn't there by default, so its mere existence is a clue. Custom Liquid that appends Disallow rules is the culprit, and SEO, redirect, or 'crawl control' apps often inject it. Match the timing against any app you recently installed or updated.
  3. Remove the bad rule and get back to a clean robots.txt. Edit robots.txt.liquid to delete only the offending Disallow block, or delete the whole template to fall back to Shopify's safe default. Save, then reload yourstore.com/robots.txt and verify your /products and /collections paths are no longer disallowed.
  4. Re-request indexing, then stop it happening again. In Search Console, run URL Inspection > Request Indexing on your top pages and click 'Validate Fix' in the Page Indexing report — recovery takes days, not minutes, so don't panic at the wait. Because Shopify won't warn you the next time, this is where a daily snapshot earns its keep: SEO Vault saves your robots.txt and crawl directives every day, alerts you the moment a Disallow appears (and which app likely added it), and lets you restore the last known-good version in one click.

Source: Google Search Console Help — Page Indexing report ("Blocked by robots.txt"); Shopify Help Center — editing the robots.txt.liquid template

Related fixes

Never lose your SEO to an app again

SEO Vault keeps a daily snapshot of every SEO field on your store — meta titles, descriptions, alt-text, tags, handles (products AND collections) — and emails you the moment something changes in bulk, with the likely app responsible. One click restores yesterday's state, just the SEO fields. Free to monitor and get alerted; $14/mo for 1-click restore.

Protect my SEO — free →