glimanaDocs
/
Open Glimana →

Issue guide · AI access

AI bots acting on user requests are blocked

SeverityInfo
CategoryAI access
ScopeWhole site
EffortSmall
Verified byRe-crawl after you mark the task done
Rule keyai_user_bot_blocked

Short answer

This rule no longer fires on its own. Blocks on user-triggered fetchers (ChatGPT-User, Claude-User, Perplexity-User, MistralAI-User) are reported under robots.txt blocks AI search bots, because the fix and the consequence are identical: a bot that cannot fetch a page cannot show it as a source.

Why it matters

User bots fetch a page when a person asks an assistant about it or pastes the URL into a chat. Blocking them means that person gets "I can't access that page" instead of your content. The decision and the fix are the same as for search bots, so they are handled together.

How Glimana detects it

Not evaluated separately. The merged rule evaluates robots.txt for the user-bot user agents along with the search bots.

How to fix it

  1. Find the affected pages in Glimana. Open AI access in the panel. The bot rows marked blocked show the verdict (robots.txt = confirmed, live request = indicator); Show samples lists the URLs that were tested and the status each one returned.

  2. Apply the fix on your platform.

    This rule no longer fires on its own; the fix lives in robots.txt blocks AI search bots, which covers search and user bots together. In short: open robots.txt, find the group that disallows ChatGPT-User, Claude-User, Perplexity-User or MistralAI-User, and remove those user agents from it (or give them an Allow: / group of their own).

    Edit robots.txt in the SEO plugin's file editor and check security plugins or host switches that add AI-bot blocks.

    Edit templates/robots.txt.liquid.

    Edit the file or route and purge /robots.txt at the CDN.

  3. Publish and clear caches. Save the file, then purge /robots.txt (and your sitemap URLs) at the CDN: Cloudflare and most CDNs cache them for up to a day. Confirm with curl -s https://your-domain/robots.txt that the live file shows the change.

  4. Verify in Glimana. Click Re-test on the AI access page, or wait for the weekly test (Sunday 03:00). The merged rule closes when the user bots are allowed again.

How the fix is verified

Through the merged rule.

Sources

robots.txt blocks AI search botsYour robots.txt disallows the bots that fetch pages to cite them in ChatGPT, Claude, Perplexity or Copilot…The server blocks or challenges automated visitorsA blanket bot-protection rule answers non-browser requests with a block or challenge, so AI crawlers probably…The server or WAF blocks AI bots (403 or a challenge page)Requests with an AI bot's user agent get a 403 or a verification page from your firewall. Search bots that…AI bots are served different contentA request with an AI bot user agent gets different content than a normal visitor. Usually a cache, security…