Search engine crawler got 5xx from your server
engine_server_errorsShort answer
Bing Webmaster Tools reports that Bing's crawler received 5xx responses from your server on one or more of the last seven days. Glimana's own crawl runs once a day and may have missed that moment, but the search engine saw it. Repeated 5xx responses make crawlers slow down, and a page that keeps returning errors is eventually dropped from the index. Check the server error log for the listed days and fix the cause; usually it is a deploy, a resource limit or a backup window.
Why it matters
Bing's crawler got errors from your server while crawling: our crawl, which runs once a day, may have missed that moment, but the search engine saw it. When a crawler sees 5xx it may drop the page from its index and slows down crawling. The same server problem almost certainly hit Googlebot too, even though Google's crawl stats are not exposed in the same way.
How Glimana detects it
Glimana imports daily crawl statistics from the Bing Webmaster connection. The rule fires when any of the last seven days has a non-zero 5xx count; the evidence lists up to five days with the error count and the number of pages crawled that day, so you can tell an isolated blip (3 errors out of 4,000 pages) from a real outage (800 errors out of 1,000). The rule needs the Bing connection to be active.
How to fix it
- Match the days to events. Deploys, cron jobs, backups, traffic spikes, certificate renewals. The error log (
/var/log/nginx/error.log,php-fpm.log, the host's control panel) for those dates tells you what failed. - Fix the cause. Common ones: PHP-FPM
max_childrenreached (queue timeouts become 502/504), database connection limits, out-of-memory, a plugin or update that fatals on certain URLs. - Check the pages themselves. Bing's crawl errors page lists the exact URLs; if they are a specific set (for example all
/search?q=pages), the fix may be to block or fix that path rather than the server. - Watch the next week. The rule re-evaluates on every crawl with fresh Bing data.
Enable WP_DEBUG_LOG briefly to catch fatal errors, check the host's resource-usage graphs for the listed days, and look for a plugin update on the same date. Rate-limit aggressive bots and raise PHP memory if the log shows "Allowed memory size exhausted".
Shopify's infrastructure rarely returns 5xx. If this fires on a Shopify store, check the status page for the listed days and any custom app proxy routes you host yourself.
Correlate the dates with your deployment history and APM (Sentry, New Relic, Datadog). Add health checks and ensure deploys are zero-downtime (blue/green, php artisan down --secret is still a 503 for crawlers).
How the fix is verified
The task closes automatically when the last seven days of Bing crawl data show no 5xx responses.