How to fix it
Allow the "Rankelle-Audit" user agent through your firewall or bot protection, then re-run the crawl. On Cloudflare this is a WAF custom rule that skips bot protection when the user agent contains "Rankelle-Audit".
When it is not a problem
Blocking unknown crawlers is a reasonable default and often deliberate. What matters is that a blocked page is unmeasured rather than healthy, so it is reported as its own state instead of being silently counted as fine.
How Rankelle handles it
- SeverityCritical. It stops a page being read, indexed or reached at all, so it outranks everything cosmetic on the list.
- DetectionDetected on a single page while crawling it, then grouped with the other pages under the same URL pattern.
- GroupingEvery page with this finding under the same URL pattern becomes one cause, with a confidence taken from how much of the pattern is affected. 400 of 400 is a template; 6 of 14 is phrased as a question.
- LandingThe affected URLs stay under cheap watch. When this finding is gone from all of them, the fix is marked landed without anyone reporting the deploy.
- VerdictAt landing, comparable untouched URLs are frozen as a control set. Twenty-eight days later the two are compared, and the fix can come back as having done nothing.
Try it on your own site, free
No card, and the free tier does not expire — the first verdict takes about six weeks to arrive.