A request claiming to be Googlebot is not sufficient proof that Google sent it. Technical SEO teams should verify the network identity using the provider's documented method before changing a firewall rule or blaming a crawler for an outage.
Separate the questions
A user-agent string answers what the client claims to be. An IP verification procedure tests whether the request came from an appropriate provider network. A successful HTTP response shows what your infrastructure served. None of those facts alone proves that a page was indexed or cited.
Google documents reverse-and-forward DNS checks and published IP ranges for verifying its crawlers and fetchers. Use the method appropriate to the specific request class. Do not assume every request associated with Google belongs to ordinary Googlebot.
Build a useful request record
Capture the UTC timestamp, requested hostname and path, method, source IP as observed by your trusted edge, user-agent, response status and any WAF decision identifier. Use the correct trusted-proxy configuration; a client-supplied forwarding header is not automatically reliable.
For a DNS check, follow the provider's documented hostname validation and confirm that forward lookup returns the original IP. A hostname containing a familiar word somewhere in its text is insufficient. Alternatively, match the address to the appropriate current published range.
Do not publish visitors' IP addresses in a public content report. Share the minimum necessary request evidence through your team's private operations process.
Test access after identity verification
- Inspect the WAF or edge decision for the verified request.
- Compare what the origin and edge returned at the same time.
- Check whether the response was the intended article or a challenge page.
- Test representative public paths after a narrowly scoped rule change.
- Retain an owner, expiry or review date for any exception.
Illustrative example: a verified crawler receives HTTP 200, but the body contains a JavaScript challenge. The status is successful at the transport layer while content access still failed. Fix the access path; do not report that the article was successfully read.
Keep the conclusion narrow
A verified visit establishes a request from that crawler class. It does not establish a recommendation, citation or crawl frequency promise. If you cannot verify identity, record it as unknown rather than automatically allowlisting it.
Read the crawler roles guide before changing policy. Next, review technical findings through AI Readiness, then pair any access finding with verified edge evidence before implementing a firewall exception.
Fact-checked: 17 October 2026. Examples are illustrative; no customer results are reported.