INDEXING & CRAWLING · FIELD GUIDE
Page not indexed by Google: find the blocking stage
Inspect the exact URL in Search Console, then determine whether Google has discovered it, can crawl it, is allowed to index it and selected it as canonical. A missing search result or a site: query is not a complete indexing diagnosis.
Start with the exact URL and intended outcome
Include the correct protocol, hostname and path. A redirect or alternate canonical may explain why that particular URL is absent while its content is indexed elsewhere. Confirm that the page is meant to be a useful public search entry point before treating exclusion as a fault.
Classify the reported stage
Use the Page indexing report for patterns and URL Inspection for the individual case. Keep the reported reason and crawl time in your notes. An indexed URL is not guaranteed to rank for a particular query, and a crawled URL is not guaranteed to be indexed.
- Unknown or undiscovered: review crawlable internal links and sitemap inclusion.
- Crawl blocked or failed: inspect robots, availability, redirects and access controls.
- Excluded or alternate: inspect noindex, duplication and Google’s selected canonical.
- Crawled but not indexed: check the page’s usefulness and duplication rather than assuming an API error.
Compare the live page with the reported version
Fetch a fresh response and check status, visible content, canonical and indexing directives. If Google’s last crawl predates your deployment, its report may still describe the old page. A browser rendering alone does not prove that the same content is available in the initial HTML or to crawlers.
Illustrative example: an Inspection result from Friday says noindex. You removed it on Monday. First prove Monday’s public response is correct; then wait for or request a new crawl. Do not “fix” the page again merely because Friday’s result remains visible.
If many pages disappeared, investigate the shared layer
Sample important URLs from several sections and compare with unaffected pages. Check recent sitewide directives, a failed template, redirects and host availability. Look at Search Console’s manual-action and security reports when relevant. A lower indexed-page count can also reflect correct duplicate consolidation.
Verify the fix in two stages
First verify your implementation: a useful public page, working internal link, correct response and intentional canonical. Then review Google’s subsequent crawl and indexing decision. Submit important eligible URLs through normal Search Console tools where appropriate; repeated submission cannot guarantee inclusion.
Acceptance record: “URL returns 200 and the intended content; no accidental noindex; linked from the category page; self-canonical is consistent with the sitemap.” Keep Google’s later indexed status as a separate field.
Continue the investigation
- Canonical and noindex: different signals, different checks
- Robots.txt blocking Google: check the exact rule
- Sitemap errors: separate fetching from URL quality
Primary references
These references describe Google’s reporting and crawling behavior. The worked examples above are illustrative, not customer results.