INDEXING & CRAWLING · FIELD GUIDE
Sitemap errors: separate fetching from URL quality
Check whether Google can fetch and parse the sitemap before investigating individual URLs. A successfully submitted sitemap helps discovery; it does not make redirects, excluded pages or low-value duplicates eligible for indexing.
Fetch the submitted URL directly
Use the exact sitemap address shown in Search Console. Record its HTTP status and body. An HTML sign-in page, application error or empty response is not a working XML sitemap, even if it returns 200. Verify that the submitted URL points to the current canonical host.
Validate the XML and sitemap index
Check XML escaping, absolute URL entries and the expected sitemap namespace. For an index, follow the child sitemap links and test them too. Use a real XML parser rather than trusting a page that looks readable in a browser. A generated sitemap must not depend on a signed-in session.
Illustrative fault: a URL with two query parameters is written into XML with an unescaped ampersand. The browser may display a parse error. Fix the serializer or URL source; do not manually patch one production response that will be overwritten on the next build.
Check what the sitemap is recommending
Include public canonical pages that return useful 200 responses and are intended for indexing. Remove redirect sources, known missing pages, account routes and excluded variants. Match internal links and canonical tags to the same preferred URL rather than sending contradictory signals.
- Fetch every listed URL when the site is small.
- Check status, canonical and indexing directives.
- Keep tracking parameters and fragment identifiers out.
- Use lastmod only for a real content modification, not every request.
Investigate an unexpectedly empty sitemap
Check publishing flags, environment variables, content filters and build-time caching. A valid empty XML document is still useless when your public pages should be listed. Compare the actual generated URL list with a reviewed content inventory instead of relying on a successful build alone.
Recheck the live response before resubmitting
After correcting the generator, inspect the deployed sitemap and a few representative pages. Submit the canonical sitemap in the matching Search Console property. Track the reported processing state separately from indexed-page counts; submission is not an indexing guarantee.
For a small SaaS, acceptance can be precise: “Every listed URL returns 200, self-canonicalizes to the same URL and has no noindex. The list excludes login, checkout and dashboard routes.” SEO Orbiter does not replace this sitemap validation step.
Continue the investigation
- Page not indexed by Google: find the blocking stage
- Canonical and noindex: different signals, different checks
- SEO traffic dropped after a migration: trace old to new
Primary references
These references describe Google’s reporting and crawling behavior. The worked examples above are illustrative, not customer results.