Sitemap Checker

What this checker tests, in two passes

A sitemap can fail in two completely different ways, and most tools only look for one of them. The file itself can be broken: wrong namespace, relative URLs, a lastmod format Google refuses to read, a size past the protocol limits. Or the file can be flawless and the URLs inside it can be the problem: pages that were deleted last quarter, entries that redirect twice before landing, pages carrying a noindex tag the sitemap contradicts.

The first pass here is the protocol check. The file is fetched, decompressed if it is gzipped, and parsed. The checker then works through the sitemaps.org 0.9 rules: the root element and namespace, the XML declaration, absolute URLs on the same host, escaping, duplicates, W3C datetime lastmod values, changefreq keywords and priority ranges, plus the 50,000 URL and 50MB per-file limits. Errors here are fatal in a specific way: a sitemap that fails to parse is not partially read, it is discarded entirely.

The second pass is the live check. The checker expands the sitemap, following up to 20 child sitemaps if you gave it an index, and fetches up to 100 of the URLs it declares. Each one is classified:

• OK. Returns 200 with no noindex tag. This is what every entry should be.
• Broken. Returns a 404, 410 or a server error. The strongest staleness signal a sitemap can carry.
• Redirect. Returns a 30x. The entry is out of date and should list the destination instead.
• Noindex. Returns 200 but a meta tag or header forbids indexing, contradicting the sitemap.
• Timeout. No response within 8 seconds. Slow pages get crawled less often.

Both passes matter because they catch different failures. A valid file full of 404s sails through every validator ever built and still teaches Google to ignore your sitemap. A broken file pointing at perfect pages never gets those pages discovered at all. One check without the other answers half the question.

How the health score works

The score is deliberately simple, and this is the actual formula rather than a marketing abstraction. The file check starts at 100: any file-level error costs 25 points, and each warning costs 5, with a floor at zero. The URL check is the percentage of sampled URLs that come back OK. The final score combines the two, weighted toward the URLs:

The URL side carries more weight on purpose. Protocol warnings are worth fixing, but Google tolerates an imperfect file far better than it tolerates being sent to dead pages. A sitemap with a formatting warning and 100 clean URLs is doing its job. A pristine file where a third of the entries are broken is not, and the weighting makes the score say so.

Read the bands like this: 90 and above is healthy, and anything you fix from here is polish. 70 to 89 means the sitemap works but is carrying problems, most often redirected entries left behind by a migration. Below 70 means enough of the sample failed that Google is meaningfully wasting crawls on this file, and the fix list below is worth working through this week rather than someday.

One honest caveat: the URL pass is a sample. It checks the first 100 entries, so a 50,000 URL sitemap is judged on its opening slice. That makes the score a fast signal rather than a census. When the sample turns up problems, the sitemap crawler gives you the full URL-by-URL table to work from.

How to fix a failing sitemap

Work top down: file errors first, then the URL problems in order of severity. The step most people skip is the fifth one, and it is the one that stops the same problems reappearing next month.

1. 01

   Fix the file errors first

   Anything the file check marks as an error either stops the sitemap parsing or gets entries silently dropped. A file Google cannot parse is discarded whole, so no URL fix matters until the file itself is valid.

2. 02

   Remove or replace the broken URLs

   Every entry that returns a 404 or a 5xx either needs the page restored or the entry removed. Dead entries are the strongest staleness signal a sitemap can send, and they drag the whole file's crawl priority down.

3. 03

   Swap redirecting entries for their destinations

   A sitemap should list final URLs only. For each entry that redirects, replace it with the URL it lands on. Keep the redirect itself in place for old links, but stop declaring the old address as current.

4. 04

   Take noindex pages out of the sitemap

   A sitemap says index this and a noindex tag says do not. Decide which instruction is right for each page, then make the sitemap and the meta tag agree. Usually that means removing the entry.

5. 05

   Fix the generator, not the file

   Almost every sitemap is generated by a CMS, a plugin or a build step. Hand-editing the XML lasts until the next deploy. Change the generator's rules so excluded pages, redirects and deleted URLs never enter the file.

6. 06

   Re-run the check and resubmit

   Run the checker again and confirm the score has moved. Then resubmit the sitemap in Search Console so Google refetches it promptly rather than on its own schedule.

Checker, validator, or crawler: which one you need

CrawlRaven has four sitemap testing tools because the question "is my sitemap working" hides four different questions. This checker is the one to run first: it answers the overall question in a single pass. The others go deeper on one axis each.

In practice the flow runs left to right. Start here for the score. If the file pass fails, the sitemap validator names the exact rule the XML breaks. If the URL pass fails, the sitemap crawler turns the sample into the full report. And if you are not sure what to test in the first place, the sitemap finder locates every sitemap a domain publishes, including the ones robots.txt forgot to mention.

What this tool checks

Frequently asked questions