Sitemap Index Generator

When you need a sitemap index, and when you do not

The hard rule comes from the sitemaps.org protocol: a single sitemap file may hold at most 50,000 URLs and 50MB uncompressed. Past either limit, search engines stop reading. A site with 80,000 pages that keeps one sitemap file is not warned about the overflow; the entries past the cap are simply ignored, and 30,000 pages lose their discovery signal without anyone noticing. At that scale a sitemap index is not an optimisation, it is the only way to declare the whole site.

Below the limit, an index is still the right structure for any site that splits its sitemaps by section: one child for posts, one for pages, one for products, one for categories. The payoff is the lastmod date on each child. When you publish three blog posts, only sitemap-posts.xml changes, its lastmod moves, and Google can re-fetch that one file instead of re-reading every URL you have. On a site with a large stable catalogue and a fast-moving blog, that difference decides how quickly new content gets found.

The case where you do not need one is just as clear. A site with a few hundred pages and a single sitemap file gains nothing from wrapping that one file in an index. One extra fetch, one extra file to keep in sync, no new information for the crawler. Keep the single sitemap, declare it in robots.txt, and revisit the question when the site grows or the sitemap splits. If you have not built the child files yet, the sitemap generator will crawl your site and produce them.

The rules a sitemap index has to follow

An index file is small and simple, which makes its failure modes easy to memorise. Every rule below comes from the sitemaps.org 0.9 protocol or from how Google documents its own handling, and breaking any of them costs you silently rather than loudly.

How to set up a sitemap index

Seven steps, and only the first one takes real effort. The order matters: the child files have to exist at their final URLs before the index that references them goes live, or Google fetches an index full of 404s.

1. 01

   Split your URLs into child sitemaps

   Divide your URLs by section, such as posts, pages and products, or into blocks of up to 50,000. If you do not have the files yet, the sitemap generator can build them for you.

2. 02

   Upload the child sitemap files

   Put each child sitemap on your site, on the same host the index will live on. Note the full URL of every file, because those URLs are what the index lists.

3. 03

   Paste the child URLs here and generate

   One URL per line. The tool validates each line, removes duplicates, adds an optional lastmod date, and produces a valid sitemapindex XML file.

4. 04

   Upload sitemap_index.xml to your root

   The root is the safest home for the index because a sitemap can only reference files at or below its own directory unless it is declared in robots.txt.

5. 05

   Declare the index in robots.txt

   Add a line reading Sitemap: https://yourdomain.com/sitemap_index.xml. Every crawler that reads robots.txt finds the index without you submitting it anywhere.

6. 06

   Submit the index alone in Search Console

   Open Indexing, then Sitemaps, and submit only sitemap_index.xml. Google discovers every child through it and reports per-child results under the same entry.

7. 07

   Validate the result

   Run the finished index through the sitemap validator to confirm the file parses, the namespace is right, and every child URL resolves on the same host.

Once the index is live, run it through the sitemap validator to confirm the file parses and the namespace is exact, or through the sitemap checker to test the file and a sample of the URLs inside it in one pass.

Common sitemap index mistakes

The same handful of mistakes accounts for nearly every broken index. All of them fail silently, which is why they survive for months.

• Listing pages in the index. An index lists sitemap files, not URLs of pages. A page URL wrapped in a sitemap element makes Google fetch an HTML document where it expects XML, and that entry contributes nothing.
• Nesting one index inside another. The protocol allows one level. An index pointing at a second index is rejected, and every sitemap below the nested one goes unread.
• Mixing hosts. Children on www when the index lives on the bare domain, or children on a CDN subdomain, are ignored. Host means the full hostname, so protocol and subdomain both have to match.
• Stamping every child with the same fresh lastmod. If every build writes today's date on every child, the field carries no information, Google learns to distrust it, and you lose the one mechanism that lets a crawler skip the sections that did not change.
• Submitting every child separately as well as the index. Google tolerates it, but the Sitemaps report now counts every URL twice under different rows, and the duplicate entries obscure which submission a problem belongs to. Submit the index alone.

Inheriting an index someone else built? Pull its full contents with the sitemap extractor, which follows every child and lists the URLs inside, so you can see exactly what the index actually declares before you change it.

What this tool does

Frequently asked questions