XML Sitemap Generator

What this sitemap generator actually does

It starts at your homepage and follows internal links, breadth first, the same way a search engine crawler does. Every page it reaches is recorded with its crawl depth and its Last-Modified header, then written into a sitemap.xml that follows the sitemaps.org protocol.

• Same-host only. Links to other domains are ignored, because a sitemap may only list URLs on the host it is served from.
• robots.txt aware. Disallowed paths are skipped and counted, so you never ship a sitemap that contradicts your own crawl rules.
• Fragments stripped. A link to /pricing#faq is the same page as /pricing, so it appears once.
• Assets excluded. Images, stylesheets, scripts, PDFs, and feeds are filtered out by extension.
• Depth-based priority. The homepage gets 1.0, and each level down drops by 0.2 to a floor of 0.1.
• Bounded and honest. The crawl stops at 100 pages or 45 seconds, and the result reports how many URLs it saw versus how many it crawled.

What it does not do

This is a one-shot crawler running inside a web request, so it has real limits worth knowing before you rely on the output.

• It does not execute JavaScript, so links rendered only by a client-side router are invisible to it.
• It does not find orphan pages. A page nothing links to cannot be reached by following links.
• It does not paginate past 100 URLs, so large sites get a partial file rather than a complete one.
• It does not stay up to date. The file is a snapshot of the moment you ran it.

How to generate and submit a sitemap

1. 01

   Run the crawl

   Paste your homepage URL and press Generate Sitemap. Leave the protocol off if you like, the tool assumes https.

2. 02

   Choose your fields

   Keep lastmod on. Switch changefreq and priority off if you only care about Google, which ignores both.

3. 03

   Check the URL list

   Scan the table under the XML. If pages are missing, they are probably reachable only through JavaScript navigation or not linked at all.

4. 04

   Download the file

   Save the XML and upload it to your web root so it resolves at yoursite.com/sitemap.xml.

5. 05

   Declare it in robots.txt

   Add a line reading Sitemap: https://yoursite.com/sitemap.xml. Crawlers other than Google depend on this to find it.

6. 06

   Submit it in Search Console

   Open Indexing, then Sitemaps, and enter the path. Google reports parse errors and indexed counts there within a few days.

The mistakes that make a sitemap worthless

A sitemap is a trust signal as much as a discovery mechanism. Google decides how seriously to take yours based on whether the URLs inside it hold up, and a file full of contradictions gets crawled less and less often.

• Dead URLs. Entries returning 404 or 410 tell Google the file is stale and unmaintained.
• Redirects. A sitemap should list final destinations only, never URLs that bounce somewhere else.
• noindex pages. Listing a page you have told Google not to index is a direct contradiction.
• Canonicalised pages. If a page names a different URL as canonical, the sitemap entry is not the version that will be indexed.
• Mixed protocols or hosts. A sitemap on https may only list https URLs on the same hostname.
• Invented lastmod dates. A build process that stamps every URL with today loses you the field entirely, because Google learns to ignore it.

Our sitemap crawler checks every URL in an existing sitemap for exactly these problems, and the sitemap validator checks the file itself against the protocol.

Sitemap limits and formats

The sitemaps.org protocol sets two hard ceilings, and hitting either one means splitting your file.

• 50,000 URLs per sitemap file, counted per file rather than per site.
• 50MB uncompressed per file, which almost always binds later than the URL count.
• Gzip is allowed. A sitemap.xml.gz counts against the uncompressed size, not the compressed one.
• Above either limit, split into several sitemaps and list them in a sitemap index file.
• A sitemap index may itself point to 50,000 sitemaps, which is more headroom than any real site needs.

Sitemap index files

Large sites usually ship a sitemap index at /sitemap_index.xml or /sitemap.xml that points at several child sitemaps split by content type, such as posts, pages, and products. WordPress with Yoast does this by default. Our extractor and validator both follow index files down through their children automatically. Splitting a sitemap is one of the first things that changes once a site passes a few thousand URLs, and the rest of what changes is in the enterprise SEO audit guide.

Sitemap fields, and which ones Google reads

The sitemaps.org protocol defines four fields per URL. Google reads two of them and ignores the other two, which it has said publicly. They stay in the tool because other crawlers and CMS integrations still read them.

Frequently asked questions