XML Sitemap Generator
Enter any URL. The crawler follows your internal links, collects up to 100 pages, and builds a sitemap.xml you can download and submit straight to Search Console.
What this sitemap generator actually does
It starts at your homepage and follows internal links, breadth first, the same way a search engine crawler does. Every page it reaches is recorded with its crawl depth and its Last-Modified header, then written into a sitemap.xml that follows the sitemaps.org protocol.
- Same-host only. Links to other domains are ignored, because a sitemap may only list URLs on the host it is served from.
- robots.txt aware. Disallowed paths are skipped and counted, so you never ship a sitemap that contradicts your own crawl rules.
- Fragments stripped. A link to /pricing#faq is the same page as /pricing, so it appears once.
- Assets excluded. Images, stylesheets, scripts, PDFs, and feeds are filtered out by extension.
- Depth-based priority. The homepage gets 1.0, and each level down drops by 0.2 to a floor of 0.1.
- Bounded and honest. The crawl stops at 100 pages or 45 seconds, and the result reports how many URLs it saw versus how many it crawled.
What it does not do
This is a one-shot crawler running inside a web request, so it has real limits worth knowing before you rely on the output.
- It does not execute JavaScript, so links rendered only by a client-side router are invisible to it.
- It does not find orphan pages. A page nothing links to cannot be reached by following links.
- It does not paginate past 100 URLs, so large sites get a partial file rather than a complete one.
- It does not stay up to date. The file is a snapshot of the moment you ran it.
Worth knowing
If your CMS already generates a sitemap, use this tool to audit it rather than to replace it. Compare what the crawler finds against what your sitemap declares, and the gap between the two is usually where the interesting problems live.
How to generate and submit a sitemap
- 01
Run the crawl
Paste your homepage URL and press Generate Sitemap. Leave the protocol off if you like, the tool assumes https.
- 02
Choose your fields
Keep lastmod on. Switch changefreq and priority off if you only care about Google, which ignores both.
- 03
Check the URL list
Scan the table under the XML. If pages are missing, they are probably reachable only through JavaScript navigation or not linked at all.
- 04
Download the file
Save the XML and upload it to your web root so it resolves at yoursite.com/sitemap.xml.
- 05
Declare it in robots.txt
Add a line reading Sitemap: https://yoursite.com/sitemap.xml. Crawlers other than Google depend on this to find it.
- 06
Submit it in Search Console
Open Indexing, then Sitemaps, and enter the path. Google reports parse errors and indexed counts there within a few days.
The mistakes that make a sitemap worthless
A sitemap is a trust signal as much as a discovery mechanism. Google decides how seriously to take yours based on whether the URLs inside it hold up, and a file full of contradictions gets crawled less and less often.
- Dead URLs. Entries returning 404 or 410 tell Google the file is stale and unmaintained.
- Redirects. A sitemap should list final destinations only, never URLs that bounce somewhere else.
- noindex pages. Listing a page you have told Google not to index is a direct contradiction.
- Canonicalised pages. If a page names a different URL as canonical, the sitemap entry is not the version that will be indexed.
- Mixed protocols or hosts. A sitemap on https may only list https URLs on the same hostname.
- Invented lastmod dates. A build process that stamps every URL with today loses you the field entirely, because Google learns to ignore it.
Our sitemap crawler checks every URL in an existing sitemap for exactly these problems, and the sitemap validator checks the file itself against the protocol.
Sitemap limits and formats
The sitemaps.org protocol sets two hard ceilings, and hitting either one means splitting your file.
- 50,000 URLs per sitemap file, counted per file rather than per site.
- 50MB uncompressed per file, which almost always binds later than the URL count.
- Gzip is allowed. A sitemap.xml.gz counts against the uncompressed size, not the compressed one.
- Above either limit, split into several sitemaps and list them in a sitemap index file.
- A sitemap index may itself point to 50,000 sitemaps, which is more headroom than any real site needs.
Sitemap index files
Large sites usually ship a sitemap index at /sitemap_index.xml or /sitemap.xml that points at several child sitemaps split by content type, such as posts, pages, and products. WordPress with Yoast does this by default. Our extractor and validator both follow index files down through their children automatically.
Frequently Asked Questions
What is an XML sitemap?
An XML sitemap is a file that lists the pages on your site you want search engines to know about. Each entry holds a URL and, optionally, when it last changed. Google, Bing, and AI crawlers read it to discover pages they might not reach by following links alone.
How many pages does this free sitemap generator crawl?
Up to 100 pages per run, with a 45 second crawl budget. If your site is larger, the tool tells you how many URLs it discovered versus how many it managed to crawl, so you know the sitemap is partial. Sites above 100 pages need a full crawler rather than a browser tool.
Where do I put the sitemap file once I have downloaded it?
Upload it to the root of your site so it resolves at yoursite.com/sitemap.xml. Then add a Sitemap line to your robots.txt pointing at that URL, and submit it in Google Search Console under Indexing, then Sitemaps.
Does the generator respect robots.txt?
Yes. Any path your robots.txt disallows is skipped, and the result tells you how many URLs were skipped for that reason. A page blocked in robots.txt should not be in your sitemap, because that combination sends crawlers two conflicting instructions.
Should I include changefreq and priority?
Google has publicly said it ignores both. They stay in the tool because some other crawlers and CMS integrations still read them, and because the sitemaps.org protocol allows them. If you only care about Google, switch both off and keep lastmod.
How is lastmod worked out?
The crawler reads the Last-Modified response header for each page. Where a server does not send one, the entry falls back to today's date. Accurate lastmod values matter: Google stops trusting the field entirely if it sees dates that do not match what actually changed.
Do I need a new sitemap every time I publish?
Yes, unless your CMS generates one automatically. A static file goes stale the moment you add a page. WordPress, Shopify, Webflow, and most modern frameworks generate sitemaps for you, so this tool is most useful for hand-built sites and for auditing what a generated sitemap actually contains.
Is the generated file valid?
It follows the sitemaps.org 0.9 schema: an XML declaration, the correct namespace, absolute URLs, and properly escaped entities. You can confirm it by feeding the downloaded file back through our sitemap validator.
Other sitemap tools
XML Sitemap Validator
Test any XML sitemap against the sitemaps.org protocol. Catches namespace errors, bad lastmod dates, size limits, and cross-domain URLs.
Use toolSitemap Crawler
Crawl every URL your sitemap declares and find the dead links, redirects, noindex pages, and canonical conflicts hiding inside it.
Use toolSitemap URL Extractor
Pull every URL out of a sitemap or sitemap index and download the list as CSV, TXT, or JSON. Handles nested indexes and gzipped files.
Use toolSitemap Finder
Find where any website hides its sitemap. Checks robots.txt directives plus nine common paths, including WordPress and gzipped variants.
Use toolVisual Sitemap Generator
Turn any site into a visual tree of its structure. See how deep your pages sit and which sections carry the most URLs.
Use toolBigger than 100 pages?
CrawlRaven crawls your whole site, then joins the crawl to your Search Console and GA4 data so you can see which sitemap URLs actually earn impressions and which ones never get indexed.