Google Index Checker
Paste up to 100 URLs and see which ones Google is blocked from indexing. Both places a noindex can hide, canonicals pointing elsewhere and non-200 responses, with the blocked URLs lifted above the healthy ones.
Short answer
This checks whether anything stops Google indexing a URL: the status code, a noindex in the head or in the X-Robots-Tag header, and where the canonical points. The header form never appears in view-source, which is why it survives undetected longest. It does not report what Google has indexed, because no third-party tool can: that lives in Search Console. Most pages that are not indexed are perfectly indexable.
Allowed to be indexed is not the same as indexed
Indexation is two different questions, and nearly every tool calling itself an index checker blurs them. One is answerable from the page itself; the other is only answerable by Google.
| Question | Who can answer it | How |
|---|---|---|
| Is anything stopping Google indexing this? | Anyone | Read the status, the robots directives and the canonical. This tool |
| Has Google indexed this? | Only Google | URL Inspection in Search Console, on a property you have verified |
| How many of my pages are indexed? | Only Google | The Pages report in Search Console, with the reason for every exclusion |
| Roughly how many, for a site I do not own? | Nobody precisely | A site: search, which Google itself calls a rough estimate |
Permission, not a promise
Passing every check here means Google is allowed to index the page. It can still decline, and routinely does, on quality grounds. That is what "Crawled, currently not indexed" means in Search Console, and no tag change fixes it.
The four things that block indexing, and how they differ
Each verdict in the table above means something different about what to do next, and two of them are frequently correct rather than broken.
| Verdict | Is it a fault? | What it means | What to do |
|---|---|---|---|
| noindex in the headers | Almost always | An X-Robots-Tag set by the server or CDN, invisible in view-source | Check the server config. This is the one that hides for months |
| noindex in the head | Usually | A meta robots tag in the markup | Deliberate on a thank-you page; a staging default otherwise |
| Does not return 200 | Yes | A 404, 410, 5xx or an unresolved redirect | Fix it, or drop the URL from whatever list declared it |
| Canonical points elsewhere | It depends | Google is told to index a different URL instead | Right on a duplicate. On anything else, a silent removal |
| Redirects elsewhere | Not usually | The URL forwards somewhere, which is normally healthy | Fix the list, not the page. A sitemap should declare URLs that answer |
Why robots.txt is not in this list
A robots.txt disallow blocks the fetch itself, so a bulk row would report nothing useful about it. It also does something people expect it not to: it blocks crawling rather than indexing, so a disallowed URL with inbound links can still appear in results with no description under it. For one URL, the page indexability checker reads robots.txt alongside everything here and names the rule that matched.
Working a list, in the order that pays
A hundred rows is a to-do list without an order until you put one on it. This is the sequence the results table already sorts into.
- 01
Start from the list you are worried about
A sitemap export, the URLs in a Search Console 'Crawled, currently not indexed' report, the pages from a migration, or a section of the site that stopped getting traffic. The sitemap extractor will hand you the full list if you do not have one.
- 02
Read the noindex rows first, both kinds
A noindex in the head and a noindex in the X-Robots-Tag header are the same instruction from two places, and the header version never appears in view-source. Those rows are the complete explanation for any page they land on.
- 03
Read the canonical rows one at a time
A canonical pointing elsewhere is correct on a genuine duplicate and a silent removal from the index on anything else. This is the one verdict here that cannot be judged in bulk, because only you know whether two URLs are the same page.
- 04
Treat redirects as a finding about the list, not the page
A redirecting URL usually lands somewhere healthy. What it tells you is that whatever produced this list is out of date, and a sitemap full of redirects wastes crawl budget on every pass.
- 05
Then go to Search Console for the other half
Everything above is what Google is allowed to do. Whether it has actually indexed a URL is a question only Google answers, through URL Inspection and the Pages report on a property you have verified.
- 06
Expect most unindexed pages to be perfectly indexable
Once nothing technical is blocking a page, indexing becomes a quality judgement. Thin pages, near-duplicates and pages whose content only appears after JavaScript runs are the ones Google crawls and then declines, and no tag change fixes that.
- Get the list from the sitemap extractor if you do not have one, which hands back every URL a site declares as CSV.
- Line the blocked URLs up against a Search Console export. A noindexed page with impressions is urgent; a noindexed page nobody has visited in a year is probably correct.
- Re-run after deploying, because the point of the fix is the row turning green, and a config change that did not take looks identical to one that did.
What this index checker reports
Up to 100 URLs a run
Sent in batches of twelve, with rows landing while the run continues
Both kinds of noindex
The meta robots tag and the X-Robots-Tag header, which view-source never shows
Canonical targets
Where a canonical points, when it points anywhere but at the page itself
Blocked lifted to the top
The URLs that cannot be indexed sorted above the healthy ones
Correct-looking cases marked amber
Canonicals and redirects flagged as judgement calls, not faults
The fix on every row
What to change, not just what was found
CSV export
Verdict, evidence and final URL per row, ready to sort against Search Console
Honest about its limits
It reports permission to index, and says so rather than implying more
Why this cannot tell you what is indexed
Frequently asked questions
Does this tell me if my page is indexed by Google?
No, and no third-party tool can do it reliably. This checks whether anything stops Google indexing a URL: the status code, a noindex in the head or the headers, and where the canonical points. Whether Google has chosen to index it is answered only by Search Console, through URL Inspection and the Pages report. The two get conflated constantly, and the distinction is the point: most pages that are not indexed are perfectly indexable.
Why not use a site: search to check indexing?
Because Google describes that number as a rough estimate and it fluctuates without anything changing. It is useful for a yes-or-no on a single URL and useless as a count. Automating site: searches at scale also gets you rate-limited and blocked quickly, which is why free tools that claim to do it either break or quietly show you something else.
What is the difference between crawling and indexing?
Crawling is Google fetching the page. Indexing is Google deciding to store it and let it rank. They fail separately and have different fixes. A robots.txt disallow blocks crawling; a noindex tag blocks indexing. Blocking a page in robots.txt does not remove it from the index, and it can leave a URL ranking with no description because Google was never allowed to read the page it is showing.
Why does the tool report a noindex I cannot see in my HTML?
Because it is in the response headers rather than the markup. X-Robots-Tag carries exactly the same directives as a meta robots tag, is set by the server or the CDN, and is completely invisible when you view source. It is the most common cause of a page that nobody can work out why it disappeared, which is why this tool reads both and says which one it found.
Is a canonical pointing elsewhere a problem?
It depends, and it is the one verdict here you have to judge page by page. On a genuine duplicate, such as a URL with tracking parameters, a canonical pointing at the clean version is exactly right. On a page that is not a duplicate, it tells Google to index a different URL instead, which removes this one from consideration without any warning anywhere.
How many URLs can I check at once?
Twenty-five on a first run, and a hundred once you unlock. They are sent in batches of twelve so a slow host cannot kill the whole run, and rows land as each batch returns rather than at the end. Each URL is one fetch, so a hundred URLs is a hundred requests against the site you are checking.
What does 'Crawled, currently not indexed' mean, and will this explain it?
It means Google fetched the page, read it, and chose not to index it, which is a quality judgement rather than a technical block. This tool will confirm nothing technical is in the way, which is worth ruling out first, and then it has told you everything it can. The remaining causes are thin content, near-duplication, or content that only appears after JavaScript runs so the crawler saw an empty shell.
Does a redirecting URL mean something is broken?
Usually not. The destination is normally healthy and the finding is about the list rather than the page. A sitemap should declare URLs that answer with 200, not ones that forward, because every redirecting URL in a sitemap spends a crawl on a page that was never the destination.
Does this check robots.txt?
Not in this bulk run, deliberately, because a robots.txt disallow blocks the fetch itself and the row would then report nothing useful. For a single URL, the page indexability checker reads robots.txt alongside everything here and names the rule that matched. For a whole file, the robots.txt tester takes a URL and tells you which directive decides it.
How long does indexing take?
Anywhere from hours to never. An established site publishing to a section Google crawls often can see same-day indexing. A new site with few inbound links can wait weeks, and a thin page may never be indexed at all. If a page has been live for more than a few weeks, passes every check here and is still absent, treat it as a content and internal-linking problem rather than a technical one.
Can I check a competitor's URLs?
Yes. Every signal this reads is public: the status code, the response headers and the markup any visitor receives. It is a fair way to see which of a competitor's sections they are deliberately keeping out of the index, which is often more revealing than what they publish.
What should I do about pages I want excluded?
Use a noindex and let Google crawl the page to see it. The pattern that does not work is a noindex plus a robots.txt disallow: Google has to crawl the page to read the directive, robots.txt forbids the crawl, and the page can stay indexed indefinitely. Allow the crawl, confirm the page has dropped out, and only then block the path if you still want to.
Other crawl and indexing tools
Website URL Extractor
Crawl any website and download every URL it links to as CSV, TXT, or JSON. Works straight from the live site, no sitemap required.
Use toolRedirect Chain Checker
Trace every redirect hop with status codes and response times. Detect loops, long chains, protocol downgrades, and 302 misuse.
Use tool404 Error Checker
Paste up to 100 URLs and find the dead ones. Hard 404s, 410s and server errors first, soft 404s that answer 200 flagged beside them, with the fix on every row.
Use toolHTTP Status Code Checker
Check the status code of up to 100 URLs at once: 200, 301, 404, 410 or 5xx, with every redirect hop, possible soft 404s and response times.
Use toolWebsite Speed Test
Test a page on mobile or desktop: real-visitor Core Web Vitals (LCP, INP, CLS) next to the Lighthouse score, with fixes sorted by time saved.
Use toolPage Indexability Checker
Find out why a page is not indexed. Checks status codes, noindex tags and X-Robots-Tag headers, robots.txt rules, canonicals, sitemap membership, and rendered content.
Use toolBroken Link Checker
Find links that no longer work, internal and external together. Checks up to 50 links on a page for 404 errors, redirects, and timeouts.
Use toolExternal Link Checker
Audit who a page links out to and on what terms. Destinations grouped by domain, with rel nofollow, sponsored and ugc, new-tab safety, and live status.
Use toolInternal Link Checker
Find the broken internal links in a page's body copy, plus redirects, nofollowed internal links and anchors that describe nothing, each with the fix it needs.
Use toolInternal Link Analyzer
See which internal links a page makes in its own writing and which come from the template, grouped by destination, with anchor text, nofollow and live status.
Use toolRobots.txt Tester
Analyze robots.txt to see which AI crawlers and search engines are blocked, find sitemaps, and identify access issues.
Use toolRobots.txt Generator
Build a correct robots.txt from presets for WordPress, ecommerce, and AI crawlers. Decide which bots to allow, add your sitemap, copy the file.
Use toolllms.txt Generator
Build a valid llms.txt from your own pages: the H1, blockquote summary and H2 link sections the proposal specifies, checked for relative URLs and duplicates.
Use toolHTTP Header Checker
Read every response header a URL returns, with the redirect chain timed hop by hop and X-Robots-Tag, header canonicals and caching gaps called out.
Use toolShopify robots.txt.liquid Generator
Build a robots.txt.liquid template that keeps Shopify's default rules in the Liquid loop and adds blocks for tag pages, vendor and type collections, filter parameters and AI crawlers.
Use toolShopify App Script Checker
Inventory every script tag on a Shopify product or collection page by origin: theme, Shopify platform, app and third-party. Flags async, defer and blocking scripts, sizes up to 30 external files and lists legacy ScriptTag API scripts.
Use toolRead up on crawling and indexing
A tool tells you what is wrong. These explain what to do about it.
429 Status Code: What Happens When Googlebot Gets One
Read guideCrawled vs Discovered Currently Not Indexed: What Each Means and How to Fix It
Read guide9 Best Screaming Frog Alternatives in 2026 (Free, Cloud & Enterprise)
Read guideTerms this tool checks
- Indexation
Indexation is the process by which search engines add web pages to their searchable database (index).
- Noindex
Noindex is a robots meta tag directive (meta name="robots" content="noindex") that tells search engines not to include a specific page in their search index.
- Canonical Tag
A canonical tag (rel="canonical") is an HTML element that tells search engines which version of a page is the preferred, authoritative copy when multiple URLs serve similar or identical content.
- Crawl Budget
Crawl budget is the number of pages a search engine will crawl on your site within a given timeframe, determined by crawl rate limit (how fast Googlebot can crawl without overloading your server) and crawl demand (how much Google wants to crawl based on popularity and freshness).
A hundred URLs checked, and the ones that matter unknown
Indexability is half the answer. CrawlRaven joins a crawl of your site to Search Console and GA4, so every blocked, canonicalised or unindexed page arrives with the impressions, clicks and sessions behind it, and you fix the ones costing you something first.