Why Is My Page Not Indexed? A Diagnostic Flow That Works

A page is missing from Google for one of four reasons: Google does not know the URL, Google knows it but has not crawled it, Google crawled it and was told not to index it (or could not read it), or Google crawled it and decided it was not worth indexing. The URL Inspection tool in Search Console tells you which of these applies in under a minute. Start there, read the exact status, then use the flow below.

Guessing is what wastes time. People rewrite content when the problem is a noindex header, or resubmit sitemaps when the problem is duplicate content. Diagnose first.

Step 0: Confirm the page really is not indexed

Do not rely on a normal Google search or even on site:. Google says the site: operator does not necessarily show all indexed URLs. The authoritative check:

1. Open Search Console and select the property that contains the URL.
2. Paste the full URL into the inspection bar at the top and press Enter.
3. Read the headline. "URL is on Google" means indexed. "URL is not on Google" means not indexed.
4. Expand Page indexing. The line right under the headline is the status you need, and the panel shows the last crawl date, the crawl result, whether indexing is allowed, the user-declared canonical and the Google-selected canonical.

If the headline says the URL is on Google but you cannot find the page in search, stop here. You have a ranking question, not an indexing one. See indexed but not ranking.

Two details that catch people out:

• Inspect the exact URL. http vs https, www vs non-www, trailing slash and uppercase letters are all different URLs to Google.
• The default result shows the indexed version, which may be weeks old. Test live URL shows what Googlebot gets right now. You often need both.

The decision table

Find your status in the left column. Details for each group follow below.

Status in URL Inspection	Stage that failed	Most likely cause	First thing to do
URL is unknown to Google	Discovery	No links, not in sitemap, very new	Add to sitemap, add internal links, request indexing
Discovered - currently not indexed	Crawl scheduling	Low crawl priority, slow server, too many low value URLs	Strengthen internal links, fix server speed, prune junk URLs
Crawled - currently not indexed	Indexing decision	Thin, duplicate or low value content, or empty render	Compare rendered HTML with what users see, improve or consolidate
Excluded by 'noindex' tag	Directive	Meta robots or X-Robots-Tag header	Remove the directive, test live, request indexing
Blocked by robots.txt	Crawl access	Disallow rule matches the URL	Edit robots.txt
Page with redirect	Canonicalisation	URL redirects to another URL	Usually nothing, inspect the target instead
Alternate page with proper canonical tag	Canonicalisation	Canonical tag points to another URL	Nothing if intended, else fix the tag
Duplicate without user-selected canonical	Canonicalisation	Google found a duplicate and you gave no canonical	Add canonicals, consolidate duplicates
Duplicate, Google chose different canonical than user	Canonicalisation	Your canonical hint was overruled	Align all signals, or differentiate the pages
Soft 404	Indexing decision	Page looks empty or like an error page	Add real content, or return a real 404
Not found (404)	Crawl result	Page returned 404 to Googlebot	Restore the page or redirect it
Server error (5xx)	Crawl result	Server failed when Googlebot fetched	Fix the server, check logs
Blocked due to access forbidden (403) or unauthorized request (401)	Crawl access	Firewall, login or bot protection	Allow Googlebot through

Group 1: Google does not know the URL

Status: "URL is unknown to Google".

Google has never seen this URL in a link, sitemap or submission. This is normal for a page published an hour ago. It is a problem for a page published a month ago.

How to confirm the cause:

• Is the URL in your sitemap? Pull your sitemap URLs with the sitemap URL extractor and search the list.
• Is the sitemap submitted and read successfully? Check Indexing > Sitemaps for a "Success" status and a recent "Last read" date.
• Does any indexed page link to it with a plain <a href>?

Fix: add the URL to the sitemap, link to it from relevant indexed pages, then use Request indexing. Full detail in URL is unknown to Google.

Group 2: Known but not crawled

Status: "Discovered - currently not indexed".

Google's documentation says this typically means Google wanted to crawl the URL but expected that doing so would overload the site, so it rescheduled the crawl. In practice I see three causes, in this order of frequency:

1. The URL looks unimportant. Few or no internal links, deep in pagination, only present in the sitemap. Google queues it behind everything else.
2. The site has far more URLs than it has value. Faceted navigation, calendar pages, internal search results and tag archives create thousands of low value URLs, and the good pages wait in the same queue.
3. The server is slow or erroring. Check Settings > Crawl stats for rising average response time or a share of 5xx responses. Google reduces crawling when a host struggles.

On a small new site this status often clears on its own within a few weeks. On a larger site it rarely does without work. See Discovered - currently not indexed and crawl budget explained.

Group 3: Crawled and told not to index

These are the easiest to fix because the cause is explicit.

Excluded by 'noindex' tag

Google fetched the page and found a noindex, either in the HTML:

<meta name="robots" content="noindex">

or in the HTTP response:

X-Robots-Tag: noindex

The header version is the one people miss, because it is invisible in the page source. Check it with the HTTP headers checker. Also check the rendered HTML in the live test, because a JavaScript framework or tag manager can inject a noindex that is not in the raw source. Steps are in Excluded by noindex tag.

Blocked by robots.txt

A Disallow rule matches the URL, so Google cannot fetch it. Test the URL against your live file with the robots.txt tester. Remember that robots.txt controls crawling, not indexing: a blocked URL with links pointing to it can still be indexed without content, shown as "Indexed, though blocked by robots.txt".

401, 403, 404 and 5xx

The inspection result shows "Page fetch: Failed" with the reason. If the page works in your browser but fails for Google, the usual suspects are bot protection on a CDN or WAF, geo blocking (Googlebot crawls mostly from US IP addresses), rate limiting, or a server that falls over under crawl load. Confirm with Test live URL, then check server logs for requests from Googlebot and the status codes they received.

Group 4: Crawled and folded into another URL

These statuses mean Google treats your URL as a version of another URL and indexes the other one. Expand the inspection result and compare User-declared canonical with Google-selected canonical.

• Page with redirect: the URL you inspected redirects. The destination is what gets indexed. Inspect the destination. This is only a problem if the redirect is unintended. Trace it with the redirect checker.
• Alternate page with proper canonical tag: your own canonical tag points elsewhere and Google agreed. Correct for parameter URLs, AMP and print versions. Wrong if a template bug points every page's canonical at the homepage or at page 1 of a series.
• Duplicate without user-selected canonical: Google found another page with the same or very similar content and you declared no canonical. Add a self-referencing canonical to the version you want, and point duplicates to it.
• Duplicate, Google chose different canonical than user: you declared a canonical and Google disagreed, usually because redirects, internal links, sitemap entries or the content itself point the other way.

The key question for this whole group: is the URL Google chose acceptable to you? If yes, there is nothing to fix. The page's content is indexed, under a different address. If no, align every signal (canonical tag, internal links, sitemap, redirects) on the URL you want. The canonical tags guide explains how Google weighs them.

Group 5: Crawled and rejected on quality

Status: "Crawled - currently not indexed" or "Soft 404".

Google fetched the page successfully, nothing blocked it, and it still stayed out. Google's description is just that the page was crawled but not indexed and may or may not be indexed in the future. It does not tell you why, so you have to investigate.

Check in this order:

1. Rendering. Run Test live URL, then View tested page > Screenshot and HTML. If the main content is missing from the rendered HTML, Google is evaluating a nearly empty page. This is common with client-side rendered apps, content behind consent walls and content loaded on scroll or click. See JavaScript SEO.
2. Duplication. Put a distinctive sentence from the page in quotes and search for it. If other URLs (yours or someone else's) show up with the same text, Google has little reason to keep a second copy.
3. Substance. Be honest about whether the page gives a searcher something they cannot get from the pages already ranking. Pages that are mostly template with a sentence or two of unique text are the typical casualties.
4. Site-level patterns. Open Indexing > Pages, click the "Crawled - currently not indexed" row and look at the examples. If they cluster in one template (tag pages, thin product variants, location pages) the fix is at template level: consolidate, enrich, or noindex the lot.

A soft 404 is the harsher version: Google thinks the page is effectively an error or empty page even though it returned 200. "No results found", "out of stock, no longer available" and blank renders are the usual triggers. See soft 404 errors and Crawled - currently not indexed.

When the whole site is affected

If most or all pages are not indexed, check these site-level causes before looking at individual URLs:

• Security & Manual actions > Manual actions. A manual action can remove some or all of a site. The report states the reason and lets you request a review.
• Security & Manual actions > Security issues. Hacked content or malware.
• Indexing > Removals. Someone with access may have filed a temporary removal for a prefix. Removals last about six months and can be cancelled.
• Site-wide noindex or Disallow: /. Usually a staging setting that reached production.
• Wrong property. You are looking at http:// or the non-www prefix while the site lives on https://www. A Domain property avoids this.
• Very new domain. A few weeks of patience, a sitemap and some external links is the fix.

If pages were indexed before and then dropped out, the causes are different. See why pages get deindexed.

After the fix: a sensible routine

1. Fix the cause and confirm it with Test live URL. The live test should show "URL is available to Google" with indexing allowed and the canonical you expect.
2. Click Request indexing once. Repeating it does not speed anything up.
3. If many URLs shared the same problem, go to Indexing > Pages, open the reason and click Validate fix. Google rechecks a sample first and then the rest. It says this typically takes up to about two weeks and can take much longer.
4. Re-inspect after one to two weeks. Look at Last crawl. If the date is older than your fix, Google has not seen the change yet. If it is newer and the status is unchanged, your fix did not address the real cause.

Doing this by hand is fine for ten URLs. For hundreds, the URL Inspection API allows 2,000 inspections per property per day, which is what TurboIndex uses to track real index status for every URL in your sitemaps and flag the ones that change state.

What not to do

• Do not delete and republish the page under a new URL to "reset" it. You lose any signals the old URL had and the new one starts from zero.
• Do not add the page to robots.txt to "clean up" the report. Blocked pages cannot be evaluated and can still be indexed without content.
• Do not stuff more keywords into a page stuck in "Crawled - currently not indexed". The issue is value and uniqueness, not keyword count.
• Do not treat every non-indexed URL as a problem. Redirects, canonicalised duplicates and deliberately noindexed pages belong in the "not indexed" part of the report. Filter the report by your sitemap to see only pages you actually want indexed.

Quick reference: which tool confirms which cause

Suspected cause	Fastest confirmation
noindex in HTML or header, robots.txt block, bad status, canonical mismatch	Indexability checker
Header-level directives	HTTP headers checker
Redirect chain or loop	Redirect checker
Content missing after render	URL Inspection > Test live URL > View tested page
Googlebot blocked by firewall	Live test fetch failure plus server logs
Low crawl priority	Crawl stats report, internal link count
Duplicate content	Quoted sentence search, Google-selected canonical field

Work from the status, confirm the cause with evidence, fix once, then give Google time. That order solves most non-indexed pages without any guesswork.