The Page indexing report (Search Console > Indexing > Pages) shows how many of your URLs Google has indexed, how many it has not, and the reason for each non-indexed URL. Roughly half of the reasons are errors you should fix, and the other half are Google confirming that redirects, canonicals and noindex tags work as intended. The skill is telling the two apart.
This page goes through every status, grouped by what you should do about it.
How to read the report
The chart. Two numbers over time: Indexed (green) and Not indexed (grey). A steady grey total is nothing to worry about. Sudden changes in either line are the thing to look for.
The filter at the top left. This is the most useful control and many people never touch it:
| Filter | Shows |
|---|---|
| All known pages | Every URL Google has come across, including junk it found by itself |
| All submitted pages | Only URLs in your submitted sitemaps |
| Unsubmitted pages only | URLs Google knows that are not in any sitemap |
| A specific sitemap | URLs from that one sitemap |
Start with All submitted pages. Those are the URLs you have told Google you want indexed, so any not indexed reason there deserves attention. "Unsubmitted pages only" is good for finding URL junk: parameters, old paths, faceted URLs.
"Why pages aren't indexed". A table of reasons with a Source column (Website means you can fix it, Google systems means it is Google's decision), a validation state, a trend and a page count. Click a row for a chart and up to 1,000 example URLs. Click any example and then the magnifier icon to inspect it.
"Improve page appearance". Below the main table. Lists warnings for pages that are indexed with a problem.
Every status at a glance
| Status | Group | Usually a problem? | Short action |
|---|---|---|---|
| Server error (5xx) | Not indexed | Yes | Fix server stability, check logs |
| Redirect error | Not indexed | Yes | Fix chains, loops, bad targets |
| Blocked by robots.txt | Not indexed | If you want the page indexed | Remove the Disallow rule |
| Excluded by 'noindex' tag | Not indexed | If you want the page indexed | Remove the noindex |
| Soft 404 | Not indexed | Yes | Add real content or return a real 404 |
| Blocked due to unauthorized request (401) | Not indexed | If the page should be public | Remove the login requirement |
| Blocked due to access forbidden (403) | Not indexed | Yes, for public pages | Stop the firewall or server rule that blocks Googlebot |
| Not found (404) | Not indexed | Only for pages that should exist | Restore, redirect or leave alone |
| Blocked due to other 4xx issue | Not indexed | Yes, for public pages | Find the status code and fix it |
| Crawled - currently not indexed | Not indexed | Yes, on important pages | Improve quality, uniqueness, internal links |
| Discovered - currently not indexed | Not indexed | Yes, on important pages | Improve links, server speed, cut URL junk |
| Alternate page with proper canonical tag | Not indexed | No | Nothing |
| Duplicate without user-selected canonical | Not indexed | Sometimes | Declare a canonical |
| Duplicate, Google chose different canonical than user | Not indexed | Often | Align canonical signals |
| Page with redirect | Not indexed | No | Nothing, keep redirects out of sitemaps |
| Blocked by page removal tool | Not indexed | Temporary | Remove the request or make it permanent |
| Indexed, though blocked by robots.txt | Indexed, warning | Yes | Choose: allow crawling, or noindex |
| Page indexed without content | Indexed, warning | Yes | Check rendering and cloaking |
| Indexed | Indexed | No | Nothing |
Errors on your side: fix these first
Server error (5xx)
Googlebot requested the URL and your server answered with a 500-level code or did not answer in time. Persistent 5xx errors make Google slow its crawling of the whole host, and URLs that keep failing are dropped from the index.
Confirm: Settings > Crawl stats > By response > Server error (5xx), and look at your server logs for Googlebot requests at the same times. Errors under load often do not reproduce in a browser.
Fix: capacity, timeouts, failing application routes, overly aggressive rate limiting or bot protection. More in server errors (5xx) and Googlebot.
Redirect error
Google's help page lists four causes: a redirect chain that is too long, a redirect loop, a redirect URL that exceeds the maximum URL length, or a bad or empty URL in the chain.
Confirm: trace the URL with the Redirect Checker.
Fix: point every redirect directly at the final 200 URL in one hop.
Blocked by robots.txt
A Disallow rule matches the URL, so Google did not fetch it. Fine for admin and cart paths. A problem when it shows up under All submitted pages, because it means your sitemap and robots.txt contradict each other.
Confirm: test the URL in the Robots.txt Tester, which shows the matching rule.
Fix: remove or narrow the rule. Remember that robots.txt is not a way to keep a page out of the index. Full walkthrough in blocked by robots.txt.
Excluded by 'noindex' tag
Google crawled the page and found noindex in a robots meta tag or an X-Robots-Tag header. Correct for thank-you pages, internal search and thin archives. Wrong for anything in your sitemap.
Confirm: view source and search for noindex, and check headers with the HTTP Headers Checker. The header version is the one people miss. Also inspect the rendered HTML, because JavaScript can inject a noindex.
Fix: remove the directive, then request indexing. See excluded by noindex tag.
Soft 404
The URL returns 200, but Google thinks the page is an error page or has no real content: "no products found", an empty category, a near-blank template, or a redirect of a dead page to an unrelated one.
Fix: if the page is gone, return 404 or 410. If it is a real page, give it real content. Do not redirect removed pages to the homepage. See soft 404 errors.
401, 403 and other 4xx
- 401: the page asks for a login. Googlebot never logs in. Either the page should not be in your sitemap, or the login wall is a mistake.
- 403: the server refused Googlebot. The usual suspects are a WAF, bot protection, a CDN rule, IP or country blocking, or hotlink protection. Do not whitelist by user agent alone, since anyone can fake it. Verify Googlebot by reverse DNS or Google's published IP ranges.
- Other 4xx: anything else, such as 410 (intended) or 429 (rate limiting Googlebot, which is not intended).
Not found (404)
The URL returned 404. Google is clear that 404s are a normal part of the web and do not hurt the rest of the site. Act only when:
- the page should exist (restore it);
- the page moved (301 to the new URL);
- internal links or the sitemap still point to it (fix the links).
Old 404s can stay in the report for a long time, because Google keeps retrying known URLs occasionally.
Google's decisions: harder to fix
Discovered - currently not indexed
Google knows the URL and has not crawled it. The help page says that typically Google wanted to crawl the URL but this was expected to overload the site, so the crawl was rescheduled. That is why the last crawl date is empty. In practice the other big cause is priority: Google does not consider the URL worth a fetch yet.
What helps: stronger internal links, a faster and more stable server, fewer junk URLs competing for crawl, and better overall site quality. Details in discovered, currently not indexed.
Crawled - currently not indexed
Google fetched the page and chose not to index it. The help page says the page may or may not be indexed in the future and that there is no need to resubmit the URL for crawling.
What helps: the usual causes are thin or duplicated content, pages with no clear purpose, and weak internal linking. Feed URLs, paginated pages and parameter URLs in this bucket can be ignored. Details in crawled, currently not indexed.
Duplicates and canonicals
Alternate page with proper canonical tag
The page has a canonical tag pointing to another URL, and Google agrees. This is the system working. No action, unless the page should have been the canonical, in which case your canonical tag is wrong. See alternate page with proper canonical tag.
Duplicate without user-selected canonical
Google found duplicates, none of them declares a canonical, and Google picked a different URL from this one. Inspect the URL to see which page Google chose.
Fix: if Google chose well, add a canonical tag to make it official and move on. If not, add self-referencing canonicals, redirect the variants (http, non-www, trailing slash, parameters) and link consistently. See duplicate without user-selected canonical.
Duplicate, Google chose different canonical than user
You declared a canonical and Google overruled it, because other signals point elsewhere. This one deserves attention.
Confirm: URL Inspection shows "User-declared canonical" next to "Google-selected canonical". Check the declared one with the Canonical Tag Checker.
Common causes: two pages with near-identical content, internal links and sitemap entries that favour the other URL, a canonical pointing to a redirect or a non-200 page, hreflang pages in the same language with the same content.
Page with redirect
The URL redirects, so the target gets indexed and this URL does not. Expected for every http to https and old to new redirect. The only action: make sure redirecting URLs are not in your sitemap. See page with redirect.
Blocked by page removal tool
Someone with access filed a request in Indexing > Removals. It hides the URL for about six months. If the block should be permanent, add noindex or return 404 before it expires. If it was a mistake, cancel the request in the Removals report.
Indexed, with warnings
Indexed, though blocked by robots.txt
Google cannot crawl the page and indexed the bare URL anyway, because other pages link to it. The result in search has no useful description. Decide what you want:
- Want it indexed properly: remove the robots.txt rule.
- Want it out of the index: remove the robots.txt rule and add noindex. Google has to crawl the page to see the noindex. This feels backwards and it is the only way that works.
Page indexed without content
Google indexed the URL and could not read any content. Causes include cloaking, a response Google cannot parse, or a page that came back empty for Googlebot. Run a live test in URL Inspection and look at the rendered HTML and screenshot.
Validate fix: how it actually works
After fixing an issue, open it and click Validate fix.
- Google checks a few URLs straight away. If they still fail, validation ends immediately.
- If they pass, the state becomes Started and Google recrawls the rest of the affected URLs over time.
- Each URL ends up Passed, Failed or Pending. One Failed URL ends the whole validation as Failed, and you start over after fixing it.
Google says this typically takes up to about two weeks and sometimes much longer. Validation is optional: Google would recrawl the URLs anyway. What you gain is somewhat higher recrawl priority and an email when it finishes.
Validation is not available for every reason. For "Crawled - currently not indexed" and similar Google-side decisions it means little, since there is nothing mechanical to verify.
Getting past the 1,000 row limit
On a site with 50,000 affected URLs, 1,000 examples are a sample. Ways to see more:
- Filter by sitemap. Split your sitemap by section or template (products, categories, blog) and submit each. Then the report tells you which template has the problem, which is worth more than a longer list. See sitemap index files for large sites.
- Add URL-prefix properties for subfolders. Each has its own report and its own 1,000 rows per issue.
- Use the URL Inspection API. It returns the same coverage state per URL, with a quota of 2,000 inspections per day per property. This is how TurboIndex builds a full index status list for every URL in your sitemaps, instead of a sample.
A sensible review routine
- Filter to All submitted pages.
- Deal with anything whose source is Website: 5xx, redirect errors, robots.txt, noindex, 4xx, soft 404.
- Look at the two "currently not indexed" buckets. Are important pages in there, and do they share a template?
- Skim the duplicate reasons for URLs that should be canonical.
- Switch to Unsubmitted pages only and look for crawl waste: parameters, internal search, faceted URLs.
- Note the numbers. Trends over weeks tell you more than any single snapshot.
Frequently asked questions
- Is it bad to have many not indexed pages?
- Not by itself. The not indexed count includes redirects, duplicates with a correct canonical and pages you deliberately noindexed. Google's help page says the goal is to have the canonical version of every important page indexed, not every URL.
- How often is the Page indexing report updated?
- It is refreshed every few days and the last update date is shown at the top right of the report. It can lag behind what URL Inspection shows for an individual URL, so a page may be indexed according to URL Inspection before the report reflects it.
- How long does Validate fix take?
- Google says validation typically takes up to about two weeks and can take much longer in some cases. Validation stops with a Failed state as soon as Google finds one URL that still has the issue.
- Why does the report show only 1,000 example URLs?
- The example table and its export are limited to 1,000 rows per issue. To see more, filter the report by individual sitemaps, create URL-prefix properties for sections of the site, or check URLs through the URL Inspection API.
- What happened to the Coverage report?
- It was renamed. The old Index Coverage report with its Error, Valid with warnings, Valid and Excluded groups became the Page indexing report, which has two groups: Indexed and Not indexed.