How to Find All Pages Indexed by Google

The only exact list of pages Google has indexed is the one Google gives you in Search Console: the Page indexing report shows the count, and the URL Inspection API tells you the answer for any single URL. Everything else, from the site: operator to third-party checkers, is an estimate or a sample. The catch is that the report's tables stop at 1,000 rows, so for a bigger site the complete list has to be built: take the URLs from your sitemap and inspect them, 2,000 a day.

Here are the five methods, from the most accurate to the roughest, and how to combine them.

1. The Page indexing report: the real count

Open Search Console, choose the property, and click Indexing, Pages. The green Indexed number at the top is Google's own count for that property. Below the chart, View data about indexed pages opens a table of indexed URLs with the date each one was last crawled, and an Export button in the top right that gives you Google Sheets, Excel or CSV.

Two limits to know:

• 1,000 rows per table. The interface and the export both stop there. If your count says 8,400, you see 1,000 examples, not a list.
• The report lags. It is updated in batches, usually a few days behind. A page can be indexed, show up in URL Inspection as "URL is on Google", and still be missing from the report for a while.

The filter at the top of the report is the way to see more than 1,000. Switch from All known pages to All submitted pages, or pick a single sitemap. The count and the table then cover only that sitemap. If you split your sitemap into files of under 1,000 URLs each (by section, by month, by product type) and submit each one, you can export the indexed URLs of each file separately and get the complete list. That is also why many large sites keep small sitemaps even though Google allows 50,000 URLs per file.

The not-indexed side of the same report tells you why the rest is missing: "Crawled - currently not indexed", "Discovered - currently not indexed", "Excluded by 'noindex' tag" and the other states, each with its own 1,000-row table. What every state means is in the Page indexing report explained.

2. The Performance report: pages that actually showed up

Performance, Search results, then the Pages tab. Every URL listed here had at least one impression in the chosen date range, and an impression is proof of indexing. Set the range to the full 16 months Search Console keeps, and you have a list of indexed pages that Google considered worth showing at least once.

Limits:

• An indexed page that got no impressions in the period is not in the list. Deep pages, new pages and pages ranking on page 5 fall through.
• The Performance report groups by the canonical URL Google chose, so a page indexed under a different canonical shows up as that other URL.
• The interface exports up to 1,000 rows. The Search Console API (searchanalytics.query) returns up to 25,000 rows per request and lets you page through the rest with startRow, which is the way to pull the whole list.

This method is most useful as a cross-check: a page in the Performance report that your other list calls "not indexed" is a sign that your other list is stale.

3. The URL Inspection API: exact, one URL at a time

The URL Inspection tool in Search Console answers for one URL: is it on Google, when was it last crawled, which canonical Google chose and, if it is not indexed, which state it is in. The URL Inspection API gives the same answer programmatically, with these numbers from Google's documentation: 2,000 inspections per day per property and 600 per minute.

Each response carries verdict (PASS, NEUTRAL, FAIL), coverageState (the same words you see in the report, such as "Submitted and indexed" or "Crawled - currently not indexed"), lastCrawlTime, robotsTxtState, indexingState, pageFetchState, googleCanonical, userCanonical and the sitemaps the URL was found in. There is no live test through the API; it returns the indexed state Google has on record.

A minimal Python script that takes a list of URLs and writes the answer per URL to a CSV:

import csv, time
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession

SITE = "sc-domain:example.com"   # or "https://www.example.com/" for a URL-prefix property
ENDPOINT = "https://searchconsole.googleapis.com/v1/urlInspection/index:inspect"

creds = service_account.Credentials.from_service_account_file(
    "inspection-key.json",
    scopes=["https://www.googleapis.com/auth/webmasters.readonly"],
)
session = AuthorizedSession(creds)

urls = [u.strip() for u in open("urls.txt") if u.strip()][:2000]

with open("index-status.csv", "w", newline="") as out:
    w = csv.writer(out)
    w.writerow(["url", "verdict", "coverageState", "lastCrawlTime", "googleCanonical"])
    for url in urls:
        for attempt in range(3):
            r = session.post(ENDPOINT, json={"inspectionUrl": url, "siteUrl": SITE, "languageCode": "en-US"})
            if r.status_code != 429:
                break
            time.sleep(60)          # per-minute limit hit, wait and retry
        idx = r.json().get("inspectionResult", {}).get("indexStatusResult", {})
        w.writerow([url, idx.get("verdict"), idx.get("coverageState"),
                    idx.get("lastCrawlTime"), idx.get("googleCanonical")])
        time.sleep(0.2)             # stays well under 600 per minute

The service account needs the Google Search Console API enabled in its project and must be added as a user of the property under Settings, Users and permissions, the same way you would add a colleague. You can also run the script with your own Google account through OAuth; the quota is the same because it belongs to the property.

How tools use this is covered in more depth, with the trade-offs, in bulk Google index checker.

4. The site: operator: a rough look

Searching site:yoursite.com lists pages Google is willing to show from that host, and site:yoursite.com/guides/ narrows it to a path. It is handy for a sanity check and for sites you do not have Search Console access to. It is not a count: Google itself says the site: operator does not necessarily return every indexed URL, the number at the top is an estimate, and the results stop after a few hundred.

Use it to answer "is this one page indexed?" quickly, and use it to spot things that should not be there, such as staging subdomains or parameter URLs. Do not use it for a total, and do not script it yourself: Google answers bursts of automated queries with captchas instead of results. For pages of your own site, the Google index checker skips the guesswork: it runs URL Inspection for up to 25 URLs at a time and gives Google's verdict and reason for each, so a check of a few dozen pages takes minutes rather than an afternoon.

5. Bing Webmaster Tools: the same question for Bing

Bing keeps its own index, and a page indexed by Google is not automatically indexed by Bing. In Bing Webmaster Tools, Site Explorer shows what Bing knows about each URL, the URL Inspection tool answers for one URL, and the index coverage views give totals. If you have not set it up, you can import your verified sites from Search Console in a few clicks. More in how to submit a URL to Bing.

Building the complete list for a big site

For anything past a thousand pages, combine the methods:

1. Start from the sitemap. It is the list of pages you want indexed, which is the list worth checking. Pull every URL out with the sitemap URL extractor, including nested sitemap index files.
2. Clean the list. Remove duplicates, trailing slash variants and anything that is not a page with the URL list cleaner. Every duplicate costs one of the 2,000 daily inspections.
3. Inspect in daily batches of 2,000. Run the script above on the first 2,000 lines today, the next 2,000 tomorrow. A 25,000-URL site takes about two weeks for a full pass.
4. Store the answers with a date. The value of the list is in the comparison: which pages were indexed last month and are not now, which new pages were picked up, which have been "Discovered - currently not indexed" for sixty days.
5. Cross-check the total against the Indexed count in the Page indexing report. They will not match exactly, because the report counts URLs Google found anywhere (old URLs, parameters, pages not in your sitemap), while your list only covers the sitemap. A big gap in either direction is a lead: more indexed than in your sitemap means Google is indexing URLs you did not intend; far fewer means a section is not getting in.

If you would rather not run the script every day, TurboIndex does this loop for every URL in your sitemaps once Search Console is connected: it inspects each one, keeps the per-URL answer and Google's reason, rechecks indexed pages on a schedule and sends an email when one drops out. The quota is the same 2,000 a day per property, so a large site still takes days for the first pass.

Reading the numbers without fooling yourself

• Indexed is not ranking. A page can be indexed and never appear for any query that people type. The Performance report is where you see whether it earns impressions.
• Google counts canonicals. If three URLs share one canonical, one of them is indexed and the other two sit under "Alternate page with proper canonical tag" or a duplicate state. That is correct behaviour, not three missing pages.
• The report counts what Google knows, not what you have. Old redirected URLs, feed URLs and parameter variations all appear under their own states. Filter by sitemap before you worry about any number.
• A single check is a snapshot. Pages move in and out. Decide what you will compare against before you start collecting, or you end up with a spreadsheet nobody reads.

If a page you expected to find is missing from every list, the diagnostic path is in how to check if a page is indexed, starting with URL Inspection and ending with the reason Google gives for leaving it out.