Every response your server sends starts with a three digit status code, and Googlebot acts on it before it looks at any content. In short: 2xx responses are passed on to indexing, 301 and 308 are strong signals that a URL has moved, 302 and 307 are weak ones, 4xx codes (except 429) tell Google the content does not exist, and 429 and 5xx tell it your server is struggling, so it slows down. The details matter when you remove pages, move a site or run maintenance.
The reference table
This follows Google's documentation on how HTTP status codes affect its crawlers.
| Code | Name | What Googlebot does | Effect on indexing |
|---|---|---|---|
| 200 | OK | Passes the content to the indexing pipeline | May be indexed. A 200 is no guarantee |
| 201, 202 | Created, Accepted | Treated as success. For 202 it waits a limited time for content | As for 200 |
| 204 | No Content | Nothing to process | Not indexed. Search Console may report a soft 404 |
| 301 | Moved Permanently | Follows the redirect | Strong signal that the target should be canonical |
| 302 | Found | Follows the redirect | Weak signal that the target should be canonical |
| 303 | See Other | Follows the redirect | Weak signal, like 302 |
| 304 | Not Modified | Tells indexing the content is unchanged since the last crawl | No change. Saves bandwidth |
| 307 | Temporary Redirect | Equivalent to 302 | Weak signal |
| 308 | Permanent Redirect | Equivalent to 301 | Strong signal |
| 400 | Bad Request | Treated as "content does not exist" | Not indexed. Indexed URLs are removed |
| 401 | Unauthorized | Same | Same. Reported as blocked due to unauthorized request |
| 403 | Forbidden | Same | Same. Reported as blocked due to access forbidden |
| 404 | Not Found | Same. Crawl frequency for the URL decreases over time | Removed from the index |
| 410 | Gone | Same as 404 | Removed from the index |
| 429 | Too Many Requests | Treated as a server error and a sign of overload. Crawl rate is reduced | Indexed URLs are kept at first, and dropped if it persists |
| 500 | Internal Server Error | Crawl rate is reduced, in proportion to how many URLs are affected | Kept at first, dropped if it persists |
| 502 | Bad Gateway | Same as 500 | Same |
| 503 | Service Unavailable | Same as 500 | Same |
Google follows up to 10 redirect hops. If the chain is longer, Search Console reports a redirect error.
To see what a list of your URLs returns right now, paste them into the Bulk HTTP Status Checker.
2xx: success is permission to evaluate, not to index
A 200 tells Google there is content to look at. Whether the page is indexed then depends on content quality, duplication and any noindex or canonical instruction. Plenty of 200 pages end up under Crawled, currently not indexed.
The 2xx problem worth knowing is the soft 404: the server says 200 while the page says "product not found", shows an empty category, or is a near empty template. Google tries to detect these and treats them as not found. Typical sources:
- A catch-all route that renders an error view with status 200 (very common in single page apps).
- Expired listings that keep the URL and replace the content with "no longer available".
- Internal search pages with zero results.
- Redirecting all missing URLs to the homepage. Google commonly treats that as a soft 404 as well.
The fix is to send the status that matches reality. There is a full walkthrough in soft 404 errors.
3xx: redirects
Permanent redirects (301, 308) tell Google the target should replace the source in the index. Temporary ones (302, 303, 307) tell it the source may come back, so it tends to keep the source URL indexed, at least at first. If a temporary redirect stays in place long enough, Google may treat it like a permanent one. The choice between them, redirect chains and server config examples are covered in 301 vs 302 redirects.
Redirected URLs appear in Search Console as Page with redirect, which is expected and not an error.
304 Not Modified
A 304 is the answer to a conditional request. Googlebot may send If-Modified-Since or If-None-Match, based on the Last-Modified or ETag you returned earlier:
GET /guide HTTP/1.1
Host: www.example.com
If-None-Match: "a1b2c3"
HTTP/1.1 304 Not Modified
ETag: "a1b2c3"
Google tells its indexing system that the content is the same as last time, and no body is transferred. On large sites this saves real server load and lets Google spend its crawling on pages that did change. Never send 304 when the content has changed.
4xx: the content does not exist
Google treats all 4xx codes except 429 the same way: the indexing pipeline is told the content does not exist. URLs that were indexed get removed, and new ones are not considered. Google also states that 4xx responses (again except 429) have no effect on crawl rate.
404 and 410
Use them for content that is gone and has no close replacement. Google keeps checking a 404 URL now and then for a long time, in case it comes back, with decreasing frequency. That is normal. Old 404s lingering in your Search Console report are not a quality problem.
If the content moved or has a clear successor, use a 301 instead, so links and users end up somewhere useful.
Apache and nginx examples for retiring a section with 410:
RedirectMatch 410 ^/old-campaign/
location ^~ /old-campaign/ {
return 410;
}
401 and 403
Pages behind a login return these, and that is fine: they should not be indexed. The trouble starts when public pages return 403 to Googlebot only, usually because a firewall, bot protection rule or country block catches Google's IP addresses. Users see a working site while Search Console fills with "Blocked due to access forbidden (403)". Compare a URL Inspection live test with what your browser gets, and check your WAF logs for verified Googlebot addresses. How to verify them is in log file analysis for Googlebot.
Do not use 4xx to slow Googlebot down
Google warns specifically against returning 401, 403 or 404 to limit crawling. They do not reduce crawl rate, and they do get your pages removed. Rate limiting is what 429 and 503 are for.
429
Treated as a server error, not as a missing page. Googlebot reads it as "overloaded" and backs off. Useful for short periods. Harmful if it continues, for the same reason as 5xx.
5xx: server errors
A 5xx makes Googlebot slow its crawling of the site, and the reduction scales with the number of URLs returning errors. Already indexed URLs stay in the index for a while. If the errors continue, they are dropped. Google does not publish an exact grace period. Its guidance on reducing crawl rate advises against serving error codes to Googlebot for longer than one to two days.
For planned maintenance, send 503 with Retry-After:
# nginx: maintenance switch
if (-f /var/www/maintenance.flag) {
return 503;
}
error_page 503 @maintenance;
location @maintenance {
add_header Retry-After 3600 always;
root /var/www/maintenance;
rewrite ^ /index.html break;
}
What not to do during maintenance:
- Serve the maintenance page with status 200. Google may index "We'll be back soon" as the content of every URL.
- Redirect everything to
/maintenancewith a 302 and a 200 on that page. Same outcome. - Return 404. That says the pages are gone.
One special case: robots.txt. If /robots.txt itself returns 5xx, Google stops crawling the host for a period, because it cannot tell what it is allowed to fetch. Keep that file static and available even when the application is down. The handling is described in the robots.txt guide, and the whole topic of outages in server errors and Googlebot.
Picking the right code
| Situation | Send |
|---|---|
| Page exists and should be evaluated | 200 |
| Page moved for good | 301 (or 308) |
| Short-lived move, A/B test, geo or login redirect | 302 (or 307) |
| Content unchanged since the crawler's last visit | 304, in response to a conditional request |
| Page deleted, no replacement | 404 or 410 |
| Page deleted, close replacement exists | 301 to the replacement |
| Login required | 401 or 403, or a 302 to the login page |
| Server overloaded, need Googlebot to ease off | 429 or 503, for hours, not weeks |
| Planned maintenance | 503 with Retry-After |
| Out of stock product that will return | 200 with clear availability information |
How to monitor status codes
- Page indexing report: rows for Not found (404), Soft 404, Server error (5xx), Redirect error, and the blocked 401 and 403 variants.
- Crawl stats report (Settings, then Crawl stats): the "By response" breakdown shows the share of each code across all Googlebot requests, with example URLs.
- Server logs: the only complete view. A daily count of Googlebot requests per status code catches a rising 5xx rate long before Search Console does.
- Spot checks: the Redirect Checker traces every hop of a chain and shows the code at each step.
Status codes are the most basic conversation between your server and a crawler. Make them truthful, and most of the confusing states in Search Console never appear.
Frequently asked questions
- Is a 410 better than a 404 for removing pages?
- For Google the two are handled the same way: the content is treated as gone and the URL is dropped from the index once it has been recrawled. Google staff have said a 410 may be processed marginally faster, but the difference is too small to justify reworking anything. Use whichever your platform supports.
- Do 404 errors hurt my site's rankings?
- No. 404s for pages that do not exist are normal and do not count against the rest of the site. They only matter when URLs that should work return 404, or when important pages with backlinks were removed without a redirect.
- Which status code should I use during maintenance?
- 503 Service Unavailable, ideally with a Retry-After header. It tells Googlebot the outage is temporary, so indexed pages are kept. Do not leave it in place for more than a day or two, because URLs that persistently return server errors are eventually removed from the index.
- How many redirects will Googlebot follow?
- Google documents that its crawlers follow up to 10 redirect hops. After that the URL is reported as a redirect error. For users and for crawl efficiency, aim for a single hop.
- What is a soft 404?
- A page that tells the user something is missing or empty while the server returns 200. Google detects many of these, labels them Soft 404 in Search Console and leaves them out of the index. Return a real 404 or 410 for missing content.