HTTP Headers Checker
See every HTTP response header a URL sends. Headers that matter for SEO are highlighted and explained.
Why look at response headers
Headers are the part of a response you never see in the browser window, and several of them change how search engines handle a page. When a page looks fine in the source code but still does not get indexed, the headers are the next place to look.
The headers worth checking
X-Robots-Tag
A value containing noindex or none keeps the URL out of Google. It is often added by a server config, a CDN rule or a staging setup, and then forgotten. The indexability checker tests this together with robots.txt and the meta tag.
Link
Can carry rel="canonical" and rel="alternate" hreflang annotations. Some platforms add these automatically. Conflicts between the header and the HTML are hard to spot, so compare them with the canonical checker and the hreflang checker.
Content-Type
Tells the client what the file is. An HTML page served as text/plain will not be rendered or indexed as a web page. A missing charset can produce broken characters in snippets.
Content-Encoding
Shows gzip, br or zstd compression. The tool asks for compressed content the same way browsers do. If a text response comes back without this header, compression is off for that URL, which makes pages slower to download.
Cache-Control, ETag and Last-Modified
Cache-Control sets how long a response may be reused. ETag and Last-Modified are validators: they let a client ask "has this changed?" and get a 304 with no body. The tool warns when none of these are present.
Vary
Lists the request headers that change the response. Vary: User-Agent is the signal Google recommends when you serve different HTML to mobile and desktop on the same URL.
Strict-Transport-Security and Server
HSTS makes browsers use https without asking. Server names the software or CDN that answered, which helps when you need to find out which layer added a header.
Next step
If the headers are clean and the page is still missing from Google, it may simply not have been crawled yet. TurboIndex submits your sitemap URLs to Google and reports which ones are indexed.
Questions and answers
What is the X-Robots-Tag header?
It carries the same directives as the meta robots tag (noindex, nofollow, noarchive and so on) but in the HTTP response instead of the HTML. It is the only way to set noindex on PDFs, images and other files without a head section. Because it is invisible in the page source, a stray X-Robots-Tag: noindex is a classic cause of pages that refuse to get indexed.
Can a canonical be set in an HTTP header?
Yes. A header like Link: <https://yoursite.com/file.pdf>; rel="canonical" is supported by Google. It is mostly used for non-HTML files. If the page also has a canonical tag in the HTML, both should name the same URL.
Do caching headers affect SEO?
Indirectly. ETag and Last-Modified let Googlebot make conditional requests and receive a small 304 Not Modified answer when nothing changed, which leaves more crawl capacity for pages that did change. Cache-Control mainly affects how fast repeat visits are.
Why are the headers here different from what my browser shows?
Servers and CDNs vary their response by user agent, cookies, location and accepted encodings. This tool sends a request without cookies from our server and identifies itself as TurboIndexBot. If the URL redirects, you see the headers of the final response.