Automated Security Header Analysis Tool For Websites — Complete Documentation & Simulation

Overview & System Context

Automated Security Header Analysis Tool For Websites: Comprehensive Web-Sourced Technical Investigation Exhaustive Research Dossier · Pure Web-Scraped Content: ~9540 Words · Real Software Screenshots: 3 1. Executive Overview & Technical Background In HTTP, the 404 HTTP status code indicates that a web client (i.e. browser) was able to communicate with a server, but the server could not provide the requested resource. The server may not have the resource or it may not wish to disclose whether it has the resource. The code is often associated with response reason Not Found and is often referred to as page not found or file not found. Often, the server generates a web page for the condition and the client displays it, and often the content indicates the error code. Since the condition is relatively commonly-encountered (i.e. due to broken or dead links), this code is one of the most recognizable on the World Wide Web.

Some websites report a "not found" error by returning a standard web page with a "200 OK" response code, falsely reporting that the page loaded properly; this is known as a soft 404. The term "soft 404" was introduced in 2004 by Ziv Bar-Yossef et al. Soft 404s are problematic for automated methods of discovering whether a link is broken. Some search engines, like Yahoo and Google, use automated processes to detect soft 404s. Soft 404s can occur as a result of configuration errors when using certain HTTP server software, for example with the Apache software, when an Error Document 404 (specified in a .htaccess file) is specified as an absolute path (e.g. http://example.com/error.html) rather than a relative path (/error.html). This can also be done on purpose to force some browsers (like Internet Explorer) to display a customized 404 error message rather than replacing what is served with a browser-specific "friendly" error message (in Internet Explorer, this behavior is triggered when a 404 is served and the received HTML is shorter than a certain length, and can be manually disabled by the user). There are also "soft 3XX" errors where content is returned with a status 200 but comes from a redirected page, such as when missing pages are redirected to the domain root/home page.

404.0 – Not found. 404.1 – Site Not Found. 404.2 – ISAPI or CGI restriction. 404.3 – MIME type restriction. 404.4 – No handler configured. 404.5 – Denied by request filtering configuration. 404.6 – Verb denied. 404.7 – File extension denied. 404.8 – Hidden namespace. 404.9 – File attribute hidden. 404.10 – Request header too long. 404.11 – Request contains double escape sequence. 404.12 – Request contains high-bit characters. 404.13 – Content length too large. 404.14 – Request URL too long. 404.15 – Query string too long. 404.16 – DAV request sent to the static file handler. 404.17 – Dynamic content mapped to the static file handler via a wildcard MIME mapping. 404.18 – Query string sequence denied. 404.19 – Denied by filtering rule. 404.20 – Too Many URL Segments.

Web servers can typically be configured to display a customised 404 error page, including a more natural description, the parent site's branding, and sometimes a site map, a search form or 404-page widget. The protocol level phrase, which is hidden from the user, is rarely customized. Internet Explorer, however, will not display custom pages unless they are larger than 512 bytes, opting instead to display a "friendly" error page. Google Chrome included similar functionality, where the 404 is replaced with alternative suggestions generated by Google algorithms, if the page is under 512 bytes in size. Another problem is that if the page does not provide a favicon, and a separate custom 404-page exists, extra traffic and longer loading times will be generated on every page view. Many organizations use 404 error pages as an opportunity to inject humor into what may otherwise be a serious website. For example, Metro UK shows a polar bear on a skateboard, and the web development agency Left Logic has a simple drawing program. During the 2015 UK general election campaign the main political parties all used their 404 pages to either take aim at political opponents or show relevant policies to potential supporters. In Europe, the NotFound project, created by multiple European organizations including Missing Children Europe and Child Focus, encourages site operators to add a snippet of code to serve customized 404 error pages which provide data about missing children. While many websites send additional information in a 404 error message—such as a link to the homepage of a website or a search box—some also endeavor to find the correct web page the user wanted. Extensions are available for some content management systems (CMSs) to do this.

Figure 1: Security Header ScannerIIIIIIIIIIIIIIIIIII – IIIII Funky caching – Generation, display and storage of dynamic content Link rot – URLs ceasing to function List of HTTP status codes The Automatic Certificate Management Environment (ACME) protocol is a communication protocol for automating interactions between certificate authorities and their users' servers, allowing the automated deployment of public key infrastructure at very low cost. It was designed by the Internet Security Research Group (ISRG) for their Let's Encrypt service. The protocol, based on passing JSON-formatted messages over HTTPS, has been published as an Internet Standard in RFC 8555 by its own chartered IETF working group.

The authorization/issuance flow has changed JWS request authorization has changed The "resource" field of JWS request bodies is replaced by a new JWS header: "url" Directory endpoint/resource renaming URI → URL renaming in challenge resources Account creation and ToS agreement are combined into one step. Previously, these were two steps. A new challenge type was implemented, TLS-ALPN-01. Two earlier challenge types, TLS-SNI-01 and TLS-SNI-02, were removed because of security issues.

Hypertext Transfer Protocol Secure (HTTPS) is an extension of the Hypertext Transfer Protocol (HTTP). It uses encryption for secure communication over a computer network, and is widely used on the Internet. In HTTPS, the communication protocol is encrypted using Transport Layer Security (TLS) or, formerly, Secure Sockets Layer (SSL). The protocol is therefore also referred to as HTTP over TLS, or HTTP over SSL. The principal motivations for HTTPS are authentication of the accessed website and protection of the privacy and integrity of the exchanged data while it is in transit. It protects against man-in-the-middle attacks, and the bidirectional block cipher encryption of communications between a client and server protects the communications against eavesdropping and tampering. The authentication aspect of HTTPS requires a trusted third party to sign server-side digital certificates. This was historically an expensive operation, which meant fully authenticated HTTPS connections were usually found only on secured payment transaction services and other secured corporate information systems on the World Wide Web. In 2016, a campaign by the Electronic Frontier Foundation with the support of web browser developers led to the protocol becoming more prevalent. HTTPS has since 2018 been used more often by web users than non-secure HTTP, primarily to protect page authenticity on all types of websites, secure accounts, and keep user communications, identity, and web browsing private.

Figure 2: Security Header ScannerIIIIIIIIIIIIIIIIIII – IIIII The Uniform Resource Identifier (URI) scheme HTTPS has identical usage syntax to the HTTP scheme. However, HTTPS signals the browser to use an added encryption layer of SSL/TLS to protect the traffic. SSL/TLS is especially suited for HTTP, since it can provide some protection even if only one side of the communication is authenticated. This is the case with HTTP transactions over the Internet, where typically only the server is authenticated (by the client examining the server's certificate).

HTTPS creates a secure channel over an insecure network. This ensures reasonable protection from eavesdroppers and man-in-the-middle attacks, provided that adequate cipher suites are used and that the server certificate is verified and trusted. Because HTTPS piggybacks HTTP entirely on top of TLS, the entirety of the underlying HTTP protocol can be encrypted. This includes the request's URL, query parameters, headers, and cookies (which often contain identifying information about the user). However, because website addresses and port numbers are necessarily part of the underlying TCP/IP protocols, HTTPS cannot protect their disclosure. In practice this means that even on a correctly configured web server, eavesdroppers can infer the IP address and port number of the web server, and sometimes even the domain name (e.g. www.example.org, but not the rest of the URL) that a user is communicating with, along with the amount of data transferred and the duration of the communication, though not the content of the communication. Web browsers know how to trust HTTPS websites based on certificate authorities that come pre-installed in their software. Certificate authorities are in this way being trusted by web browser creators to provide valid certificates. Therefore, a user should trust an HTTPS connection to a website when all of the following are true: The user trusts that their device, hosting the browser and the method to get the browser itself, is not compromised (i.e. there is no supply chain attack). The user trusts that the browser software correctly implements HTTPS with correctly pre-installed certificate authorities. The user trusts the certificate authority to vouch only for legitimate websites (i.e. the certificate authority is not compromised and there is no mis-issuance of certificates). The website provides a valid certificate, which means it was signed by a trusted authority. The certificate correctly identifies the website (e.g., when the browser visits "https://example.com", the received certificate is properly for "example.com" and not some other entity). The user trusts that the protocol's encryption layer (SSL/TLS) is sufficiently secure against eavesdroppers. HTTPS is especially important over insecure networks and networks that may be subject to tampering. Insecure networks, such as public Wi-Fi access points, allow anyone on the same local network to packet-sniff and discover sensitive information not protected by HTTPS.

Additionally, some free-to-use and paid WLAN networks have been observed tampering with webpages by engaging in packet injection in order to serve their own ads on other websites. This practice can be exploited maliciously in many ways, such as by injecting malware onto webpages and stealing users' private information. HTTPS is also important for connections over the Tor network, as malicious Tor nodes could otherwise damage or alter the contents passing through them in an insecure fashion and inject malware into the connection. This is one reason why the Electronic Frontier Foundation and the Tor Project started the development of HTTPS Everywhere, which is included in Tor Browser. As more information is revealed about global mass surveillance and criminals stealing personal information, the use of HTTPS security on all websites is becoming increasingly important regardless of the type of Internet connection being used. Even though metadata about individual pages that a user visits might not be considered sensitive, when aggregated it can reveal a lot about the user and compromise the user's privacy. Deploying HTTPS also allows the use of HTTP/2 and HTTP/3 (and their predecessors SPDY and QUIC), which are new HTTP versions designed to reduce page load times, size, and latency. It is recommended to use HTTP Strict Transport Security (HSTS) with HTTPS to protect users from man-in-the-middle attacks, especially SSL stripping. HTTPS should not be confused with the seldom-used Secure HTTP (S-HTTP) specified in RFC 2660.

The Electronic Frontier Foundation, opining that "In an ideal world, every web request could be defaulted to HTTPS", has provided an add-on called HTTPS Everywhere for Mozilla Firefox, Google Chrome, Chromium, and Android, which enables HTTPS by default for hundreds of frequently used websites. Forcing a web browser to load only HTTPS content has been supported in Firefox starting in version 83. Starting in version 94, Google Chrome is able to "always use secure connections" if toggled in the browser's settings. Prior to version 117, Google Chrome displayed a lock icon in the address bar, which has since been replaced by a "tune" icon. Many users believe that a lock icon implies a website is safe, ignoring other security concerns.

The security of HTTPS is that of the underlying TLS, which typically uses long-term public and private keys to generate a short-term session key, which is then used to encrypt the data flow between the client and the server. X.509 certificates are used to authenticate the server (and sometimes the client as well). As a consequence, certificate authorities and public key certificates are necessary to verify the relation between the certificate and its owner, as well as to generate, sign, and administer the validity of certificates. While this can be more beneficial than verifying the identities via a web of trust, the 2013 mass surveillance disclosures drew attention to certificate authorities as a potential weak point allowing man-in-the-middle attacks. An important property in this context is forward secrecy, which ensures that encrypted communications recorded in the past cannot be retrieved and decrypted should long-term secret keys or passwords be compromised in the future. Not all web servers provide forward secrecy. For HTTPS to be effective, a site must be completely hosted over HTTPS. If some of the site's contents are loaded over HTTP (scripts or images, for example), or if only a certain page that contains sensitive information, such as a log-in page, is loaded over HTTPS while the rest of the site is loaded over plain HTTP, the user will be vulnerable to attacks and surveillance. Additionally, cookies on a site served through HTTPS must have the secure attribute enabled. On a site that has sensitive information on it, the user and the session will get exposed every time that site is accessed with HTTP instead of HTTPS.

Figure 3: Security Headers Analysis & Monitoring | CSP, HSTS, X-Frame | SecurityBot A certificate may be revoked before it expires, for example because the secrecy of the private key has been compromised. Newer versions of popular browsers such as Firefox, Opera, and Internet Explorer on Windows Vista implement the Online Certificate Status Protocol (OCSP) to verify that this is not the case. The browser sends the certificate's serial number to the certificate authority or its delegate via OCSP (Online Certificate Status Protocol) and the authority responds, telling the browser whether the certificate is still valid or not. The CA may also issue a CRL to tell people that these certificates are revoked. CRLs are no longer required by the CA/Browser forum, nevertheless, they are still widely used by the CAs. Most revocation statuses on the Internet disappear soon after the expiration of the certificates.

Netscape Communications created HTTPS in 1994 for its Netscape Navigator web browser. Originally, HTTPS was used with the SSL protocol. The original SSL protocol was developed by Taher Elgamal, chief scientist at Netscape Communications. As SSL evolved into Transport Layer Security (TLS), HTTPS was formally specified by RFC 2818 in May 2000. Google announced in February 2018 that its Chrome browser would mark HTTP sites as "Not Secure" after July 2018. This move was to encourage website owners to implement HTTPS, as an effort to make the World Wide Web more secure.

Launched in 2016, the HTTP Observatory enhances web security by analyzing compliance with best security practices. It has provided insights to over 6.9 million websites through 47 million scans. Developed by Mozilla, the HTTP Observatory performs an in-depth assessment of a siteâIIs HTTP headers and other key security configurations.

Its automated scanning process provides developers and website administrators with detailed, actionable feedback, focusing on identifying and addressing potential security vulnerabilities. The tool is instrumental in helping developers and website administrators strengthen their sites against common security threats in a constantly advancing digital environment.

The HTTP Observatory provides effective security insights, guided by Mozilla's expertise and commitment to a safer and more secure internet and based on well-established trends and guidelines. A professional, production-ready command-line tool for analyzing HTTP security headers. This tool helps security professionals, developers, and system administrators assess the security posture of web applications by analyzing critical security headers and providing actionable recommendations.

The Security Headers Checker performs comprehensive analysis of HTTP security headers to identify vulnerabilities and misconfigurations. It provides detailed reports with security scores, grades, and specific recommendations to improve your web application's security posture. Contributions are welcome! Please read our contributing guidelines and submit pull requests for any improvements.

The tool performs automated scans of websites to check for the presence, absence, and configuration of critical security headers, then generates reports with findings and recommendations. HTTP security headers are one of the most effective and least expensive ways to harden your web application against common attacks. By adding a handful of response headers, you can instruct browsers to enforce HTTPS, block clickjacking, prevent cross-site scripting, and stop a range of injection-based vulnerabilities — all without changing a single line of application code.

Despite their power, security headers remain widely misconfigured. A 2025 scan of the top one million websites found that fewer than 25% deployed a meaningful Content Security Policy, and nearly 40% still lacked basic protections like X-Content-Type-Options . The gap between what is available and what is deployed represents a massive, avoidable attack surface.

This guide covers every security header you need to know in 2026, explains what each one does, shows exactly how to configure them in Nginx and Apache, and links to free tools so you can audit your security headers right now. Whether you are a developer shipping a side project or a sysadmin hardening production infrastructure, this is the single reference you need. Every time a browser requests a page, the server responds with HTTP headers before the body content arrives. Most headers are mundane — Content-Type , Cache-Control , Date — but a special subset exists purely to tell the browser how to behave from a security standpoint. These are collectively called HTTP security headers .

Security headers work on the principle of defense in depth. They do not replace secure coding practices, input validation, or proper authentication. Instead, they add a browser-enforced safety net that limits the damage when something inevitably goes wrong. A cross-site scripting flaw is far less exploitable when a strict Content Security Policy blocks inline script execution. A compromised ad network cannot frame your login page when X-Frame-Options is set to DENY .

Because these headers are enforced by the browser, they protect every user who visits your site with a modern browser — no client-side library or browser extension required. There are several compelling reasons to invest a few minutes in configuring security headers: Below is the complete list of security headers you should evaluate for every production site. We cover each in detail, and many have their own dedicated deep-dive article.

HSTS tells the browser to always connect to your site over HTTPS, even if the user types http:// in the address bar. This eliminates the brief, dangerous window of an unencrypted first request that attackers can exploit with SSL-stripping tools like sslstrip . To learn more about configuring HSTS correctly, including common pitfalls and preload submission, read our detailed guide: What is HSTS and How to Configure It Properly . You can also test your HSTS configuration instantly.

CSP is the single most powerful security header available. It lets you define an allowlist of sources from which the browser may load scripts, styles, images, fonts, frames, and other resources. Anything not on the list is blocked. CSP is also your most effective defense against cross-site scripting (XSS). When script-src does not include 'unsafe-inline' , injected script tags simply will not execute. For a complete walkthrough, see Content Security Policy (CSP) Explained: From Basics to Advanced . You can test your CSP configuration to find weaknesses.

This header prevents your pages from being embedded in