Home IT infrastructure Distribution of the TLS SNI standard

Distribution of the TLS SNI standard

by admin

Distribution of the TLS SNI standard
With their observations on the percentage of expansion TLS SNI of the TLS protocol (RFC 4366) in the total volume of encrypted HTTPS requests are shared by Akamai experts.
The past few years have seen an explosion in the number of clients supporting TLS SNI (a standard that makes HTTPS much more scalable). While in early 2014 only 85% of HTTPS client requests were made using TLS SNI, today many users of the Akamai service receive up to 99% of requests in TLS SNI format.
This indicates a global trend of deploying SNI-only sites. At the same time 31% of Alexa Top 100k sites with valid HTTPS certificates report this certificate to client only if request is in TLS SNI format.
Distribution of the TLS SNI standard
There were two serious problems with the HTTPS transition: the first was the inaccessibility of the certificates themselves, and the second was the technical difficulties with the TLS virtual hosting. Easily accessible automatically issued Domain Validation (DV) certificates, provided by services such as Let’s Encrypt, solved the first problem. TLS Server Name Indication (SNI) is a technology that solves the second problem. Until recently, its widespread adoption has been hampered by the large number of clients that do not support this standard.
The growth in the use of TLS SNI proves that it can already be considered a universal standard. The rest of the customers who do not support TLS SNI can solve this problem within about a year, if they want to have access to all sites at all. Especially with the growing use of HTTPS — already about half of the pages are loaded over HTTPS, according to Firefox.
TLS is at the heart of HTTPS and is used for encryption and authentication. When Netscape developed SSL v3 and its fairly successful IETF-standardized descendant, TLS 0.1, they found that virtual hosts were not ready to use them. To authenticate, the client sends a "ClientHello" message to the server and expects a "ServerHello" message in response, containing a security certificate that identifies the server name. If the name in the certificate does not match the name the client expects to receive, a properly configured client should terminate the connection and issue an error message. But in the case of shared hosting one server serves dozens, hundreds, thousands, or even millions of sites and in this case the server must know exactly which certificate to return to the client.
If the ClientHello request does not contain the site name, the server serving HTTPS on port 443 has no choice but to allocate a separate IP for each certificate (and therefore site). For a CDN service like Akamai deployed in hundreds or thousands of locations, this means allocating hundreds or thousands of virtual IPs for each certificate. And due to the finiteness of the pool of freely allocated IPv4s, this quickly becomes a fundamental problem that makes growth and scaling difficult.
Server Name Identification (SNI) was introduced in TLS in 2003 in RFC 3546. It allowed clients to include the name of the server they wanted to connect to in the ClientHello request, allowing the server to understand which name certificate to report back to the client in the ServerHello message. This SNI extension was the route information that made virtual hosting possible, similar to the routing role of the HTTP headers that were introduced from the beginning, when HTTP/1.0 was adopted in 1997. In addition, load balancers in data centers now have the ability to use TLS SNI to distribute traffic directed to a common IP address to the correct backend servers.
Distribution of the TLS SNI standard
IPv6 and HTTP/2 could certainly help solve the problem. HTTP/2 itself already requires clients to send TLS SNI. And IPv6’s virtually infinite address space solves the virtual address problem, and for that reason Akamai has been introducing "default" dual addressing for its IPv4+IPv6 clients since 2016. But most of the very clients that don’t support TLS SNI don’t also support HTTP/2 and IPv6.
Lack of full customer acceptance made it difficult to implement TLS SNI in practice. Previously widespread operating systems, such as Windows XP and Android 2.2, did not support TLS SNI, and the introduction of a strict requirement for SNI requests in HTTPS would have left them out. In 2013, Akamai saw that only 80% of customers supported the protocol. Even as late as April 2015, only 90% of HTTPS requests contained SNI. The decrease in the number of clients not supporting this standard is also due to other reasons changing the overall situation with HTTPS protocols, such as the discontinuation of support for SHA-1 certificates.
The situation has changed dramatically over the past two years. Today at Akamai, an average of 98% of requests contain TLS SNI on the client side. However, most of the non-SNI supporting clients are individual applications developed privately and using legacy protocols on their individual routes. Therefore, the median participant in the HTTPS exchange – the hardware that gives the client a certificate – already sees 99% of requests that include SNI. And 12% of those points already see 99.9% of SNI-authenticated traffic.
If we exclude China, where SNI is used to a much lesser extent, and the US, where there are a number of bots/agents that do not support SNI, the median participant, based on a sample of Akamai service users, will see 99.4% of requests containing SNI. Even in the US, the median participant sees 98.4% of SNI containing queries.
Taking a closer look at non-SNI customers, according to Akamai they include :

  • Modern clients that use intermediate proxies to intercept HTTPS traffic, such as for parental control or corporate network traffic control. These intermediate proxies degrade the reliability of TLS encryption, and the logic of their work, forcing to spoof server names, contradicts the ideology of SNI. In addition, these systems often do not validate certificates at all.
  • The remaining systems are running Windows XP and even older versions of the OS. It’s not quite clear how they connect at all after SHA-1 is canceled, but they may also be behind intermediate proxies that don’t ask for certificates.
  • Some bots/robots owned by search engines. We contacted some of the top listings, and they said that in the near future they will still be introducing TLS SNIs too. There seem to be plenty of both benign and malicious bots continuously scanning the entire IPv4 space looking for HTTPS servers.
  • Various clients and bots based on old versions of WebKit, Java, curl and python. And those clients that use more recent versions of these libraries should still have implemented mechanisms for proper initialization of exchanges based on TLS SNI. This includes a wide range of clients, from malware to various headerless browsers to analyze site performance.
  • Some private apps (download managers, game console apps, some mobile apps). They tend to be isolated to certain sites.

Some sites wanted to switch completely to TLS SNI and were willing to lose 10% to 20% of their visitors. Now moving to TLS SNI "cuts off" only 1% of visitors, which are mostly bots, malware, and insecure clients, so there will be many more who want to move to TLS SNI. Let’s Encrypt has issued about 27 million free certificates, prompting a significant number of website operators to switch to HTTPS. Akamai has been issuing TLS SNI-only certificates as part of its batch offerings since late 2015, thus encouraging sites to migrate to that standard and, in fact, making room for the future – forcing their possible users to migrate to TLS SNI as well.
The shift to using SNI exclusively is already underway. Today, roughly 75% of Alexa top 1000 sites are accessible via HTTPS, of which 12% use only TLS SNI (that is, they only report a validated certificate in response to an SNI request). Among Alexa top-100k sites, only 55% already use HTTPS, but among them 31% are SNI only, even though many of them are still available via HTTP.
The more sites will require TLS SNI, the more pressure clients will feel to implement the SNI mechanism in their protocols. The decreasing number of non-SNI clients encourages sites to move faster to new certificates, and this, in turn, forces non-SNI clients to move to TLS SNI, similar to what once happened with HTTPS per se.
The trend is clear: TLS SNI is becoming a universal, common, and maybe even "default" standard in HTTPS. The client software developers must understand this and keep in mind the need to correctly implement this protocol in their packages.

You may also like