It's a little late but here it is! The analysis of the Alexa Top 1 Million sites for February of 2019. We have good news, increased numbers, new comparisons and more!
The crawl
I can't believe this is the 8th report I've done on the Alexa Top 1 Million and the project has been going for over 4 years now. As always, the data has been taken from my crawlers so you can check out the raw data over there and maybe consider making a small donation to keep them running! Without any further interruption though, let's dive into the data and look what's changed over the last 6 months.
February 2019
The stats are in and we're seeing increases and decreases where we want them! Here are the highlights of the Feb 2019 data:
HTTPS
Not as much growth as I'd like but another 12.9% increase in HTTPS in the top 1 million taking us to a little over 58% in total.
You can see on the graph above the rate of growth is slowing, and it's even more obvious here.
Still though, the line is going in the right direction and we should be really happy with that, I just want to make sure we're not starting to see the introduction of a plateau...
HPKP
I talked in my last report about how I was giving up on HPKP and that Chrome was deprecating it, but while the deprecation hasn't taken place just yet, we've seen a massive blow to the HPKP numbers with PKP headers dropping 92% and PKPRO headers dropping 97%.
Whilst the graph is now showing a trend in use at the top, which is a complete reverse from the last report, the real thing to note is the massive reduction in the Y axis values. Here's the HPKP graph from Aug 2018 for comparison purposes.
Digging further into this data, it seems the entire crash in the use of HPKP can be attributed to one site, Tumblr. In the previous report I noted that the vast majority of sites using HPKP were *.tumblr.com and obviously had the policy site for them. In this report there isn't a single trace of HPKP on those same sites.
Security Headers
Things are continuing to go well for the adoption of Security Headers with growth in usage in all the right places.
Use of CSP saw a rise of almost 24%, STS just over 9.5% and XXP saw just over 11% too. That's some pretty significant growth but again the numbers are slightly down on the last report. We're still going in the right direction though!
Let's Encrypt
Following the growth in HTTPS we're also seeing continued growth for Let's Encrypt too.
The numbers look a little erratic lower down the ranking but there is more movement of sites there compared to those near the top of the ranking which tend to be a lot more stable. That aside though, there is still a definite increase in the presence of Let's Encrypt certs in the Alexa Top 1 Million! For those interested here are the top 10 issuing intermediates in the Alexa Top 1 Million:
C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 204,625
C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA 56,089
C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO ECC Domain Validation Secure Server CA 2 50,398
C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2 30,856
C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = CloudFlare Inc ECC CA-2 24,634
C = US, ST = TX, L = Houston, O = "cPanel, Inc.", CN = "cPanel, Inc. Certification Authority" 18,169
C = US, O = Amazon, OU = Server CA 1B, CN = Amazon 16,632
C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL RSA CA 2018 14,754
C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA 12,416
C = US, O = DigiCert Inc, OU = www.digicert.com, CN = GeoTrust RSA CA 2018 9,670EV Certificates
The EV numbers are interesting this time around because in previous scans we've seen HTTPS numbers increasing but not seeing EV maintain their market share. The EV number would still rise, but they were being outpaced by DV resulting in a net loss of market share for EV. This scan sees a further blow to the use of EV certificates which were present on 25,158 sites in August 2018 but only 23,705 sites in February 2019. An extra 63,136 sites turned on HTTPS in the last 6 months, but there's 1,453 less EV certificates.
The largest issuer of EV certificates in the Alexa Top 1 Million is ranked as the 13th largest certificate issuer overall and only issued 6,146 EV certificates. No EV issuer made it into the top 10 largest issuers as you can see in the list above.
C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 Extended Validation Server CA 6,146CAA
Certificate Authority Authorisation tracking was first added in Feb 2018 so this one year one from when I looked at the first numbers. There's been continued use in the growth of CAA and some interesting trends have emerged.
I wondered if the few prominent spikes were some kind of anomaly in the Aug 2018 scan but they are still here and present in the Feb 2019 scan! There is a small cluster of sites that set the CAA records in particularly high numbers and while they have moved up the ranking slightly, you can see that the pattern is still the same.
security.txt
I introduced tracking for the new security.txt file in the last scan so this will be the first time we can compare numbers, and.... they're down...
From 11,994 back in Aug 18 down to 5,805 in Feb 19 that's a pretty significant drop. Nothing has changed on the crawler in that time, same infrastructure and code, so I don't think it's anything I did. Looking at the data there are some big chunks of sites lower down the ranking that used to have security.txt files, which you can see on the blue line, and they no longer have files in such numbers. It could be that the group dropped out of the ranking or they're a group of managed sites that had it removed, but, they're definitely not there now....
Referrer Policy
The Referrer Policy is another recently added metric so we can do some comparison now. As a header that controls a security/privacy feature we of course want to see an increase in adoption, and we have!
Feature Policy
Another relatively new security based header that we can now have some comparison with is Feature Policy. With the ability to control the use of powerful APIs in the browser like geolocation and camera/microphone, Feature Policy is definitely something you should look at adding to your site!
General Stats
Every time the crawler runs it outputs some high level stats for that crawl which can make for some interesting reading themselves. Here are the stats for the crawl this data was based on:
Total Rows: 938343
Security Headers Grades:
A 20,470
A+ 1,752
B 14,220
C 34,593
D 130,501
E 15,070
F 721,653
R 84
Sites using strict-transport-security:
130,651
Sites using content-security-policy:
41,186
Sites using content-security-policy-report-only:
2,770
Sites using x-webkit-csp:
627
Sites using x-content-security-policy:
1,681
Sites using public-key-pins:
698
Sites using public-key-pins-report-only:
94
Sites using x-content-type-options:
158,968
Sites using x-frame-options:
143,998
Sites using x-xss-protection:
130,214
Sites using x-download-options:
19,483
Sites using x-permitted-cross-domain-policies:
18,653
Sites using access-control-allow-origin:
36,081
Sites using referrer-policy:
28,545
Sites using feature-policy:
1,341
Sites redirecting to HTTPS:
556,651
Sites using Let's Encrypt certificate:
204,519
Top 10 Server headers:
Apache 198,084
nginx 171,906
cloudflare 111,225
Microsoft-IIS/8.5 28,679
LiteSpeed 26,899
nginx/1.14.1 23,558
GSE 21,394
Microsoft-IIS/7.5 17,857
Microsoft-IIS/10.0 14,177
openresty 14,121
Top 10 TLDs:
.com 457,263
.org 45,883
.ru 40,909
.net 37,584
.de 29,923
.uk 15,743
.pl 15,161
.br 13,903
.in 12,780
.ir 11,420
Top 10 Certificate Issuers:
C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 204,519
C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA 56,341
C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO ECC Domain Validation Secure Server CA 2 50,267
C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2 31,143
C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = CloudFlare Inc ECC CA-2 25,039
C = US, ST = TX, L = Houston, O = "cPanel, Inc.", CN = "cPanel, Inc. Certification Authority" 18,323
C = US, O = Amazon, OU = Server CA 1B, CN = Amazon 16,615
C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL RSA CA 2018 14,836
C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA 12,539
C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA 9,961
Top 10 Protocols:
TLSv1.2 332,418
TLSv1 3,908
TLSv1.1 72
Top 10 Cipher Suites:
ECDHE-RSA-AES256-GCM-SHA384 147,503
ECDHE-RSA-AES128-GCM-SHA256 112,867
ECDHE-ECDSA-AES128-GCM-SHA256 47,190
ECDHE-RSA-AES256-SHA384 11,975
DHE-RSA-AES256-GCM-SHA384 3,031
ECDHE-RSA-AES256-SHA 2,008
0 1,837
AES256-SHA 1,405
DHE-RSA-AES256-SHA 1,402
AES256-GCM-SHA384 1,059
Top 10 PFS Key Exchange Params:
ECDH, P-256, 256 bits 309,541
ECDH, P-384, 384 bits 9,146
ECDH, P-521, 521 bits 5,148
DH, 1024 bits 3,407
DH, 2048 bits 1,339
DH, 4096 bits 143
ECDH, B-571, 570 bits 44
DH, 3072 bits 10
ECDH, brainpoolP256r1, 256 bits 1
ECDH, brainpoolP384r1, 384 bits 1
Top Key Sizes:
2048 bit 261,228
256 bit 47,807
4096 bit 24,823
384 bit 244
3072 bit 219
1024 bit 207
8192 bit 20
4056 bit 4
2432 bit 2
3248 bit 1
Sites using CAA:
13,462TLS
Regular readers will know I'm a big fan of TLS, regularly talking about it on my blog and delivering The Best TLS Training in the World authored by Ivan Ristic. I added metrics to the crawler to look at various aspects of TLS configuration and whilst things are improving, we still have a long way to go in some areas. Take public keys presented by servers as an example, RSA keys are still by far the single the dominant choice.
Of the keys presented, RSA 2048bit keys are by far the most popular, followed by ECDSA 256bit keys and then, quite surprisingly, a huge amount of RSA 4096bit keys! Now there's every chance that these servers aren't using the RSA key exchange, which would come with a hefty performance impact, but still, using RSA keys of this size is still a big thing. The other thing to note is that there's almost no RSA 3072bit keys at all...
Looking at the cipher suites the 3 most popular suites negotiated are all pretty good, ECDHE-RSA-AES256-GCM-SHA384, ECDHE-RSA-AES128-GCM-SHA256 and ECDHE-ECDSA-AES128-GCM-SHA256 came in 1st, 2nd and 3rd respectively. Unsurprisingly we see 2 suites using RSA keys for authentication at the top of the list and an ECDSA suite only making it in 3rd place. They do all support Forward Secrecy though and that's a great thing using the EC version of the DHE key exchange.
Next up is the TLS protocol itself and my crawler supports everything up to TLSv1.2, there no TLSv1.3 just yet. That means the values you see on these diagrams are the highest TLS protocol version the server at the other end supports.
TLSv1.2 is way out there in the lead, there's a tiny sliver of TLSv1 down at the bottom and as you'd probably expect almost no TLSv1.1 at all. Remember, these protocol version are the highest the site we scanned can support, it's a bit worry to see anything other than TLSv1.2 on here really. To show that another way, the vast majority of sites do support TLSv1.2 but there are still some out there that don't.
I'm hoping to have the crawlers update with TLSv1.3 support before the next report so we should be able to include some numbers for that next time.
Get the data
I make all of the data that this report was based on available in my Google Sheet here. It includes the raw numbers, the graphs and a few other bits of information that you might like.
The raw data is output daily my crawler over at crawler.ninja and the text summary files can also be seen daily here and JSON files here.