Right now the entire Internet is taking a journey and our destination is a world where the only connection used to load a site is a secure one. We've come a long way, yes, but we still have a long way to go.
The beginning
In the beginning the Internet was a safe place where everyone was trusted and this trust is visible in all of the protocols that power the Internet. DNS, BGP, HTTP, ARP, WPAD and more. All of these protocols allow us to connect and to communicate and they are all built on trust. We send a DNS request and get a DNS response and we hope that it's correct. There is no security, an attacker can see the requests we make and even modify the response, but this is how it has been since the beginning. With the explosive growth of the Internet came unwanted and untrusted parties and so the threats began to develop. The industry has started to address these problems by introducing secure versions of protocols, like DNSSEC and HTTPS, but these mechanisms become available and are opt-in, resulting in a pretty low adoption rate for any of them.
Introducing security
Back in 1994 the first encryption protocol was standardised and secure communication for web browsers was introduced, we had SSLv2 (Secure Sockets Layer version 2). Unfortunately SSLv2 didn't really last long and SSLv3 was introduced only a year later in 1995. To give some time reference for that, the 1st generation iPod was still over 6 years away!
SSLv3 did have a pretty good run and was finally broken in late 2014 with the POODLE Attack meaning the protocol had a pretty good innings of almost 20 years! After SSLv3 the protocol name was changed to TLS (Transport Layer Security) for non-technical reasons and we iterated through TLSv1.0 in 1999, TLSv1.1 in 2006 and TLSv1.2 in 2008. Right now the lowest protocol version we can use is TLSv1.0 but only until June 2018 when the PCI (Payment Card Industry) are making changes to their DSS (Data Security Standard) and will not allow the use of TLSv1.0 for transmission of payment card data. As it stands right now sites can support an incredibly robust deployment of TLS and serve over https:// instead of http://. Whilst this is great and we're seeing an enormous increase in the number of sites serving over https:// in recent months, there are some additional concerns.
Redirecting to a secure scheme
Historically when a site began to serve over https:// they would issue a 301 permanent redirect from the http:// scheme to the https:// equivalent and all was great in the world. The browser tried to communicate over an insecure connection, we told them not to do that and they come back over a secure connection, great! Except it wasn't so great. In 2009 a researcher called Moxie Marlinspike released a tool called SSLStrip. The tool leveraged a simple problem to launch a devastating attack to do exactly what it says and strip SSL from communications. The problem is that the 301 redirect, the message to tell the browser to upgrade to a secure connection, is sent over an insecure connection, making it vulnerable to an attacker. I talked about using SSLStrip on the WiFi Pineapple and it's a really simple attack to carry out with enormous consequences. The humble 301 redirect was no longer sufficient.
We needed to be more strict
The problem of SSLStrip was real and it was a highly effective tool, it had to be dealt with. To address the problem a new mechanism was standardised called HSTS (HTTP Strict Transport Security) which was pretty much the most simple mitigation that could have been conceived. When the browser and the server communicate the server will tell the browser to remember that this site uses https:// and to continue to use https:// when communicating with this site in the future no matter what happens. By asking the browser to remember this when the user comes back to our site the browser won't connect over http:// but will instead connect over https://. This removes the problem of the 301 redirect and we now have a secure connection, awesome!
There was however one tiny problem (sorry), what if the browser has never been to the site before? It wouldn't know, it couldn't know, so it'd use an insecure connection first and leave the user vulnerable to attack. The only way we could fix this is if the browser already knew the site wanted https:// before it had even spoken to the site, but we can't do that, can we?
Going all in on security
The problem with HSTS was known as the TOFU (Trust On First Use) problem. If a browser had never been to a site before it couldn't know they want to use https://, the browser would connect on http:// and the user is vulnerable. The TOFU problem was fixed with something called HSTS Preloading which is another incredibly simple mechanism. With HSTS Preloading you can do exactly what it sounds like, you can preload your HSTS setting into the browser. You go to the HSTS Preload Site, you submit your domain for inclusion in the list and your domain name is built into the source code of the browser!
This fixes the TOFU problem nicely because now, even if the browser has never been to your site, it already knows about HSTS because it's quite literally built into the browser itself.
Present day
Everything that you've read so far is our journey to the present day, this is exactly where we are right now. Yes we're still trying to drive HTTPS adoption, HSTS is lagging behind that and HSTS preloading even further behind that, but achieving everything we've just looked at is the best you can do right now. We now need to start looking forwards at what the next steps will be and how we can progress things beyond where they currently are. The 301 from http:// to https:// addresses the symptom, not the problem, much like HSTS and HSTS Preloading address other symptoms and not the underlying problem. No one knows exactly what the next steps will be for the wider industry, but here are my predictions for the coming years.
HTTP is bad
We've already seen the introduction of browser warnings for http:// pages and this trend is only going to continue and become more widespread. In the current version of Chrome (v62 at the time of writing) any web page served over http:// that has a user input field will flag 'Not Secure' in the address bar. The user is about to input some data into the page and the browser is simply letting them know the page is not secure, which is entirely correct. You can read more details on Chrome's approach to these warnings and their plans for the future but the end goal, which I can't wait for, is this:
There are no official deadlines but given the tremendous progress being made on encrypting the web right now I'm hopeful that by late 2018 or early 2019 we will see some big steps being taken down the road towards marking all http:// as insecure.
Switching the default scheme
If you open your browser right now and type any address into the address bar like twitter.com the browser will automatically fill in the scheme for you to help out. The problem is that the scheme the browser will default to, that all browsers will default to, is http://... The browser still wants to do insecure things by default. Of course, this is precisely what HSTS and HSTS Preloading address, but they address the symptoms and not the underlying problem. Why does the browser still want to do insecure things by default? The answer to that question is that most of the time insecure is what the website on the other side of the connection wants to do so the browser is correctly guessing the scheme a majority of the time. To fix the problem the browser would need to default to https:// instead of http:// but right now that just isn't feasible. Right now because a minority of sites use https:// this means that the browser would guess the wrong scheme a majority of the time, causing performance issues and additional network traffic for users across the Internet. We can't realistically consider switching the default scheme until that results in an overall improvement to the user experience and not an overall detriment. We will get there, and it makes complete sense for the browsers to switch their default scheme when the time is right, but doing that introduces another problem for us to consider.
Déjà vu
Imagine a world where the browser defaulted to https:// and wanted to do secure things by default, ahh the joy. You type in example.com and the browser reaches out to the host on 443 to establish a secure connection for you. We wouldn't need HSTS or Preloading because everything is secure. If that host serves over https:// it all goes according to plan and we arrive on a secure page with no opportunity for SSLStrip to catch us out. The problem arises though when the host doesn't have https:// support and the secure connection doesn't work, the browser is left with only one choice. If the https:// connection fails because the site doesn't support https:// the browser has to fall back to using http:// instead which will fix the issue. The side effect of this though is that we'd become vulnerable to a highly effective variant of the SSLStrip attack. Instead of SSLStrip abusing the 301 redirect sent over http:// all it would have to do now is block the https:// connection. When the browser can't connect over https:// it will assume the site doesn't support it and downgrade to http://, we're right back to where we started. Downgrade attacks aren't a new thing and we've come across them in various situations before so even if the browser does default to https://, if we can't prevent downgrade attacks then it's not actually as big a step forward as you might hope.
Fallback prevention
I mentioned above that in a default https:// world we might not need HSTS or Preloading anymore but I actually think we will, only with a slightly new purpose. HSTS indicates a host wishes to use https:// exclusively and is designed to stop SSLStrip attacks as mentioned above. In a world where the browsers had switched their default scheme from http:// to https:// then HSTS could be used to indicate hosts where the browser should never downgrade to http://. It's a minor re-interpretation of the purpose of HSTS but serves to achieve the same end result, secure comms all the time. We could even continue to use the HSTS Preload List to fix the TOFU problem for downgrade attacks. Whilst many people see the HSTS Preload list only being a short term measure during our transition to a secure web, I think it will exist for some time more.
Bye bye http://
With browsers defaulting to https:// and HSTS re-purposed for fallback prevention there is only one logical step left to take. Browsers need to disable support for http:// and port 80. Not only do they need to default to https:// but the whole idea of http:// needs to be scrapped. Until this happens we shouldn't close port 80 on our own servers, even in the present day, because closing port 80 is bad for security. With that change all sites will be required to support https:// and we can start to consider deprecating mechanisms like HSTS and the Preload List too. The reason we can't deprecate those mechanisms right now is that there is actually one more thing that we get from HSTS that I haven't mentioned yet.
Hard fail on error
So many of us have seen some kind of error page on https://, I'd bet that probably all of us have actually, and I'd further bet that a great deal of people have clicked through one of them and used a page with the big red https:// warning at the top too. Most of the time I come across these when trying to use WiFi in a hotel or an airport and the captive portal is intercepting my connection. There have been some recent improvements about the messaging around these errors but the fact always remains that allowing a user to bypass a warning about https:// will continue to be dangerous. When a site enabled HSTS the other protection they are granted, alongside the protection from SSLStrip attacks, is that users can't click through https:// warnings and compromise the security of the connection. HSTS takes these error messages from soft fail, where the use can click through them, to hard fail where the user can't bypass them. The errors become fatal. When the browsers finally arrive at defaulting to hard fail on errors we can then finally abandon HSTS and the Preload List, but not before.
We made it!
The above is a prediction of how I see the web advancing with regards to encryption and connection security. We had a slow start and it really took a long time for https:// to start gaining traction but it's really taking off now.
Even with the steps I've covered here there are still other considerations along the road. For example, when sites are testing https:// right now they can have their site available over https:// but not redirect their traffic from http:// until they are ready. If the browser were to change it's default scheme this would make things a little more difficult for testing your deployment. Whilst that wouldn't have too big an impact, and the problem becomes smaller as more sites transition or launch on https:// already. there are some easy ways to work around the issue, it just goes to show that there will be countless little things like this that come up along the way and put bumps in the road. For now the only thing we can do is continue to drive adoption of https:// across the web. The more sites we can get to support https:// the faster we can look at taking the next step on the path to a https:// only world where things like 'secure' cookies and mixed-content are no longer a problem!