Yesterday it was announced that Let's Encrypt are to revoke a large number of certificates after identifying a bug in their CA software, Boulder. This blog is here to keep a record of details related to this event and help those affected.


What happened

Let's Encrypt found a bug in their CA software, called Boulder, that meant they may not have been properly checking CAA records. They announced the bug here and explain that they suspended issuance for a short period while they deployed a fix.

As a result of this bug they could have mis-issued a large number of certificate and as a result will be revoking a large number of certificates, announced here. The revocations will start taking place at 2020-03-04 20:00 UTC (3:00pm US EST) and if your certificate is one of the 3m+ affected, it will need to be replaced ASAP. I have created a list of affected domains that you can check. If your domain is on this list you need to renew your certificate ASAP if you haven't already.

To renew a Let's Encrypt certificate simply go through the same procedure you normally would for issuing new certificates to obtain a new one. That is all that is required to fix this issue.


Identifying sites still affected

I'm currently updating my crawler fleet to identify sites that still need to take action and will publish the list here shortly once the first scan is complete. Hopefully this list will reduce in size throughout the day as organisations take action to renew.

If you have a small number of sites you can scan them here: https://checkhost.unboundtest.com/

Here is the list of all affected domains. If your domain is on this list and you have not renewed already, you need to renew now: https://www.dropbox.com/s/tp47jrpge6brv2g/log.txt?dl=0

I've started to publish lists of sites still using affected certificates. These will be updated regularly and you can check them here: https://github.com/ScottHelme/le-scan


What will happen to sites that don't renew their certificate?

For websites that don't renew their certificate before the deadline, there is a good chance that they may end up looking like this.



BadSSL has a demo page that you can visit here in your favourite browser to see exactly what it looks like, but the chances are it will be a fairly serious error message like the one above, shown in Chrome.


News coverage

The BBC have already covered the story with comments from Alan Woodward so hopefully the mainstream coverage will help notify those who need to renew. I also have a Twitter thread that I'm keeping up to date with the latest information so follow that thread if you want live information.



I thought revocation was broken?

Revocation is totally broken when expected to protect us against an active network adversary. Those users who are not under attack will still see effective revocations and receive the error message shown above. For the time being this means that users will see websites break if they do not renew their certificates. If you're interested in more details on how revocation is broken and how it might be fixed, see my latest blog post on CRLite.

If you want proof that revocation is broken you can disable the checks by blocking requests to the Let's Encrypt OCSP responder and sites will start working again. A hosts entry should do the trick:


127.0.0.1 ocsp.int-x3.letsencrypt.org