Brotli Compression

Following a few performance related blogs recently this one is following a similar trend and is going to look at Brotli compression. It was announced by Google in September 2015 and it claims to offer 20%-26% better compression than existing compression algorithms. That's a pretty noteworthy improvement to compression which would bring a nice performance boost with it.

Brotli

Announced on the Google Open Source Blog, Brotli is promising a pretty considerable boost in compression and the benefits here are both clear and very desirable. Smaller pages and assets result in less bandwidth being used which results in faster page load times, lower bandwidth costs and improved battery performance on mobile devices. Who doesn't want their website to be faster and cheaper?!

Support - Browsers

Firefox supports Brotli in v44+ and Chrome does so in v51+ which brings support on all 6 platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView). Between Firefox and Chromium we're probably looking at being able to support Brotli on a pretty hefty chunk of the web, if not a majority, very soon indeed. A user agent can indicate support for Brotli by including br in the accept-encoding request header. Here are my current request header for Chrome and Firefox:

Chrome (53.0.2785.101 m): accept-encoding:gzip, deflate, sdch, br
Firefox (48.0.2): accept-encoding:gzip, deflate, br

The simple addition of the br flag lets the server know that the browser can use Brotli and if the server can compress using Brotli, it should.

Support - Servers

Right now I can only find NginX that has a module for Brotli support. Whilst this is a little dissapointing it's to be expected for such a new feature and hopefully other platforms will soon follow. If I've missed anything please feel free to drop me a link in the comments below and I will update the article.

Performance

The main thing that I really wanted to know was just how much of a difference Brotli compression was going to make first hand. Vlad Krasnov did a very detailed analysis of this on the CloudFlare blog and it's well worth checking out. For my own satisfaction I also wanted to have a play around with Brotli and see what it was capable of. The tests are really basic and I'm just satisfying my own curiosity here, but the results look good!

At first glance it does appear that Brotli is offering quite a good gain. Looking at the numbers side by side it's a little more clear.

Unsurprisingly the jpg image, which is already compressed, saw little improvement but the size of the page itself did see a notable gain using Brotli. A 1/3 reduction in the size of the page is a considerable improvement and the CSS file saw almost a 10% drop in size when using Brotli too. My testing here is, admittedly, incredibly limited but it does look like Brotli has something to offer.

Adding Brotli support to NginX

There are 2 different ways that you can implement Brotli compression. There is static compression where you create a compressed version of your files with a .br extension and NginX will serve those, saving the compression overhead of doing it on the fly. Then there is dynamic or stream compression which is useful for things like pages with dynamic content that NginX will need to compress on each request. We're going to implement both. First of all you need to install Brotli on your server. I have a directory called nginx in my home folder where I build/install all of my Nginx related modifications but feel free to update the paths as needed.

cd ~/nginx/
git clone https://github.com/google/brotli.git
cd brotli
sudo python setup.py install
cd tests
make

If you don't have all of the necessary tools installed the following command will install them all.

sudo apt-get install git python2.7 python-dev

Next we need to install the libbrotli wrapper.

cd ~/nginx/
git clone https://github.com/bagder/libbrotli
cd libbrotli
./autogen.sh
./configure
make
sudo make install

Finally we need to get the NginX Brotli module.

cd ~/nginx/
git clone https://github.com/google/ngx_brotli

Now that we have everything ready to build NginX all you need to do is add the module to your configure arguments. If you're not familiar with building NginX from source then I have full instructions in my article on how to build the PageSpeed module into NginX. I won't duplicate the information here but use that article as a reference if you need. If not, simply add a new parameter to your configure arguments.

./configure --add-module=/home/scott/nginx/ngx_brotli
... #rest of configure arguments
make
sudo make install
sudo service nginx restart

NginX is now ready to start using Brotli compression and all we need to do is enable it.

sudo nano /etc/nginx/nginx.conf

http {
    ...
    brotli on;
    brotli_static on;
    brotli_types *;
    ...
}

The first directive you're adding enables dynamic compression and the second one enables static compression. The third lists which mime types that Brotli should be used for and I've gone for a wildcard here but you can be specific and list only the ones you want. There are a few more options that you can configure and details can be found on GitHub. That should be enough to get Brotli up and running so you can reload this config and check the content-encoding header for br on your responses to see it in action.

sudo service nginx restart

What's happening now though is that NginX is compressing every response on the fly, including things like images, JS and CSS, which are static files. To save this overhead we can compress them and save a compressed version on disk that NginX will serve instead. There is a nice tool provided in the library to do this.

~/nginx/brotli/bin/bro --input main.min.css --output main.min.css.br

Again, update the path to the appropriate location where you installed the library but this is the only command required to create a compressed version of one of your files. When looking for main.min.css NginX will also look for main.min.css.br and if it finds it, it will serve that instead. This saves the CPU overhead of compressing the file on the fly for every request and makes Brotli even faster. You can also put together a little script to compress all of the appropriate files for when you make changes.

#!/bin/bash
for x in `find /var/www/ghost/content/themes/futura/assets/js/ -type f -iname '*.js'`
do
    /home/scott/nginx/brotli/bin/bro --input ${x} --output ${x}.br
done

for x in `find /var/www/ghost/content/themes/futura/assets/css/ -type f -iname '*.css'`
    do /home/scott/nginx/brotli/bin/bro --input ${x} --output ${x}.br
done

This will go through my theme folder and find all .js and .css files and create a .br version of them for me! Call the script any time you update a file and you're good to go. I've also created another script to do exactly the same thing for all of my image folders too.

You now have Brotli compression support on your server for those that support it and will still fall back to other compression methods for those that don't. It's a win-win!