Nginx + Brotli on Debian Stretch… Without Any Source Monkeying!

Network transfer speeds are the ultimate performance bottleneck most web sites face. It doesn’t really matter how fast and powerful a server is, or how fast and powerful a visitor’s computer is; for the web site to get from A to B, data has to make its way across aging, rat-nibbled wires. Even under optimal conditions, data can’t travel faster than light, meaning you would have to wait at least 8 minutes for a web page hosted on the sun to even begin loading.

TL;DR, at this final benchmarking frontier, every byte counts.

Static Assets

Now that the Internet is all fancy, web pages are having to do more: more interactivity, more sexy, more whizbang. And since most of this is happening in realtime right there in the visitor’s browser, that means larger and more complicated Javascript, stylesheets, web fonts, and flexible graphical formats like SVG.

Which, of course, means more damn bytes to send over the wires.

One common way to mitigate this is by pre-compressing static text assets server-side, making for smaller payloads traveling over the wires. Compression and decompression both take time, just not as much time as sending as it would to send the original, bloated sources. Historically, Gzip has been the compression method of choice. But now, there’s a new player on the scene: Brotli.

For reference, the following shows the relative disk usage for all of the Javascript, CSS, and SVG source files used by this very site.

Compressed sizes relative to the original (minified) text sources; Gzip (9) and Brotli (11)

As you can see, Gzip basically cuts everything down to about a third of the original size, while Brotli gets it down to about a quarter. Not too shabby!

Serving It

Not every browser supports Gzip or Brotli — that would be too easy — so you can’t just link directly to compressed versions. In icky browsers like Safari or IE, they’d come through all gibberishy.

Luckily, Nginx has modules for both formats that make it really easy. These modules both come in two flavors which can be toggled independently of one another:

  • Static: Nginx will check to see whether a pre-compressed file exists and if it does, and if the visitor’s browser supports it, it will seamlessly serve that instead. For example, a request for style.css might return style.css.br or style.css.gz.
  • Dynamic: Nginx will compress the requested asset on-the-fly and send that (again, only if the visitor’s browser supports it).

The dynamic mode is completely hands-off and independent of site content, but does require an ongoing resource expenditure to compress and re-compress everything. The static mode, on the other hand, requires the site developer generate their own compressed files, but once that’s done, the hard part’s over, and again, it’ll be hands-off.

Gzip

In Debian Stretch, the nginx-full package comes with native Gzip support. To enable it, just add the following to your nginx.conf‘s http{} block.

##
# Gzip Settings
##

# Dynamic mode.
gzip on;
gzip_comp_level 6;
gzip_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;

# Static mode.
gzip_static on;

For additional Gzip settings and documentation, visit the module’s main page.

Brotli

Brotli, though, didn’t make the cut for Stretch. But fear not, we’ve packaged up a nice little alternative .deb that adds the capability. But before we get into the how, a quick note: most major browsers will only support Brotli compression over HTTPS connections. But you’re a good person and are already covering everything with an SSL certificate anyway, right? 🙂

Anyhoo, onto the install!

# Import the signing key.
wget -qO - https://apt.blobfolio.com/public.gpg.key | sudo apt-key add -

# The repository only accepts HTTPS connections. Some Debian
# environments oddly lack HTTPS Apt support out-of-the-box. Just in
# case, run the following to make sure HTTPS support is installed.
sudo apt-get install apt-transport-https

# Add the repo to your sources and update the package list.
sudo echo "deb [arch=amd64] https://apt.blobfolio.com/debian/ stretch main" > /etc/apt/sources.list.d/blobfolio.list
sudo apt-get update

# Now here's the tricky part. If you are already running nginx-full,
# it needs to be uninstalled and replaced. This can be done by
# running the following:
sudo apt-get remove -y nginx nginx-full && sudo apt-get install -y nginx-full-brotli

# If you haven't installed nginx yet, you can just run the regular
# install command:
sudo apt-get install nginx-full-brotli

Before you add any Brotli settings to your nginx.conf, make sure that your configuration is set to include any enabled, dynamic modules. Dynamic modules are a new addition to Debian’s Nginx packages, so if you have an older config it might not be doing it. The following include line can go at the top of the config file with any other non-scoped declarations (i.e. before any event{} or http{} blocks):

user www-data;
worker_processes auto;
pid /var/run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
...

Okay, now you can enable Brotli by throwing the following next to your Gzip settings:

##
# Brotli Settings
##

# Dynamic mode.
brotli on;
brotli_comp_level 6;
brotli_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;

# Static mode.
brotli_static on;

Looks familiar, don’t it? Again, more details on the available settings can be found at the module’s main page.

Tweaking It

You can choose to enable dynamic compression, static compression, or both.

Dynamic compression, particularly Gzip, is a pretty safe and helpful bet for environments with files that aren’t necessarily under your direct control (like all the crap that comes with a CMS, for example). But if you go this route, know that there’ll be a sweet spot. Maximum compression will ensure the smallest possible files, but will also require more CPU power and time to achieve. For most servers, a compression level of 6 is a good place to start.

Static compression, on the other hand, is ideal for stable, predictable environments, particularly those with some sort of build process in place. By integrating compression into your task manager (e.g. Grunt), compressed copies can be generated automatically whenever you make changes to the source files. And since this is happening once, on your computer, you can set the compression levels to the maximum without having to worry about server overhead.

Security

Let’s talk BREACH real quick. In certain contexts, compression can undermine the privacy of TLS/SPDY transfers. This particular threat has been largely mooted on multiple fronts since its debut, but, well, there are still a lot of old-ass configurations and browsers in the wild, so somebody, somewhere, will always be susceptible.

To be extra safe, limit your asset compression to inconsequential things like general libraries, etc., and avoid compressing sensitive things like authenticated API responses or account pages.

Also, while we’re on the topic, the version of Nginx included in Debian Stretch supports HTTP/2, so if you haven’t already done so, you should enable that for any SSL-capable virtual hosts. It is real easy, just throw an http2 on any SSL listen lines, like:

server {
	listen 443 ssl http2;
	listen [::]:443 ssl ipv6only=on http2;
	...
}

That’s it! Sleep tight, don’t let the bed bugs byte.