Nginx Static Compression, Gzip and Brotli
Network transfer speeds are the ultimate performance bottleneck most web sites face. It doesn't really matter how fast and powerful a server is, or how fast and powerful a visitor's computer is; for the web site to get from A to B, data has to make its way across aging, rat-nibbled wires. Even under optimal conditions, data can't travel faster than light, meaning you would have to wait at least 8 minutes for a web page hosted on the sun to even begin loading.
TL;DR, at this final benchmarking frontier, every byte counts.
Static Assets
Now that the Internet is all fancy, web pages are having to do more: more interactivity, more sexy, more whizbang. And since most of this is happening in realtime right there in the visitor's browser, that means larger and more complicated Javascript, stylesheets, web fonts, and flexible graphical formats like SVG.
Which, of course, means more damn bytes to send over the wires.
One common way to mitigate this is by pre-compressing static text assets server-side, making for smaller payloads traveling over the wires. Compression and decompression both take time, just not as much time as sending as it would to send the original, bloated sources. Historically, Gzip has been the compression method of choice. But now, there's a new player on the scene: Brotli.
For reference, the following shows the relative disk usage for all of the Javascript, CSS, and SVG source files used by this very site.
As you can see, Gzip basically cuts everything down to about a third of the original size, while Brotli gets it down to about a quarter. Not too shabby!
Serving It
Not every browser supports Gzip or Brotli — that would be too easy — so you can't just link directly to compressed versions. In icky browsers like Safari or IE, they'd come through all gibberishy.
Luckily, Nginx has modules for both formats that make it really easy. These modules both come in two flavors which can be toggled independently of one another:
- Static: Nginx will check to see whether a pre-compressed file exists and if it does, and if the visitor's browser supports it, it will seamlessly serve that instead. For example, a request for
style.cssmight returnstyle.css.brorstyle.css.gz. - Dynamic: Nginx will compress the requested asset on-the-fly and send that (again, only if the visitor's browser supports it).
The dynamic mode is completely hands-off and independent of site content, but does require an ongoing resource expenditure to compress and re-compress everything. The static mode, on the other hand, requires the site developer generate their own compressed files, but once that's done, the hard part's over, and again, it'll be hands-off.
Gzip
In Debian Stretch, the nginx-full package comes with native Gzip support. To enable it, just add the following to your nginx.conf's http{} block.
##
# Gzip Settings
##
# Dynamic mode.
gzip on;
gzip_comp_level 6;
gzip_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;
# Static mode.
gzip_static on;
For additional Gzip settings and documentation, visit the module's main page.
Brotli
Brotli, though, didn't make the cut for Stretch. To enable this functionality, you'll need to build Nginx from source. Were you to build from the raw, upstream sources — as most tutorials suggest — you would be unable to use any of the libnginx* or nginx-common packages provided by Debian. Instead, we'll show you how to modify an existing nginx-full/nginx-light Debian package.
While Debian provides Nginx in three different build flavors, each available in two versions, you'll only need to rebuild the one flavor+version combination you're actually using. For this tutorial, we're assuming that combination is nginx-full+1.13. If you're using the nginx-light or nginx-extras variants, just substitute any mention of "full" with "light" or "extras" respectively. If you aren't using the backports branch, just remove -t stretch-backports any time you see it and swap out any mention of "1.13." with "1.10.".
Before we begin, you'll need to make sure you have the deb-src sister repos in your /etc/apt/sources.list file. Your APT sources should look something like this:
# Main goods.
deb http://ftp.us.debian.org/debian/ stretch main
deb-src http://ftp.us.debian.org/debian/ stretch main
# Security updates.
deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main
# Other updates.
deb http://ftp.us.debian.org/debian/ stretch-updates main
deb-src http://ftp.us.debian.org/debian/ stretch-updates main
# Backports.
deb http://ftp.us.debian.org/debian stretch-backports main
deb-src http://ftp.us.debian.org/debian stretch-backports main
If you had to make changes to your APT sources, run sudo apt-get update to pull those down.
Okay, onto the build!
# Make sure your system has the build dependencies.
sudo apt-get build-dep -t stretch-backports nginx-full
# We'll also need "git" and "brotli" for the Brotli module.
sudo apt-get install git brotli
# Make yourself a working directory and go to it.
mkdir /tmp/nginx && cd /tmp/nginx
# Fetch Debian's package source.
sudo apt-get source -t stretch-backports nginx-full
# Download the module sources. If you aren't building from
# backports, swap "1.13.3" with the version you've got.
git clone https://github.com/google/ngx_brotli.git /tmp/nginx/nginx-1.13.3/ngx_brotli
cd /tmp/nginx/nginx-1.13.3/ngx_brotli && \
git submodule update --init --recursive
Almost done! Before building, we just need to add --add-module=/tmp/nginx/nginx-1.13.3/ngx_brotli \ to the build configuration. The Debian package sources include details for all flavors, but you only need to add this to the flavor you care about, e.g. "full". To do this, open /tmp/nginx/nginx-1.13.3/debian/rules in your editor of choice. Scroll past the "common" flags to find the flavor-specific flags, and throw that line in the middle of the flavor's configs, between the --with rules and --add-dynamic-module rules.
Build it!
# Get back to the root source directory.
cd /tmp/nginx/nginx-1.13.3
# Build some packages.
dpkg-buildpackage -b
The above command will generate all the Nginx-related deb packages in the parent directory /tmp/nginx. Listing that directory is kind of overwhelming, but you actually only need the one file you altered, e.g. the one with "nginx-full" somewhere in the name.
That package can be installed the usual way: sudo dpkg -i /tmp/nginx/THE-DEB-FILE.
If you didn't have any Nginx packages installed previously, you'll likely have to follow up the above command with sudo apt-get install -f to install the dependencies.
Just in case Debian didn't handle this itself, restart the Nginx service: sudo service nginx restart.
We're done with the build sources, so unless you wanted to keep them for reference, you can go ahead and delete the entire /tmp/nginx directory.
NOW you can finally enable Brotli(!) by throwing the following next to your Gzip settings in e.g. /etc/nginx/nginx.conf:
##
# Brotli Settings
##
# Dynamic mode.
brotli on;
brotli_comp_level 6;
brotli_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;
# Static mode.
brotli_static on;
Looks familiar, don't it? Again, more details on the available settings can be found at the module's main page.
Tweaking It
You can choose to enable dynamic compression, static compression, or both.
Dynamic compression, particularly Gzip, is a pretty safe and helpful bet for environments with files that aren't necessarily under your direct control (like all the crap that comes with a CMS, for example). But if you go this route, know that there'll be a sweet spot. Maximum compression will ensure the smallest possible files, but will also require more CPU power and time to achieve. For most servers, a compression level of 6 is a good place to start.
Static compression, on the other hand, is ideal for stable, predictable environments, particularly those with some sort of build process in place. By integrating compression into your task manager (e.g. Grunt), compressed copies can be generated automatically whenever you make changes to the source files. And since this is happening once, on your computer, you can set the compression levels to the maximum without having to worry about server overhead.
Security
Let's talk BREACH real quick. In certain contexts, compression can undermine the privacy of TLS/SPDY transfers. This particular threat has been largely mooted on multiple fronts since its debut, but, well, there are still a lot of old-ass configurations and browsers in the wild, so somebody, somewhere, will always be susceptible.
To be extra safe, limit your asset compression to inconsequential things like general libraries, etc., and avoid compressing sensitive things like authenticated API responses or account pages.
Also, while we're on the topic, the version of Nginx included in Debian Stretch supports HTTP/2, so if you haven't already done so, you should enable that for any SSL-capable virtual hosts. It is real easy, just throw an http2 on any SSL listen lines, like:
server {
listen 443 ssl http2;
listen [::]:443 ssl ipv6only=on http2;
...
}
That's it! Sleep tight, don't let the bed bugs byte.