How You Manage Debian Apt Sources Is Changing

There are poorly-communicated changes afoot in the world of apt that affect how users of Debian-based Linux distributions will need to manage their third-party repositories (apt sources) and keys going forward.

On the bright side, these changes are actually for the better, and aren't something you need to take care of immediately. The pace of adoption will likely be quite slow, so it will be a while before they become mandatory.

But that said, if you're running Debian Buster (or later) or Ubuntu Impish (or later), it would be a good idea to spend a little time transitioning from the Old Way to the New Way the next time you find yourself with nothing to do.

This article will lay everything out for you, which should hopefully make it pretty easy. ;)

Hello Deb822!

The first change affects how you define your apt sources.

The changes are more easily visualized than explained. Let's pretend you want to add apt-fast to your system.

The Old Way: /etc/apt/sources.list.d/apt-fast.list

deb http://ppa.launchpad.net/apt-fast/stable/ubuntu impish main

The New Way: /etc/apt/sources.list.d/apt-fast.sources

Types: deb
URIs: https://ppa.launchpadcontent.net/apt-fast/stable/ubuntu
Suites: impish
Components: main

Basically, instead of cramming everything into a single line in a .list file, the new "Deb822" format wants you to break each bit out in a .sources file.

(The root directory, /etc/apt/sources.list.d, remains the same.)

All of the fields in the above example are required and must be present even if empty. For example, the Sublime Text repo doesn't have a defined "Component", so would need to look like this:

Types: deb
URIs: https://download.sublimetext.com
Suites: apt/stable/
Components:

As with the Old Way, you can combine multiple entries into a single file by leaving a blank line between definitions. But you might not even need to.

If, for example, you had your main Ubuntu sources in /etc/apt/sources.list as follows:

deb https://mirrors.edge.kernel.org/ubuntu/ impish main universe restricted multiverse
deb https://mirrors.edge.kernel.org/ubuntu/ impish-backports main universe restricted multiverse
deb https://mirrors.edge.kernel.org/ubuntu/ impish-security main universe restricted multiverse
deb https://mirrors.edge.kernel.org/ubuntu/ impish-updates main universe restricted multiverse

That could now be moved to /etc/apt/sources.list.d/00-ubuntu.sources and written like:

Types: deb
URIs: https://mirrors.edge.kernel.org/ubuntu/
Suites: impish impish-backports impish-security impish-updates
Components: main universe restricted multiverse

(You don't need a main /etc/apt/sources.list at all.)

"Why the 00- prefix?" I hear one of you ask. Good question!

Under the new Deb822 arrangement, if two repositories happen to overlap each other, priority is automatically given to the first one loaded. The 00- prefix ensures these canonical entries are shoved to the top of the stack.

But wait, there's more!

There are a lot of different fields that can be added to the .sources definitions beyond the four required ones. Some of them define additional repository options, like Architectures, while others override apt's default settings.

Here's the full list:

There are also a few dangerous options you should probably never use:

Migrating to Deb822.

Now that you know what the Deb822 format looks like, migrating is pretty easy.

All you need to do is rename each source in /etc/apt/sources.list.d from [whatever].list to [whatever].sources, open them in your preferred text editor, and transcribe the components from the single-line form into the Deb822 form.

If you still have a main listing at /etc/apt/sources.list, that should also be moved into /etc/apt/sources.list.d and given a .sources extension.

You can still comment out lines in a .sources file with a leading #, so if you want to give yourself a way back in case things go south, just comment out the original single-line entry until you're sure you've transcribed it correctly.

Afterwards, just run sudo apt-get update to make sure all your sources are still being pulled and that nothing broke.

And done!

Goodbye Apt-Key.

The next big change has to do with how users manage the public keys for their third-party apt sources. This change not only improves security and performance, but will also make it much easier to manage your keys going forward, so is well worth the effort!

In short, apt-key has been deprecated, and its warning message contains a typo, so is no help at all. So what's the deal?

Basically, instead of adding a new key this way:

wget -qO- https://download.sublimetext.com/sublimehq-pub.gpg | \
sudo apt-key add -

You now need to add it this way:

wget -qO- https://download.sublimetext.com/sublimehq-pub.gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/sublimetext-archive-keyring.gpg >/dev/null

(The gpg --dearmor bit in the middle is only necessary for ASCII armored keys, but 99.9% of keys you'll find online are armored, even if they're incorrectly named with the binary .gpg extension, as Sublime's is.)

One gotcha with the above is that the key needs to be owned by root, but readable by everyone else. If either of those don't happen, you'll need to fix it manually with:

# Fix the permissions:
sudo chmod 644 /usr/share/keyrings/sublimetext-archive-keyring.gpg

# Fix the ownership:
sudo chown root: /usr/share/keyrings/sublimetext-archive-keyring.gpg

The key thing to note is that all third-party keys should now be placed in /usr/share/keyrings, and they should all follow the basic naming pattern of {THE_REPO}-archive-keyring.gpg.

This is where the security improvements show up. The problem with apt-key is that anything you add to it gets trusted globally, which isn't ideal. The /usr/share/keyrings area, on the other hand, requires manual linking in the corresponding Deb822 .sources file, so can only be used for the repository it's supposed to be used for.

In keeping with the Sublime Text example, its source file should now be updated to read:

Types: deb
URIs: https://download.sublimetext.com
Suites: apt/stable/
Components:
Signed-By: /usr/share/keyrings/sublimetext-archive-keyring.gpg

The new Signed-By field accepts either a path or a fingerprint, however if we're being honest, all fingeprints look the same, so for ease-of-maintenance, I'd recommend sticking with paths.

Embedded Keys (Update 06/2023)

As of apt 2.3.10, signed-by keys can be embedded directly into .sources files!

I personally prefer this to the path-based approach referenced throughout the article. I like having everything related to a repository in one place. It's tidier and prevents orphaned keys being left behind whenever a repository is removed from the system.

(If you don't care about this, feel free to skip ahead to the next section.)

Inline keys use the same Signed-By: field as standalone keys. The only difference is that instead of a path, you feed it the entire (armored) key, like:

Signed-By:
 -----BEGIN PGP PUBLIC KEY BLOCK-----
 .
 thegibberishasciikeydatagoesherethegibberishasciikeydatagoeshere
 thegibberishasciikeydatagoesherethegibberishasciikeydatagoeshere
 thegibberishasciikeydatagoesherethegibberishasciikeydatagoeshere
 thegibberishasciikeydatagoeshere
 -----END PGP PUBLIC KEY BLOCK-----

There are three things to note:

1. Each line of the key must be indented with a single space.
2. Empty lines must be represented with a dot.
3. The BEGIN/END markers (and blank line before the hash) must always be written exactly as shown above. (Ignore your key's actual markers if different, and use these instead.)

That's all there is to it!

Migrating Apt Keys.

This is a three-part process:

1. Every third-party apt source needs its own keyring under /usr/share/keyrings;
2. Each corresponding .sources file needs a Signed-By line pointing to its keyring under /usr/share/keyrings/;
3. All non-distribution-owned keys need to be removed from the Old global keyring;

That non-distribution-owned qualifier is important. If the global keyring contains keys owned by Debian or Ubuntu, they were probably put there by system packages, and shouldn't be messed with! Haha.

So, what is the best way to go about this?

I would personally recommend pulling up the websites corresponding to each of your .sources to locate their repository setup instructions.

If their sites mention a key fingerprint, like Ubuntu PPAs do, you can steal the existing key from the global keyring like this:

# Export and copy the key into place.
apt-key export THEHEXFINGERPRINTGOESHERE | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/THEREPONAMEGOESHERE-archive-keyring.gpg >/dev/null

(Just change THEHEXFINGERPRINTGOESHERE and THEREPONAMEGOESHERE to the appropriate values.)

If they just give you a straight link to the key instead, do this:

wget -qO- THEURLGOESHERE | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/THEREPONAMEGOESHERE-archive-keyring.gpg >/dev/null

(Again, change THEURLGOESHERE and THEREPONAMEGOESHERE.)

Once you've done that and verified the ownership/permissions as mentioned in the last section, add the corresponding Signed-By line to the .sources file, then run sudo apt-get update to verify it is still fetching correctly.

Rinse and repeat until all your sources have been updated!

With that out of the way, all that's left to do is clean up the global keyring!

# List all keys.
apt-key list

# Delete each one (that isn't a Debian- or Ubuntu-owned key).
# Repeat this over and over and over again.
sudo apt-key del THEHEXFINGERPRINTGOESHERE

apt-key helpfully lists each fingerprint, but unhelpfully does so with spaces injected between the blocks. You'll need to manually remove that whitespace for the apt-key del command.

But hey, once you've done that, you're done!

Or rather, after you run sudo apt-get update one last time, you're done!

Future Things.

Once you've migrated all of your existing sources and keys, you'll just need to remember to ignore outdated instructions when adding new ones, and add them correctly. Haha.

For random third-party sites that give you the signing key, the process is no different than what we've already covered.

But Ubuntu PPAs will be a little annoying until they update their website or add-apt-repository helper to work with the New World Order.

Let's take a look at apt-fast again as an example of how you can manually add a PPA in the meantime.

To fetch this key manually, you'll need to do a bit of bullshit. Look for the fingerprint (highlighted above), then:

# First, import it into your *personal* keyring.
gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B

# Now export it to the right place.
gpg --export A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B | \
sudo tee /usr/share/keyrings/apt-fast-archive-keyring.gpg >/dev/null

# And finally delete the copy from your personal keyring.
gpg --delete-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B

From there, you just need to create /etc/apt/sources.list.d/apt-fast.sources, transcribing the single-line copy provided by the PPA page into the appropriate Deb822 format.

We didn't explicitly mention it earlier, but one neat trick with Deb822 is that it allows you to combine binary and source repositories into a single entry.

Most people don't use sources, but let's pretend that's what we want here. In that case, it would look like this:

Types: deb deb-src
URIs: https://ppa.launchpadcontent.net/apt-fast/stable/ubuntu
Suites: impish
Components: main
Signed-By: /usr/share/keyrings/apt-fast-archive-keyring.gpg

That's it! Run sudo apt-get update once again and you should be off to the races!