Basic Debian package workflow

October 19, 2024

This page records my daily commands used in packaging, building and maintaining a Debian package, as well as the build environment setup.

Basic workflow

If you want to update a package with a new upstream release:

gbp clone ...
gbp import-orig $UPSTREAM_TAR_URL
gbp import-orig --uscan --verbose --upstream-version=6.1.2 --upstream-tag='upstream/%(version)s+dfsg'
gbp export-orig  # if no .tar file
dh_make -i -p package_version -c custom --copyrightfile ../LICENSE.md
# Build with various tools

debsign ../.changes
dput

If there is already a packaged version, and you want to adopt a new upstream version:

gbp import-orig --uscan --verbose --upstream-version=$UPSTREAM_VERSION --upstream-tag='upstream/%(version)s+dfsg'

This will respect d/watch and d/copyright to exclude unnecessary files from the import source, and here we use --upstream-tag to specify the upstream tag format.

  • https://blog.hosiet.me/blog/2016/09/15/make-debian-package-with-git-the-canonical-way/

The output of this step is d/copyright. Some developers regard this step as the most boring and time-consuming process. But it's a must-have file in Debian packages, which are distributed directly or through the mirrors worldwide. Therefore, the package maintainer must ensure the license of all files are consistent, and compatible with DFSG.

There exist automation tools easing the borden of maintainers:

  • scan-copyrights: just simply run scan-copyrights
  • debmake: run debmake -c, or -cc with more detailing outputs, or -k to compare the source tree license information with the d/copyright file

Currently my personal license checking workflow is:

# generate basic license info, with file paths merged
scan-copyrights
# check the license
debmake -cc
debmake -k

Add patches

Use gbp pq to manage the source code patches, note that you should stash or commit all local changes before entering the pq state:

gbp pq import
# step 1:
# modify the code here, note that all modifications can appear outside of d/
# each commit is regarded as a patch

# step 2:
# if everything is OK, apply the patch:
gbp pq export [--patch-numbers] [--commit]

When upgrading packages to a new upstream version, one can follow the steps to refresh patches and re-apply them to the new upstream source:

gbp import-orig --uscan --verbose --upstream-version=x.y
gbp pq import --force --time-machine=4
gbp pq rebase # fix conflicts in this step
gbp pq export [--patch-numbers] [--commit]
  • https://manpages.ubuntu.com/manpages/xenial/man1/gbp-pq.1.html

After accepted

Once your upload has been accepted by the FTP-master team, you should:

  • Tag the version and upload to Salsa: git tag -s $debian_version -m "msg" && git push --tags

Building environment

sbuild with mmdebstrap

sudo apt install mmdebstrap apt-cacher-ng
mmdebstrap --variant=standard \
           --aptopt='Acquire::http { Proxy "http://127.0.0.1:3142"; }' \
           --verbose \
           unstable $HOME/.cache/sbuild/sid.standard.tgz \
           "http://mirrors.sustech.edu.cn/debian/ unstable"

# In the debianized source directory:
DEB_BUILD_OPTIONS="parallel=16" \
sbuild --chroot-mode=unshare \
       --chroot=$HOME/.cache/sbuild/sid.standard.tgz \
       --dist unstable \
       --chroot-setup-commands='chmod 777 /dev/shm'

# If experimental
SBUILD_CONFIG=$HOME/.experimental.sbuildrc DEB_BUILD_OPTIONS="parallel=8" \
sbuild --chroot-mode=unshare \
       --chroot=$HOME/.cache/sbuild/sid.standard.tgz \
       --dist experimental \
       --chroot-setup-commands='chmod 777 /dev/shm'

The extra content of ~/.experimental.sbuildrc is:

$extra_repositories = [ 'deb http://mirrors.ustc.edu.cn/debian/ experimental' ];
$build_dep_resolver = 'aptitude';

To log into the shell after build failure or before the build directory prune, add '%SBUILD_SHELL' as the value of the following commands:

  • --build-failed-commands, run if any error in the building process
  • --anything-failed-commands, run if any failure occurs
  • --chroot-cleanup-commands, after the build and test, just before the building directory to be pruned

Note duplicated packages will be fetched when building a software, so we harness apt-cacher-ng as the proxy to cache the content that will be downloaded repeatedly. When creating the sbuild environment, we should specify the proxy address (http://127.0.0.1:3142 here, not https).

If a package depends on something not in the mirror (e.g., still in the NEW queue), you can specify them to sbuild explicitly via --extra-package option.

References:

  • https://stephan.lachnit.xyz/posts/2023-02-08-debian-sbuild-mmdebstrap-apt-cacher-ng/
  • https://manpages.ubuntu.com/manpages/focal/en/man5/sbuild.conf.5.html
  • https://wiki.debian.org/sbuild#Enabling_experimental
  • https://manpages.debian.org/unstable/sbuild/sbuild.1.en.html

autopkgtest for runtime tests

Create the image for later runtime test:

sudo autopkgtest-build-podman -m http://mirrors.ustc.edu.cn/debian --vendor debian --release unstable -p http://10.88.0.1:3142

where the apt-cacher-ng service IP is the host IP (gateway) inside the podman network bridge:

podman network inspect podman --format "{{range .Subnets}}Subnet: {{.Subnet}}, Gateway: {{.Gateway}}{{end}}"

You have to run the autopkgtest-build-podman command with sudo, as rootless podman containers have no permission to access the host network.

After successfully building a package, invoke the regression tests with podman backend like this:

sudo autopkgtest -B -s \
	../jupyter-cache_1.0.0-1\~exp1.changes \
	-- podman autopkgtest/debian:unstable

You should define the test commands in d/test/control.

  • https://manpages.ubuntu.com/manpages/jammy/en/man1/autopkgtest-build-docker.1.html
  • https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst

For ROCm packages, you should build a backend with GPU supported like this:

# See https://apt.rocm.debian.net/ to update apt sources
apt install pkg-rocm-tools rocm-podman-support
sudo rocm-podman-create -r experimental -m http://http://10.88.0.1:3142/debian
# Test the GPU-enabled backend
sudo rocm-podman-run --rm -it rocm/debian:experimental ls -l /dev/kfd
sudo autopkgtest -B -s ../rocm-llvm_6.1.2+dfsg-1\~exp1_amd64.changes -- podman+rocm rocm/debian:experimental

Download the ROCm related utilities from ROCm team apt site: https://apt.rocm.debian.net.

If you want to specify the "release" (experimental, sid, stable, etc.) of some specific packages, use --pin-packages option, .e.g, --pin-packages=experimental=rocminfo,hipcc,src:rocr-runtime forces binary packages rocminfo and hipcc, as well as all packages stem from source package rocr-runtime, are fetched from the experimental archive.

Misc. commands

  • List the files contained in a .deb file:

    dpkg -c xxx.deb

  • Decompress the .deb file into a directory:

    dpkg -x xxx.deb ./yyy

  • Remove rpath header in the generated binaries:

    chrpath -d path/to/binary

    Note that Debian discourages the presence of rpath in the ELF files: https://wiki.debian.org/RpathIssue.

  • List package dependencies:

    dpkg -I xxx.deb, #or apt-rdepend xxx