debian-med-logo

Table of Contents

Introduction

Debian Med is a “Debian Pure Blend” with the aim to develop Debian into an operating system that is particularly well tailored for the requirements for medical practice and research.

The Debian Med project presents packages that are associated with medicine, pre-clinical research, and life sciences. Its developments are mostly focused on three areas for the moment: medical practice, imaging and bioinformatics.

Over the previous years, several initiatives have spawned that address the scientific disciplines like chemistry or bioinformatics. Debian Med is not a competition to these efforts but a platform to present the packages to the community as a Debian Pure Blend.

How to Contribute

The Debian Med team is open for everybody to contribute. All team packages are LowThresholdNmu. Every newcomer who is interested in a specific package is kindly invited to put an additional ID to the Uploaders field and push commits to the master branch of the package.

From the developer to the user, there is a long chain of tasks in which we always welcome participation. First we must keep ourselves informed about the software landscape in biology and medicine. Software to be packaged is chosen according to criteria such as users' need and the consistency of the distribution.

Once in Debian, the software is monitored for its quality and bugs are fixed, if possible in collaboration with the upstream maintainer(s). All this work would not be very useful if it remains confidential.

We also dedicate some time to advertise it to the world via www.debian.org and to ease the integration of new members.

Please contact us on debian-med@lists.debian.org if you want to help to make medical and biological software available to Debian users. Read the Membership section if you're interested in joining us.

If you speak a language other than English, you can contribute rightaway with translations of package descriptions at ddtp.debian.org.

When working on these, you will find immediate targets for improvements of the original English versions, too. For these, though, you need access to Debian Med's source code repository. Very welcome are tutorials that guide Debian users towards the use of packages to their immediate benefit. You may also consider to write respective articles for Magazines, be they online or in print. Or just help point the developers of a software to the Debian package and suggest respective link to their home page, maybe supported by a badge from https://badges.debian.net/.

Communications

The official mailing list of the Debian-Med team is debian-med@lists.debian.org ; you can read the archives of the list and subscribe at https://lists.debian.org/debian-med/. The official chat room of the Debian-Med team is https://matrix.to/#/#debian-med:matrix.org.

Like all Debian spaces, the mailing list and the chat room of the Debian-Med team is subject to the Debian Code of Conduct.

Membership

To request membership to this group, please go on our page on Salsa, or directly follow this link. Remember that you must have an account on Salsa.debian.org before requesting membership (see here to request an account on Salsa.debian.org).

Essential readings

Contributing to this Policy

This policy is itself maintained in Git and all team members have write access to its repository. Clone it with:

git clone git@salsa.debian.org:med-team/policy.git

Make changes to policy.rst, commit and push.

To publish a new version, refresh the HTML document with the make policy command, commit and push it.

Behind the scenes, a GitLab Continuous Integration & Delivery job will grab the updated HTML file and copy it to https://med-team.pages.debian.net/policy/. Thus, here will be a small delay before the changes are visible.

Repositories

We use Git repositories, hosted by Debian. You can have a look at each repository through Salsa's web interfaces.

Give me the source!

To check the sources of a package (referred as <package> below) in our repositories, use the debcheckout command, from the devscripts package.

  • If you are a member of Debian Med or a Debian developer, with account name <username>, you have write permission.

    If the package is already in the archive:

    debcheckout --user <username> --git-track '*' <package>

    For draft packages that are only on Salsa:

    gbp clone <username> git@salsa.debian.org:med-team/<package>.git

    For packages managed with Git, the option --git-track ensures that the clone has all the branches tracked. This is needed when using the gbp buildpackage helper. Alternatively to debcheckout, the command gbp clone will also track the relevant branches:

    gbp clone git@salsa.debian.org:med-team/<package>.git

  • For read-only access with debcheckout, remove the --user option. With gbp clone, use an anonymous URL like the following:

    gbp clone https://salsa.debian.org/med-team/<package>.git

Common Git repository structures

gbp buildpackage

Most of our packages using Git and stored on Salsa are managed following the standard layout of the gbp buildpackage helper. The master branch contains the full source package. The upstream branch contains the upstream source, after repacking when necessary (non-free files, large convenience code copies, …). The pristine-tar contains the data necessary to recreate an original tarball from the repository with a reproducible checksum.

Debian releases are tagged with names like debian/debianversion and upstream releases are tagged with names like upstream/upstreamversion.

Social Git

For some projects, like the ones hosted on GitHub or GitLab, it may be easier to forward changes made in the Debian package if this one is itself mirrored on the same platform, as a clone. In that case, the layouts may vary from package to packages, and the branch that contains the Debian work will probably not be the master one. If it is not the default branch, It must be indicated in the VCS URL with the -b option.

Even when you are not using gbp buildpackage, please include a `debian/gbp.conf <#debian-gbp.conf>`__, to document the layout of the repository.

Other branches

Changes uploaded to other distributions than unstable can be stored in other branches, named for instance experimental, stable, etc.

Git tips

Set the devscripts variables DEBEMAIL and DEBFULLNAME

debcheckout will then set git's options user.email and user.name accordingly.

Configure Git to commit using your packager name and address.

The --global option is to say Git these are the default parameters for every Git repository you commit to, without it the settings will be per-repository only:

git config --global user.name "$DEBFULLNAME"
git config --global user.email "$DEBEMAIL"

To create a new local git repository

Before you create a new git repository you probably want to check whether your target project just exists. In the Gitlab interface of Salsa you can seek for software packaged by the Debian Med team.

When the upstream sources are distributed as compressed tar archives (tar.gz, …):

mkdir package
cd package
git init
gbp import-orig --pristine-tar /path/to/package_version.orig.tar.gz
git clone https://salsa.debian.org/med-team/community/package_template.git /tmp/package_template
mv /tmp/package_template/debian . ; rm -rf /tmp/package_template

The above steps will create a repository with the appropriate layout for gbp buildpackage , with three branches: master (where the Debian development will happen), pristine-tar used by the pristine-tar tool during the package build process to recreate the original tarball, and upstream , which will contain the upstream source. .. _debcheckout-git-track:

To clone and follow every branch of a git repository.

When the package is already in the Debian archive, you can use the debcheckout command with its --git-track='*' option.

To update the upstream, master and pristine-tar branches at once, use the gbp pull.

To track extra upstream branches, simply check them out.

With recent versions of git, the remote branch will be automatically tracked when running git checkout. For example, when a pristine-tar branch is available upstream and not yet tracked locally, the command git checkout pristine-tar will implicitly run the command git branch -t pristine-tar origin/pristine-tar.

To create a pristine-tar branch when it is missing.

See the documentation of git-buildpackage for details. Use a similar command for any other missing branch.

git checkout --orphan upstream
git rm -rf .
git commit --allow-empty -m 'Initial upstream branch.'
git checkout -f master

Creating a new repository on Salsa

Before pushing to salsa.debian.org for the first time, an empty repository needs to be created there in the Gitlab interface.

Each package is kept in its own Git repository. Now, on your local machine add the salsa repository as a remote:

git remote add origin git@salsa.debian.org:med-team/<pkg>.git

To change the default branch.

If the Debian work is not on the master branch, change the default branch by editing the HEAD file in the bare repository on Salsa, and replace master by the name of the new default branch. For a branch called debian/unstable the contents of the file will refs/heads/debian/unstable.

To push the package.

(make sure you've added the salsa remote!), do the following: git push origin master. For the first push, it's necessary to specify origin master. The next time you will push, a git push will suffice. Or use the --set-upstream option, helps future use of git pull.

git push --set-upstream

To push all your work

Be sure to also do a run git push with --all, and one with --tags if you created new tags.

git push --all --set-upstream
git push --tags

To tag a release

git tag debian/x.y-z. You can also easily retroactively make tags: git tag debian/x.y-z <commit hash>. Remember to git push --tags.

If upstream manages their sources with Git.

The following makefile script can help producing a version number when no Git tag is available:

SOURCEPKG=$(shell dpkg-parsechangelog | sed  -n 's/^Source: \(.*\)/\1/p')
UPSTREAM=$(shell dpkg-parsechangelog |  sed -n 's/^Version: \(.*\)-[^-]*/\1/p')
SHA1=$(lastword $(subst ~g, ,$(UPSTREAM)))
ORIG=${SOURCEPKG}_${UPSTREAM}.orig.tar.gz

describe-current-version:
    git describe --tags upstream | sed 's,^release-,,;s,-,+,;s,-,~,;'

get-orig-source:
    git archive --format=tar $(SHA1) | gzip -9 > ../$(ORIG)

To make gbp buildpackage build the package with pdebuild.

Add the following to the configuration file ~/.gbp.conf or debian/gbp.conf:

[DEFAULT]
builder = ~/bin/git-pbuilder

pristine-tar = True
pbuilder-options=--source-only-changes

With this configuration file you're specifying that gbp buildpackage will use ~/bin/git-pbuilder as the builder script. This is an example script you can use:

#!/bin/sh
set -e

pdebuild --pbuilder cowbuilder --debbuildopts "-i\.git -I.git $*"
rm ../*_source.changes

This will build the package inside the default cowbuilder chroot, while passing any more parameters directly do dpkg-buildpackage.

Other options in ~/.gbp.conf:

  • pristine-tar = True: specifies that we are using the pristine-tar branch to store the necessary information about the original source tarball in pristine-tar
  • pbuilder-options=--source-only-changes: Creates a source changes file to do source only uploads. That's extremely important since for Debian 11 only source uploads are allowed.

Updating a source package managed with Git

Most source packages maintained as Git repositories in Debian Med are using the gbp buildpackage helper toolkit. In doubt, try this one first.

gbp buildpackage's command gbp import-orig makes it easy to update of the upstream branch using upstream releases distributed as a compressed archive, and to merge these changes in the master branch. Its option --pristine-tar is useful for stabilizing the MD5 sum of the “orig.tar.gz” produced when building a source package from the repository alone (not doing so results in archive rejection of package updates). With recent versions of gbp buildpackage, it is often unnecessary to rename the freshly downloaded original upstream archive.

Assuming you use gbp buildpackage which is strongly recommended in the Debian Med team you want to import the new upstream source by using

gbp import-orig --pristine-tar --no-interactive PACKAGE_VERSION.orig.tar.gz

If you do not use gbp buildpackage, please use pristine-tar anyway to register the compressed archives that you upload to Debian as original upstream sources (the orig.tar.{gz|bz2|xz} file), with a command such as the following, where tarball is a path to the archive file and tag is the upstream release tag for that archive (as a fallback, if you created the tarball from a given branch, use the name of the branch).

pristine-tar commit tarball tag

Commit messages are sent to our commit mailing list with the git-multimail tool. To avoid sending too many messages, consider setting the option multimailhook.maxCommitEmails to a low value, for instance 20; the default is 500…

There is also a more or less automated workflow using the routine-update package. It provides a shell script which usually does what you want to upgrade some package to the latest upstream version.

Packaging

Newcomer guidelines for building proper Debian packages

Some newcomers tend to go the create DEBIAN dir, move files around and `dpkg-deb -b` way to create Debian packages. Short answer: Forget about this. The only way to the official Debian mirror leads via proper source packages. The right way to build Debian packages is described in Debian New Maintainers' Guide.

See also the package template on Salsa.

There is also the option of joining the Mentoring if the Month as student to get a personal training how to start working inside the Debian Med team.

Announcing intent to package

If you intent to work on a Debian package you should follow the normal Debian rules and file a WNPP bug report.

It is a good idea to keep the Debian Med mailing list debian-med@lists.debian.org in CC and to set it as the owner of the ITP to keep your co-workers informed. This will ensure that we notice if for some reason the package has not been uploaded.

In addition, please add the package to the “task” file where it fits the best, and document your ITP number using the WNPP field name.

Backports

Debian offers backports to provide up-to-date software for users of the official stable releases. Backports of Debian Med packages should be kept as branches in our Git repositories. The branch should be named using the release code names. As an example, the backports branch for Debian 9 ("stretch") should be named debian/stretch-backports, in accordance with DEP-14. If you are developing on Stable, make sure to get the devscripts package from stretch-backports for dch --bpo to set the version properly.

In Git, preparing a backport can be as simple as:

git checkout debian/VESIONTAG
git checkout -b debian/stretch-backports
dch --bpo

And then building the package using a Stable chroot after making any necessary adjustments to the packaging. See the wiki on backports and the main backports website for more detailed instructions.

When preparing subsequent versions of a backport, the changelog should include all the changes since the previous backported version. You will want to make sure you set up your cloned repository to use dpkg-mergechangelogs (from the dpkg-dev package) to avoid getting unnecessary merge conflicts. See its manpage section "Integration with Git" for how to set that up.

git checkout debian/stretch-backports
git merge debian/TO_BE_BACKPORTED_VERSION
dch --bpo

Now, you can try building against a Stable chroot.

PPA for Ubuntu

Debian Med operates a Personal Package Archive (PPA) on Launchpad, where packages are backported for Ubuntu. There is currently no team policy on what to build.

Because the space in the PPA is currently limited (4.0 GiB), we keep in the PPA only packages for still maintained LTS Ubuntu version. Packages targeting an older Ubuntu will be removed.

Please keep in mind issues like the possibility to upgrade to the next Ubuntu stable release. Packages that are backports can be made inferior in version by using a tilde. If the package contains additional development, a version number without the tilde will make it higher, but not as high as the next Debian revision. For example: 2.12.0-1~natty1 (backport in PPA) < 2.12.0-1 (from Debian in Ubuntu) < 2.12.0-1natty1 (in PPA, containing additions) < 2.12.0-2 (from Debian in Ubuntu). Please follow the Ubuntu's recommendations for versioning packages in PPA.

Packages sent to this PPA may be kept as branches in our repositories.

Lintian checks are adapted to Ubuntu by setting the environment as follows: LINTIAN_PROFILE=ubuntu.

Debian unstable chroot on Ubuntu

cowbuilder-dist sid create cowbuilder-dist sid login

Derivatives working together with Debian Med

Debian Med is proud that derivatives (like for instance Bio-Linux) are profiting from our work inside Debian and we try to establish strong connections to these derivatives. With Bio-Linux the connection is as strong that a common workflow was created where Bio-Linux developers are injecting their packaging straight into the Debian Med version control system. To make sure that there will be no conflicts with the Debian revisions some attention should be payed to the revision numbering. If the derivative is creating a new package (either from scratch or an upgraded version) it should get the version <upstreamversion>-0<derivativename><derivativerevision> (which is the versioning scheme usually used by Ubuntu).

R packages

Debian R packages should be maintained within the Debian R Packages Team.

GNU R sometimes introduces backward incompatibilities, so the current practice is to make packages depend on versions equal or higher to the one against which they were built. When using r-base-dev, this can be acheived by adding the substitution variable ${R:Depends} in the Depends field of the binary package.

Tasks

The Debian Med Debian Pure Blend is organised by tasks, that group packages around broad themes such as medical imaging for instance. The tasks list programs that are already packaged in Debian as well as packages in preparation.

The tasks files are not hosted in the Debian Med repositories, but in the Debian Blends repository. Nevertheless, all members of the Debian Med project on Salsa have write access to the Blends Git repository. You can easily check out its sources with the command debcheckout -a debian-med.

The syntax of the tasks files is very similar to Debian control files, and described in the Debian Blends website.

Policy

debian/control

  1. Section.

    Should be “science” for the source package.

  2. Priority.

    Should be “optional” unless forbidden by the Debian policy (see §2.5). Packages of priority “extra” are excluded from some QA tests.

  3. Maintainer.

    Maintainer should be Debian Med Packaging Team debian-med-packaging@alioth-lists.debian.net. Please subscribe to this list if you list yourself in the Uploaders: field of one of Debian Med's packages. You can refer to the QA page corresponding to this email to gather information about the packages.

  4. Uploaders.

    Please add yourself as an uploader when you have a significant interest in a package. Being Uploader means that you are expected to answer to the bug reports. For more occasional works, you can do a team upload.

  5. Build-Depends.

    For new packages of tools that produce architecture-dependant binary packages (anything besides Architecture: all), then the following dependencies SHOULD be listed to skip building on non-64-bit and non-little-endian architectures: architecture-is-64-bit, architecture-is-little-endian.

  6. Standards-Version.

    Please always use the latest unless there are concerns for backporting. If no changes are needed, please indicate this fact in the changelog, and increment the value of the field.

  7. Homepage.

    should be documented whenever possible

  8. Vcs-Git: and Vcs-Browser:.

    Please use the following templates, and refer to the Debian Policy § 5.6.26 for details:

    Vcs-Browser: https://salsa.debian.org/med-team/<package>
Vcs-Git: https://salsa.debian.org/med-team/<package>.git

  9. Testsuite: autopkgtest.

    Field and value to declare that a testsuite compatible with autopkgtest is available. Such testsuite can be executed via the adt-run command from the autopkgtest package or the sadt command from the devscripts package.

  10. Rules-Requires-Root.

    Should be no, if possible.

It is a very good idea to use Config::Model to unify the formatting of debian/control. To do so make sure you have installed

apt-get install cme libconfig-model-dpkg-perl

and than you can simply call

cme fix dpkg-control

to get a properly formated, sanity checked debian/control file. Please note that sometimes you need to call this more than once. In case you want to use the cme GUI you also need to

apt-get install libconfig-model-tkui-perl

which enables you to do something like

cme edit dpkg

debian/changelog

Packages hosted in our Git repository, that have been modified but not uploaded must use UNRELEASED as a distribution name. This can be done automatically by declaring DEBCHANGE_RELEASE_HEURISTIC=changelog in ~/.devscripts and using dch.

debian/upstream

We use the bibliographic information which should be stored in the file debian/upstream. The purpose of specifying this is to enhance the contact to upstream which thus gets an extra reward of their work if their citations show up on pages inside the Debian domain and if users more popularly are asked to cite upstream when working with the program in question.

A new development are registries of research software (e.g. SciCrunch's RRID, bio.tools, OMICtools). The metadata allows for references to publications and entries in these registries alike.

debian/gbp.conf

Include this file to document the layout of the repository. Packages managed with gbp buildpackage may omit default values.

[DEFAULT]
# The default name for the upstream branch is "upstream".
# Change it if the name is different (for instance, "master").
upstream-branch = upstream
# The default name for the Debian branch is "master".
# Change it if the name is different (for instance, "debian/unstable").
debian-branch = master
# gbp import-orig uses the following names for the upstream tags.
# Change the value if you are not using gbp import-orig
upstream-tag = upstream/%(version)s
# Always use pristine-tar.
pristine-tar = True

debian/README.source

This file is recommended by the Policy (§ 4.14) from version 3.8.0 for documenting source package handling. Please follow the recommendation. For instance, this file is needed when we use a patch system, when the upstream sources are in another format than gzipped tar archive, when we repack the sources,…

debian/README.test

This file was (recommended by the Security team) for describing to others than the regular maintainer how the package's functionality can properly be tested.

debian/source/format

This file sould contain “3.0 (quilt)” in order to use this source format. Other formats should be avoided unless they bring a specific advantage.

debian/source/options

For packages not using quilt patches, for example when committing changes directly to the Debian branch, this file should contain “single-debian-patch” in order to emulate the 1.0 format. This is better than using the 1.0 format directly because the 3.0 (quilt) format brings other advantages, in particular the conservation of file permissions in the debian directory.

debian/salsa-ci.yml

To run continuous integration tests at each push in our Salsa forge, the Salsa CI team provides a pipeline that can be activated by including some template files available from their repository. The simplest and recommended way is to put the include statements in a debian/salsa-ci.yml file, and configure the repository to recognise that file, as documented in the `salsa-ci-team/pipeline repository's README file. It may happen that CI pipeline configuration is stored in debian/gitlab-ci.yml instead, for project which not necessarily used routine-update for setting the initial file.

The usual debian/salsa-ci.yml fresh out of routine-update will look like:

---
include:
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/pipeline-jobs.yml

If for some packages, an attempt to build on 32 bits architecture is a waste of precious CI runner CPU cycles, then it is possible to disable such test entirely:

---
include:
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/pipeline-jobs.yml

variables:
  SALSA_CI_DISABLE_BUILD_PACKAGE_I386: "true"

Again, refer to the README provided by the Salsa CI team for more options.

debian/symbols

Symbols files in libraries usually require more effort for maintenance. However, these can be very important to detect ABI changes that are requiring bumping the soname and thus the library package name. Since not all upstream are dealing with sonames a symbols file should be provided if possible. (It might be sufficient to provide a file symbols.amd64 and by doing so reducing the maintenance effort of maintenance to amd64 architecture only.)

The KDE team has written a helper that could simplify the creation and maintenance of symbols files. The manpage for this helper is available online.

Debhelper

Debhelper uses compatibility levels to control the behaviour of its commands. We currently recommend to use the level 10 which is available in current Stable and backported to Oldstable. However, there is no urgent need to touch packages only because it has an older Debhelper version.

It is strongly recommended to use the short dh notation in debian/rules files which makes code factorisation very simple and easy to understand the packaging for other members of the team. Even complex packaging becomes quite transparent this way.

Version control systems

Source package stored in a Git repository on Salsa

Git repositories should be stored on Salsa.

Tags

Tags indicate the revision corresponding to uploaded packages. For a released package debian/ is added before the package version number.

New package

Try to inject a new package only after successfully building it with dpkg-buildpackage (or any wrapper around it). Use a file like debian/DRAFT to mention when the package is a draft.

The Debian Med Blend tasks

Once you injected a new package please make sure that it is mentioned in the appropriate tasks file in the Git source of the debian-med Blend package. Some team members watch the changes in the Debian Med packaging pool but it helps if the maintainer of a new package verifies that everything is in the right place.

Building and tagging the packages

We prefer that uploaded packages are built in a chroot, to provide similar build environment to the whole team. After upload, please tag the Git repository.

Handling patches

Often happens that the upstream code doesn't fit well into the Debian distribution: be this wrong paths, missing features, anything that implies editing the source files. When you directly edit upstream's source files, your changes will be put into a .diff.gz file if you use the 1.0 source format and in a monolithic patch if you use the 3.0 (quilt) format. To better organise the patches and group the by function, please use a patch handling system which keeps patches under the debian/patches directory.

The 3.0 (quilt) Dpkg source format provides its own patch system. You can use it with the quilt command.

Using quilt

Using quilt is rather easy.

First, make sure you have correctly setup quilt: open .quiltrc in your home directory (create it if you don't have one), and make sure it looks like this:

QUILT_DIFF_ARGS="--no-timestamps --no-index"
QUILT_REFRESH_ARGS="--no-timestamps --no-index"
QUILT_PATCHES="debian/patches"

After this, you're ready to start working with quilt. See also the instructions in the New Maintainer's Guide.

Creating a patch

To create a patch, use the new command. Run:

quilt new <patch_name>.patch

This will create (if it doesn't exist yet) a debian/patches/series file, which contains all the patches to be applied by quilt. Moreover, the new patch is also the topmost (the currently applied).

Now start editing files, with:

quilt edit <file>

and repeat the process for each file the patch is involved with. At the end, run

quilt refresh

This will compare the noted state of the edited files with the current state, and will produce a patch in debian/patches. Remember: the patch is currently applied (you can check this with quilt applied).

Make sure to write the patch header and use the DEP3 format:

quilt header -e --dep3

Applying and unapplying patches

Just two easy commands to do the job:

  • quilt pop will unapply the topmost patch.
  • quilt push will apply the next patch in debian/patches/series.

You can just add a "-a" flag to the commands above, to respectively apply/unapply all patches in the series.

Tip

You can check which patches are applied/unapplied with, respectively, quilt applied and quilt unapplied.

Editing patches

To edit a patch, first make it the topmost:

quilt push <patch_name>

If the patch is already applied, but is not the topmost, run quilt pop until it becomes the currently applied one.

You can now run quilt edit on the files you want to change, and, when you're done, quilt refresh.

Renaming patches

Sometimes it's useful to rename a patch. Without any hassle, do:

quilt rename -P <old_name>.patch <new_name>.patch

Updating patches

After updating a package to a new upstream version, it can happen that the patches do not apply cleanly anymore. Try quilt pop -a, to remove all patches, and then quilt push one patch at a time.

When there was any fuzzing, do quilt refresh before the next quilt push.

If a patch does apply cleanly because it is not needed anymore (applied upstream, etc.), it can be removed with quilt delete <patchname>.patch && git rm debian/patches/patchname (assming the source package is managed with Git, which is the case in our team).

If the patch is still needed, force its application with quilt push -f, editing the files indicated while reviewing the corresponding *.rej files, and then update the patch with quilt refresh.

Other commands

Please see man 1 quilt to have a comprehensive list of commands and the UsingQuilt page on the Debian Wiki.

Embedding Large Test Data

Often a lot of times, there's a need to fetch external data for adding autopkgtests. We choose to make a separate orig tarball for data if the size of data far exceeds the size of package.

Step - 1:

Update gbp.conf with the following:

[DEFAULT]
pristine-tar=True
filter=[ '.gitignore', '.travis.yml', '.git*' ]
component=['debian-tests-data']

Step - 2:

Usually the orig tarball of existing package is: pkgname_version.orig.tar.xz or pkgname_version.orig.tar.gz The data tar should be named: pkgname_version.orig-debian-tests-data.tar.xz or pkgname_version.orig-debian-tests-data.tar.gz respectively.

Migrate outside the source directory, Simply fetch the data(from external source) and make the tarball: tar cJf pkgname_version.orig-debian-tests-data.tar.xz data

Step - 3:

Delete the corresponding upstream tag pertaining to the current version (if already present): git tag -d upstream/version Import the orig tarball with: gbp import-orig --pristine-tar ../pkgname_version.orig.tar.xz

NB: Make sure that pkgname_version.orig.tar.xz and pkgname_version.orig-debian-tests-data.tar.xz

After this, you should see a debian-tests-data in the source tree and running origtargz should also generate a data tarball. Simply use the debian-tests-data to seek data for running autopkgtests:

Either move files from this directory to AUTOPKGTEST_TMP or install it in a separate binary.

Packages: stringtie and radiant can serve as examples here. For further reading, check Javascript Team Embedding wiki

FAQ

How to easily help upstream exactly reproduce build/test errors that may be specific to Debian?

For packages already uploaded, you can adapt the Dockerfile below to share with upstream so they can see the error and be able to inspect and troubleshoot with the exact same versions and patches applied:

FROM docker.io/debian:sid-slim

WORKDIR /build

RUN apt-get update && apt-get install -y --no-install-recommends devscripts curl debian-keyring
RUN dget https://deb.debian.org/debian/pool/main/s/sunpy/sunpy_5.1.1-4.dsc
WORKDIR /build/sunpy-5.1.1
RUN apt-get -y build-dep .
RUN dpkg-buildpackage -uc -us -b

Be sure to change the RUN dget line and the second WORKDIR /build/ line to match the package.

The Dockerfile even works for testing non-amd64 systems with the --platform option to docker build (or podman build). The platform name is linux/arm/v5 for the armel architecture, linux/arm/v6 for the armhf architecture, and linux/mips64le for mips64**el**.

What to do if a large package does not build on a specific autobuilder architecture?

Some of our target packages are large regarding memory consumption or requiring more computing power to build on not so powerful architectures. Since the Debian autobuilder infrastructure consists of differently equiped hosts which are picked at random it might help to ask at

<arch>@buildd.debian.org

to blacklist the weaker candidates.