Learnings from the Debian/Ubuntu Package

Package Maintenance
, ,
1

I’ve noticed that some of the upstream Debian patches patch out some Atuin features that should be gated behind feature-flags. We already should have some feature flags that allow for compiling out specific features:

  1. disable-atuin-ai.patch should be gated through an explicit --feature ai.
  2. disable-benches.patch should only be enabled with the appropriate flag.
  3. disable-opentelemetry.patch patches out local atuin opentelemtry and it’s only gated through a profiling-traced build.
  4. disable-vendor.patch disables the vendored version of reqwest.
  5. disable-tree-sitters.patch disables the tree sitters atuin relies on for syntax highlighting.
  6. disable-self-update.patch disables automatic updating.
  7. disable-other-platforms.patch disables Windows builds.

I am wondering whether we could make it easier for @ncts to build Atuin with some sort of build flag array cargo build --no-default-features --features ... and help avoid maintaining the large array of upstream patches. I researched this and it seems the root cause is how debcargo resolves dependencies — even if a dependency is optional = true, debcargo will check to find an appropriate dpkg for it.

There are some clear improvements on our end we could do:

  • The syntax highlighting is not gated behind a feature flag. It should be.
  • Replacing daemonize with daemonix seems like a no-brainer. Done in #4099.
  • Loosening the dependency version restrictions seems like it’s easy and correct to do.
  • Loosening the rust compiler version should be relatively safe/easy to do too. Done in #4100
  • Simplifying benches by adding required-features to avoid compiling it.

It looks like that’s about it with respect to what we could do on our end to make packaging just a smidge easier for Debian/Ubuntu.

Let me know if I’m missing something, @ncts!

Thanks!

3 Likes
2

Thanks for taking the time to go through the patches!

The patch series can be roughly split into these categories:

  • Version mismatch (the one in Debian is too old or too new)
    • This includes Rust (or rustc) itself, which roll-back-rust-version-to-packaged.patch is for
    • relax-deps.patch, which “relaxes” dependencies versions to what we have; it’s a collection of simple version changes that don’t need code changes
    • those downgrade/upgrade-foo-previous-version-new-version.patches, which do require code change
  • Dependency not packaged yet
    • disable-clap_complete_nushell.patch, well, we do have this crate, but Nushell itself isn’t in Debian so it’s not very useful and kept there
    • disable-metrics.patch, lacking metrics-exporter-prometheus
    • disable-opentelemetry.patch, similarly, lacking opentelemetry and friends
    • disable-tree-sitters.patch, where we do have tree-sitter-bash but not tree-sitter-fish, and I wanted to go for it when both are there (just lazy)
    • disable-atuin-ai.patch, lacking the eye_declare set; I wasn’t sure which of the ways to package Rust things in Debian to use
  • Doesn’t fit
    • disable-other-platforms.patch, obviously, Debian doesn’t run on a (native) Windows system, and no one has yet tried to cross-compile for Windows on Debian
    • disable-self-update.patch, since Debian handles updates
    • disable-benches.patch, since benchmarks are mostly for upstreams to establish their own development baselines, while we work on distribution
    • disable-vendored-bash-preexec.patch, since Debian tries to avoid vendoring and use packaged versions; also since bash-preexec IIRC is less preferable than ble.sh, the latter is packaged
    • disable-vendor.patch, same reason, for bundled TLS
  • Replacement (with in-Debian or better alternatives, because sometimes it’s easier to patch the code and use packaged alternatives than going through NEW (though NEW is quite fast these days), and sometimes there is a security advisory)
    • replace-daemonize-daemonix.patch, the latter case, because an unmaintained crate advisory was issued for daemonize

(The concrete patch names are just the currently applied ones, mainly so you can get an idea.)

These categories can then be tackled differently by their nature:

  • Version mismatch
    • No need to worry too much for downgrades, since it’s the distro team lacking behind, and distros vary, there’s no way to fit everyone. We frequently organize upgrade transitions in Debian for key packages/crates like syn/toml/reqwest/hyper; this requires significant effort and coordination, so it’s natural to lag for a while.
    • Upgrades, on the other hand, are much like your upstream development work; should be upstreamed as soon as possible. I’d mostly just been lazy sending them.
  • Dependency not packaged yet: No need to worry too much, as distros would eventually catch up (ideally).
  • Doesn’t fit: usually only the distro’s or even the maintainer’s concern. As long as you do it in established and/or reasonable ways, I don’t think there’s too much to worry for the downstream.
  • Replacement
    • Security advisories should obviously be upstreamed; but
    • Temporary workarounds for unavailable deps with Debian packaged ones are inherently Debian-only and should not be upstreamed.

Feature flags would help with selectively enabling features; they’d simplify patches to exclude a feature flag and removing its dependencies, rather than sifting through and patching out all code using them. Still, you are right that even with optional = true, a crate needs to be available locally; it’s cargo behavior, though, not debcargo. Besides, debcargo merely generates corresponding Build-Depends and Depends following cargo’s requirements. Anyway, the Debian package of atuin didn’t use debcargo.

I appreciate the consideration and effort to make distro maintainers’ lives easier, as you are always very friendly and responsive to our questions. I’d say the current patch set is manageable from our side. A hypothetical major concern from my perspective is changes in architecture and organization of the project that collide with certain aspects of distro packaging specifics, which would be a real time sink, but this has not manifested so far.

(Sorry for the late reply, there was a surge of work in real life.)

1 Like