nfpm.native_libs: Add DEB package depends from packaged pex_binaries #22905
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
NOTE on using SOInfo dataclass: We need slightly different things for deb vs rpm. For deb, we need just the soname to search for relevant packages, and parsing the so_info string did not seem wise when I can just preserve the data as elfdeps returned it. So, we now have a SOInfo dataclass (a limited mirror of the elfdeps.SOInfo dataclass).
I reviewed dh_shlibdeps and dpkg-shlibdeps sources so that we can do something similar without all the heavy-weight baggage of native tooling (which includes a complete installation of a specific distro release, because dpkg-shlibdeps uses symbols and shlibs files from other installed packages to identify packages that provide required sonames.) The whole point of nfpm is to be able to build the package for any version of a distro on any version of that distro or some other distro or even on a different OS. So, we necessarily cannot replicate all of the dpkg-shlibdeps logic, but we can do quite a bit thanks to the package search API.
We need to select the relevant package(s) based on the .so file path, following (a simplified set of) standard LD lookup rules. This commit updates the deb_search_for_sonames script and rule so that .so file names are available for that selection process.
This only adds package dependencies without any version constraints. It might be possible to add a version constraint based on the so_version. But, this is good enough for now.
This encapsulates the logic, based on dpkg-shlibdeps, that filters SONAMEs.
And give the tests more time (API calls can be flaky and retries take time. But once cached, the test shouldn't repeat).
Add these fields to the `nfpm_deb_package` target: - `distro` - `distro_codename` These are required as input parameters for this script: - `pants.backend.nfpm.native_libs.deb.search_for_sonames`
cognifloyd
force-pushed
the
cognifloyd/nfpm-native_libs-deb-scripts
branch
from
299d344 to
6789c47
Compare
cognifloyd
force-pushed
the
cognifloyd/nfpm-native_libs-deb
branch
from
328119c to
0b40697
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Series Overview
This is the fourth (and final) in a series of PRs that introduces a new backend:
pants.backend.npm.native_libsInitially, the backend will be available as:
pants.backend.experimental.nfpm.native_libsI proposed this new backend (originally named
bindeps) in discussion #22396.This backend will inspect ELF bin/lib files (like
lib*.so) in packaged contents (for this PR series, only inpex_binarytargets) to identify package dependency metadata and inject that metadata on the relevantnfpm_deb_packageornfpm_rpm_packagetargets. Effectively, it will provide an approximation of these native packager features:rpm:rpmdeps+elfdepsdeb:dh_shlibdeps+dpkg-shlibdeps(These substitute${shlibs:Depends}in debian control files have)Goal: Host-agnostic package builds
This pants backend is designed to be host-agnostic, like nFPM.
Native packaging tools are often restricted to a single release of a single distro. Unlike native package builders, this new pants backend does not use any of those distro-specific or distro-release-specific utilities or local package databases. This new backend should be able to run (help with building deb and rpm packages) anywhere that pants can run (MacOS, rpm linux distros, deb linux distros, other linux distros, docker, ...).
Previous PRs in series
PR Overview
TODO: