Migrate project management to uv #668
Conversation
trying to get report written and read by both autotest and notebook runs
Only verf test
Only metric tests
uv inexact
seems numpy 2.3.5 still deadlocking on ubuntu for python [3.11 .. 3.13] pinning numpy < 2.3.2 for python < 3.14, but > 2.3.4 for python > 3.14
There was a problem hiding this comment.
workflow to publish release -- maybe more motivation here to build some smoke tests
There was a problem hiding this comment.
can be refined after merge
_versioneer.py
Outdated
There was a problem hiding this comment.
can be dropped, no longer using tag based dynamic versioning -- running action on tag push now
There was a problem hiding this comment.
There's also related versioneer cruft in pyemu/_version.py which should be removed
There was a problem hiding this comment.
sufficient to just point to
__version__ = importlib.metadata.version(__name__)
in __init__.py?
There was a problem hiding this comment.
options now in pyproject.toml
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
|
@nathanjmcdougall, you might have some thoughts to add to this PR. |
|
@mwtoews, keen to hear your thoughts on this blitz. |
| 'tomli; python_version < "3.11"', | ||
| ] | ||
| build-backend = "setuptools.build_meta" | ||
| requires = ["uv_build>=0.9.6,<0.10.0"] |
There was a problem hiding this comment.
If you're interested in a uv-compatible way of having dynamic versioning for the project, I could set up hatchling as the build system instead.
There was a problem hiding this comment.
I think static is ok. We were using setuptools before but the idea for uv was to bring it under one tool. It seems to be working fine and we can do the dynamic part from GHA when pushing a tag.
There was a problem hiding this comment.
I'm a personal fan of ["hatchling", "hatch-vcs"] build backend requirements, since they support dynamic versioning. Using UV does not necessary require using a uv_build build backend.
There was a problem hiding this comment.
This is bit of blindspot for me. I was enjoying the idea of stripping out the need to another tool in the project. And that uv build backend is rapid as. Currently we are using setuptools with versioneer to deal with all the dynamic versioning complexity. Moving to uv and the uv_build backend seemed to simplify things a lot but with the cost of moving version to static metadata. I understand that dynamic metadata is a slightly "unpreferred" option in some quarters. The GHA action seems like a good way to tie versioning back to the VCS. I am open to suggestions and improvements to this but most keen on least worst, simplest option!
There was a problem hiding this comment.
You will always need a build backend, it's just a matter of choosing which one. uv_build is treated specially by uv because it doesn't need to spin up Python processes to run the build, it can just run from within uv. That makes it faster than hatchling but in my experience, both are fast enough that the difference is negligible.
Using hatchling with hatch-vcs allows for the same sort of dynamic versioning as versioneer although it's a more modern configuration option. Hatchling used to be the default build backend in uv before uv_build was an option.
This is what hatchling with hatch-vcs config looks like:
[build-system]
build-backend = "hatchling.build"
requires = [
"hatch-vcs>=0.5.0,<0.6.0",
"hatchling>=1.27.0,<1.28.0",
]
[project]
dynamic = [ "version"]
[tool.hatch.version]
source = "vcs"
[tool.hatch.build.hooks.vcs]
version-file = "pyemu/_version.py"Versus with uv_build:
[build-system]
build-backend = "uv_build"
requires = ["uv_build>=0.9.6,<0.10.0"]
[project]
version = "1.4.0.dev3"So they're very similar but you don't need to maintain the .dev3 suffixes manually etc. if you want to do local builds. Personally I'd say that would be the main driver for whether you'd benefit from dynamic versioning - if you like to do a lot of local wheel builds and distribute dev versions, to disambiguate different wheel files.
With hatch-vcs, if you build then it will use git tags to determine the version of the wheel, adding dev suffixes etc. based on the commit hash. It will also create the __version__ value in pyemu/_version.py (or wherever you specify)
| [project] | ||
| name = "pyemu" | ||
| dynamic = ["version"] | ||
| version = "1.4.0.dev3" |
There was a problem hiding this comment.
uv version --bump <BUMP> is a handy hook for keeping this current, but releases won't be triggered until a git tag is pushed to pypest/pyemu (and the tag starts with 'v')
| run: uv publish | ||
| # The part below with testpypi only | ||
| # TODO: remove the next two lines for production usage | ||
| env: |
There was a problem hiding this comment.
currently just pointing to test.pypi
mwtoews
left a comment
mwtoews
left a comment
There was a problem hiding this comment.
I've been using UV for the past year, it's been excellent. A few misc notes below...
_versioneer.py
Outdated
| 'tomli; python_version < "3.11"', | ||
| ] | ||
| build-backend = "setuptools.build_meta" | ||
| requires = ["uv_build>=0.9.6,<0.10.0"] |
There was a problem hiding this comment.
I'm a personal fan of ["hatchling", "hatch-vcs"] build backend requirements, since they support dynamic versioning. Using UV does not necessary require using a uv_build build backend.
_versioneer.py
Outdated
There was a problem hiding this comment.
There's also related versioneer cruft in pyemu/_version.py which should be removed
| uses: actions/checkout@v5 | ||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v7 | ||
| - name: Install Python |
There was a problem hiding this comment.
Why is a python installation step necessary? If using the uv_build backend, my understanding is that the build can be done without Python at all.
| run: uv python install 3.13 | ||
| - name: Build | ||
| run: uv build | ||
| # TODO: add some smoke tests here |
There was a problem hiding this comment.
If you do decide to add smoke tests, I think as a matter of security, you should separate the build+test and publish steps into separate stages. e.g. here
https://github.com/usethis-python/usethis-python/blob/main/.github/workflows/release.yml
The risk is that a compromised test dependency injects a payload into the testing runtime which pre-emptively publishes a malicious package to PyPI, since it's scoped under the id-token: write permission. Technically this risk can exist even in the build step, but the only dependency in this case is uv_build, so the risk is no higher than using uv in the first place.
You can read more about this from the Astral team here
https://docs.pyx.dev/publishing#trusted-publishing
For isolation reasons, we strongly recommend that you perform your distribution build in a separate job that your publishing job depends on. This ensures that your build environment does not have access to the short-lived credentials used for publishing.
In general (in a follow up PR) it might be good to consider using zizmor to shore up the GitHub Actions config a bit, from a security perspective
Moving to uv (https://docs.astral.sh/uv/) for project management, build and deployment.
This includes:
oh and moving coverage tracking to codecov I think this could be decoupled if that is too offensive.
It's probs not perfect but something to build on?