Support for uv lockfiles as an alternative to pex lockfiles (cherry-pick of #23302)#23320
Merged
benjyw merged 2 commits intopantsbuild:2.32.xfrom May 6, 2026
Merged
Conversation
…ld#23302) uv lockfiles are first-class entities that users check in. When a pex is needed, we use uv to create a venv, which it does very efficiently, and then use pex's `--venv-repository` feature to build a pex from (a subset of) that venv's requirements. Note that we still create VenvPexes to run tools, and in some cases this might be unnecessary, since we already have the uv venv. But utilizing those venvs directly would be an even bigger change (and this one is already massive), and there are caching tradeoffs to figure out . So we leave that for the future (and probably for the new python backend). Note also that the dynamics of lockfile generation remain pants's (you must generate explicitly) and not uv's (lockfiles are automatically updated for you when input requirements change). The latter is desirable, but again this is for the future. The general thrust of the changes is: - Add uv as a third lockfile format (after pex and the deprecated legacy constraints file "locking") - Update the lockfile metadata to include the format (and also the name of the resolve that created the lockfile). - Make lockfile metadata reading much more robust and useful, so we can get the format from it reliably. Unfortunately this logic is tricky since it has to handle pex lockfiles both with and without separate metadata files. For uv we simply require separate metadata files and do not add any metadata inside the lockfile itself, as we foolishly did for pex lockfiles at one point. - Update the lockfile diffing code to support diffing across all combinations of old/new and pex/uv. - Update the existing `uv` subsystem for more general use. - Add src/python/pants/backend/python/util_rules/uv.py, which runs `uv`, first generating ephemeral pyproject.toml and uv.toml config files. We handle environments correctly, which requires some `realpath` shenanigans. - Update some iffy logic in PythonToolBase that parses lockfiles to find useful info for help messages. It was previously relying on pex lockfile internals (which are not guaranteed to be stable) and it now also relies on uv lockfile internals (similarly not stable). However now at least it fails gracefully if the lockfile parsing fails and just prints a less useful error message. - Use all this in src/python/pants/backend/python/util_rules/pex.py, instead of the previous, uv pex building solution. Note that this supersedes the more limited (but much appreciated) change in pantsbuild#23197 that implemented only the uv venv -> pex part. It also supersedes the competing pending changes in pantsbuild#22949 and pantsbuild#23212, which implemented partial support for uv resolution.
cburroughs
approved these changes
May 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
uv lockfiles are first-class entities that users check in.
When a pex is needed, we use uv to create a venv, which
it does very efficiently, and then use pex's
--venv-repositoryfeature to build a pex from(a subset of) that venv's requirements.
Note that we still create VenvPexes to run tools, and
in some cases this might be unnecessary, since we
already have the uv venv. But utilizing those venvs
directly would be an even bigger change (and this one
is already massive), and there are caching tradeoffs
to figure out . So we leave that for the future (and
probably for the new python backend).
Note also that the dynamics of lockfile generation remain
pants's (you must generate explicitly) and not uv's
(lockfiles are automatically updated for you when
input requirements change). The latter is desirable, but
again this is for the future.
The general thrust of the changes is:
legacy constraints file "locking")
also the name of the resolve that created the lockfile).
and useful, so we can get the format from it reliably.
Unfortunately this logic is tricky since it has to handle
pex lockfiles both with and without separate metadata files.
For uv we simply require separate metadata files and do
not add any metadata inside the lockfile itself, as we
foolishly did for pex lockfiles at one point.
across all combinations of old/new and pex/uv.
uvsubsystem for more general use.which runs
uv, first generating ephemeral pyproject.tomland uv.toml config files. We handle environments correctly,
which requires some
realpathshenanigans.to find useful info for help messages. It was previously relying
on pex lockfile internals (which are not guaranteed to be stable)
and it now also relies on uv lockfile internals (similarly not stable).
However now at least it fails gracefully if the lockfile parsing fails
and just prints a less useful error message.
instead of the previous, uv pex building solution.
Note that this supersedes the more limited (but much appreciated)
change in #23197 that implemented only the uv venv -> pex part.
It also supersedes the competing pending changes in #22949
and #23212, which implemented partial support for uv resolution.