uv: support private-index auth via named indexes, extra_env_vars, and keyring-provider#24
Open
Affanmir wants to merge 2 commits intobenjyw:uv_lockfilefrom
Open
uv: support private-index auth via named indexes, extra_env_vars, and keyring-provider#24Affanmir wants to merge 2 commits intobenjyw:uv_lockfilefrom
Affanmir wants to merge 2 commits intobenjyw:uv_lockfilefrom
Conversation
… keyring-provider When using the new uv resolver from pantsbuild#23302, authenticating to a private index (e.g. GCP Artifact Registry, AWS CodeArtifact, Azure Artifacts) required embedding `user:token@` directly in the index URL because: 1. The generated `uv.toml` only emitted `default = true` / `name = "extra-N"` for indexes — never a stable, user-chosen name. uv's per-index credential env vars (`UV_INDEX_<NAME>_USERNAME` / `_PASSWORD`) need a matching name to bind to, and uv has no name-less fallback form. 2. The uv subprocess only inherited env vars listed in the *global* `[subprocess-environment].env_vars`, which leaks credentials into every other tool subprocess (pytest, mypy, twine, ...). 3. There was no way to opt the generated `uv.toml` into uv's `keyring-provider = "subprocess"` setting, even when `keyring` and a backend like `keyrings.google-artifactregistry-auth` were installed system-wide. This change adds three small, opt-in pieces: - `[python-repos].indexes` now accepts a pex-compatible `name=URL` form (e.g. `"my-index=https://art.example.com/simple"`). The name is emitted as the `name` field of the corresponding `[[index]]` block in the generated `uv.toml`. URLs whose query strings contain `=` are not split. - A new `[uv].extra_env_vars` option that mirrors `TestSubsystem.EnvironmentAware.extra_env_vars`. Variables read here are forwarded only to the uv subprocess. - A new `[python].uv_keyring_provider` option which, when set, emits `keyring-provider = "<value>"` as a top-level key in the generated `uv.toml`. The pex-side keyring gap (pex's internal pip venv can't see system-installed `keyrings.*` backends) is acknowledged but pre-existing and out of scope here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
1 issue found across 6 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/python/pants/backend/python/util_rules/pex_requirements.py">
<violation number="1" location="src/python/pants/backend/python/util_rules/pex_requirements.py:417">
P2: The named-index regex rejects valid mixed-case URL schemes, so a spec like `name=HTTP://...` is misparsed as an unnamed URL and yields invalid uv config.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
The named-index regex required a lowercase scheme on the lookahead, so specs like `name=HTTPS://art.example.com/simple` (or any non-lowercase scheme) fell through `_split_named_index()` as unnamed and were emitted as a literal `url = "name=HTTPS://..."` in the generated `uv.toml` — which uv then tried to fetch as a URL. Per RFC 3986 §3.1, URI schemes are case-insensitive (lowercase is just the canonical form). Widen the lookahead character classes to `[A-Za-z]` so `HTTPS://`, `Http://`, `git+HTTPS://`, etc. all parse correctly. Add a regression test covering four mixed-case schemes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
|
I've implemented this here: pantsbuild#23322 The env var part is basically copied from here (thanks) but the named index stuff is part of a larger change to support pointing specific requirements to named indexes. Feel free to close this and comment there (and also merge that change in and try it out, before I upstream it...) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a PR-on-PR for pantsbuild#23302 (which is open against
pantsbuild:main). Targetsbenjyw:uv_lockfileso it can ride along with that PR.Why
We ran pantsbuild#23302 locally against a private GCP Artifact Registry and hit a 401. The only working workaround was embedding
oauth2accesstoken:$TOKEN@…directly in the index URL — which leaks short-lived tokens into checked-in config and forces an out-of-band refresh loop.Three things conspired to break the standard uv auth paths:
uv.tomlhad no stable index name.ResolveConfig.uv_config()emitteddefault = truefor the first index and auto-generatedname = "extra-N"for the rest. uv's per-index credential env vars (UV_INDEX_<NAME>_USERNAME/_PASSWORD) need a matching name to bind to, and uv has no name-less fallback form, soUV_INDEX_LAAM_SDK_USERNAMEhad nothing to bind to.get_uv_environmentspread[subprocess-environment].env_varsinto the uvProcess.env, but addingUV_INDEX_*there leaks credentials into every other tool subprocess (pytest, mypy, twine, …).keyrings.google-artifactregistry-authinstalled system-wide, there was no option to emitkeyring-provider = "subprocess"in the generateduv.toml.What
Three small, opt-in pieces — all on the uv path. No behavior change for existing users:
[python-repos].indexesnow acceptsname=URL. A pex-compatible form already accepted by pex; we now also use the name when emitting the[[index]]block in the generateduv.toml. URLs whose query strings contain=are not mis-split (regex requires the LHS to be a simple identifier and the RHS to start with<scheme>://).New
[uv].extra_env_vars— uv-scoped passthrough, mirrorsTestSubsystem.EnvironmentAware.extra_env_vars. Forwards listed vars only to the uv subprocess.New
[python].uv_keyring_provider— when set, emitskeyring-provider = "<value>"as a top-level key in the generateduv.toml. Typical value:"subprocess", which has uv shell out to the systemkeyringbinary (works with backends likekeyrings.google-artifactregistry-auth,keyring.codeartifact, …).Out of scope
Pex's pip-resolver venv can't see system-installed
keyrings.*backends — that gap exists inmainand is not introduced by pantsbuild#23302. Filing as a separate follow-up after this lands.Test plan
pants test src/python/pants/backend/python/util_rules/pex_requirements_test.py— all 84 cases pass, including 3 new tests covering named-index emission, the=-in-query-string ambiguity, andkeyring-provideremission.pants test src/python/pants/backend/python/util_rules/pex_test.py src/python/pants/backend/python/goals/lockfile_test.py src/python/pants/backend/python/util_rules/lockfile_metadata_test.py src/python/pants/backend/python/subsystems/python_tool_base_test.py— all green.pants fmt lint checkon all 5 edited Python files — autoflake, flake8, ruff, ruff-format, visibility, and mypy all clean.pypiserver+ basic-auth locally, plus the original GCP Artifact Registry case using[python].uv_keyring_provider = "subprocess"andkeyrings.google-artifactregistry-auth— both should resolve withoutuser:pass@in the URL.🤖 Generated with Claude Code
Summary by cubic
Adds private index auth support for the
uvresolver via named indexes, uv-scoped env var passthrough, and an optional keyring provider. Also fixes named-index parsing to accept mixed-case URL schemes.New Features
[python-repos].indexesacceptsname=URL(e.g.,my-index=https://art.example.com/simple); the generateduv.tomluses the name in[[index]]. URLs with=in query strings are not mis-split.[uv].extra_env_varsforwards only listed vars to theuvsubprocess (e.g.,UV_INDEX_<NAME>_USERNAME/_PASSWORD) to avoid global leakage.[python].uv_keyring_provideremitskeyring-provider = "<value>"(e.g.,"subprocess") inuv.tomlto use system keyring backends.Bug Fixes
name=URL(e.g.,HTTPS://,Http://,git+HTTPS://) so named indexes parse correctly inuv.toml.Written for commit daa8907. Summary will update on new commits.