Skip to content

uv: support private-index auth via named indexes, extra_env_vars, and keyring-provider#24

Open
Affanmir wants to merge 2 commits intobenjyw:uv_lockfilefrom
Affanmir:uv-private-index-auth
Open

uv: support private-index auth via named indexes, extra_env_vars, and keyring-provider#24
Affanmir wants to merge 2 commits intobenjyw:uv_lockfilefrom
Affanmir:uv-private-index-auth

Conversation

@Affanmir
Copy link
Copy Markdown

@Affanmir Affanmir commented May 4, 2026

This is a PR-on-PR for pantsbuild#23302 (which is open against pantsbuild:main). Targets benjyw:uv_lockfile so it can ride along with that PR.

Why

We ran pantsbuild#23302 locally against a private GCP Artifact Registry and hit a 401. The only working workaround was embedding oauth2accesstoken:$TOKEN@… directly in the index URL — which leaks short-lived tokens into checked-in config and forces an out-of-band refresh loop.

Three things conspired to break the standard uv auth paths:

  1. The generated uv.toml had no stable index name. ResolveConfig.uv_config() emitted default = true for the first index and auto-generated name = "extra-N" for the rest. uv's per-index credential env vars (UV_INDEX_<NAME>_USERNAME / _PASSWORD) need a matching name to bind to, and uv has no name-less fallback form, so UV_INDEX_LAAM_SDK_USERNAME had nothing to bind to.
  2. Env-var passthrough was global-only. get_uv_environment spread [subprocess-environment].env_vars into the uv Process.env, but adding UV_INDEX_* there leaks credentials into every other tool subprocess (pytest, mypy, twine, …).
  3. No way to enable uv's keyring backend. Even with keyrings.google-artifactregistry-auth installed system-wide, there was no option to emit keyring-provider = "subprocess" in the generated uv.toml.

What

Three small, opt-in pieces — all on the uv path. No behavior change for existing users:

  • [python-repos].indexes now accepts name=URL. A pex-compatible form already accepted by pex; we now also use the name when emitting the [[index]] block in the generated uv.toml. URLs whose query strings contain = are not mis-split (regex requires the LHS to be a simple identifier and the RHS to start with <scheme>://).

    [python-repos]
    indexes = ["my-index=https://art.example.com/simple"]
  • New [uv].extra_env_vars — uv-scoped passthrough, mirrors TestSubsystem.EnvironmentAware.extra_env_vars. Forwards listed vars only to the uv subprocess.

    [uv]
    extra_env_vars = ["UV_INDEX_MY_INDEX_USERNAME", "UV_INDEX_MY_INDEX_PASSWORD"]
  • New [python].uv_keyring_provider — when set, emits keyring-provider = "<value>" as a top-level key in the generated uv.toml. Typical value: "subprocess", which has uv shell out to the system keyring binary (works with backends like keyrings.google-artifactregistry-auth, keyring.codeartifact, …).

    [python]
    resolver = "uv"
    uv_keyring_provider = "subprocess"

Out of scope

Pex's pip-resolver venv can't see system-installed keyrings.* backends — that gap exists in main and is not introduced by pantsbuild#23302. Filing as a separate follow-up after this lands.

Test plan

  • pants test src/python/pants/backend/python/util_rules/pex_requirements_test.py — all 84 cases pass, including 3 new tests covering named-index emission, the =-in-query-string ambiguity, and keyring-provider emission.
  • pants test src/python/pants/backend/python/util_rules/pex_test.py src/python/pants/backend/python/goals/lockfile_test.py src/python/pants/backend/python/util_rules/lockfile_metadata_test.py src/python/pants/backend/python/subsystems/python_tool_base_test.py — all green.
  • pants fmt lint check on all 5 edited Python files — autoflake, flake8, ruff, ruff-format, visibility, and mypy all clean.
  • Manual repro against a private index: pypiserver + basic-auth locally, plus the original GCP Artifact Registry case using [python].uv_keyring_provider = "subprocess" and keyrings.google-artifactregistry-auth — both should resolve without user:pass@ in the URL.

🤖 Generated with Claude Code


Summary by cubic

Adds private index auth support for the uv resolver via named indexes, uv-scoped env var passthrough, and an optional keyring provider. Also fixes named-index parsing to accept mixed-case URL schemes.

  • New Features

    • [python-repos].indexes accepts name=URL (e.g., my-index=https://art.example.com/simple); the generated uv.toml uses the name in [[index]]. URLs with = in query strings are not mis-split.
    • [uv].extra_env_vars forwards only listed vars to the uv subprocess (e.g., UV_INDEX_<NAME>_USERNAME/_PASSWORD) to avoid global leakage.
    • [python].uv_keyring_provider emits keyring-provider = "<value>" (e.g., "subprocess") in uv.toml to use system keyring backends.
  • Bug Fixes

    • Accept mixed-case URI schemes in name=URL (e.g., HTTPS://, Http://, git+HTTPS://) so named indexes parse correctly in uv.toml.

Written for commit daa8907. Summary will update on new commits.

… keyring-provider

When using the new uv resolver from pantsbuild#23302, authenticating to a private
index (e.g. GCP Artifact Registry, AWS CodeArtifact, Azure Artifacts)
required embedding `user:token@` directly in the index URL because:

1. The generated `uv.toml` only emitted `default = true` / `name = "extra-N"`
   for indexes — never a stable, user-chosen name. uv's per-index credential
   env vars (`UV_INDEX_<NAME>_USERNAME` / `_PASSWORD`) need a matching name
   to bind to, and uv has no name-less fallback form.
2. The uv subprocess only inherited env vars listed in the *global*
   `[subprocess-environment].env_vars`, which leaks credentials into every
   other tool subprocess (pytest, mypy, twine, ...).
3. There was no way to opt the generated `uv.toml` into uv's
   `keyring-provider = "subprocess"` setting, even when `keyring` and a
   backend like `keyrings.google-artifactregistry-auth` were installed
   system-wide.

This change adds three small, opt-in pieces:

- `[python-repos].indexes` now accepts a pex-compatible `name=URL` form
  (e.g. `"my-index=https://art.example.com/simple"`). The name is emitted
  as the `name` field of the corresponding `[[index]]` block in the
  generated `uv.toml`. URLs whose query strings contain `=` are not split.
- A new `[uv].extra_env_vars` option that mirrors
  `TestSubsystem.EnvironmentAware.extra_env_vars`. Variables read here are
  forwarded only to the uv subprocess.
- A new `[python].uv_keyring_provider` option which, when set, emits
  `keyring-provider = "<value>"` as a top-level key in the generated
  `uv.toml`.

The pex-side keyring gap (pex's internal pip venv can't see system-installed
`keyrings.*` backends) is acknowledged but pre-existing and out of scope here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/python/pants/backend/python/util_rules/pex_requirements.py">

<violation number="1" location="src/python/pants/backend/python/util_rules/pex_requirements.py:417">
P2: The named-index regex rejects valid mixed-case URL schemes, so a spec like `name=HTTP://...` is misparsed as an unnamed URL and yields invalid uv config.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread src/python/pants/backend/python/util_rules/pex_requirements.py Outdated
The named-index regex required a lowercase scheme on the lookahead, so
specs like `name=HTTPS://art.example.com/simple` (or any non-lowercase
scheme) fell through `_split_named_index()` as unnamed and were emitted
as a literal `url = "name=HTTPS://..."` in the generated `uv.toml` —
which uv then tried to fetch as a URL.

Per RFC 3986 §3.1, URI schemes are case-insensitive (lowercase is just
the canonical form). Widen the lookahead character classes to `[A-Za-z]`
so `HTTPS://`, `Http://`, `git+HTTPS://`, etc. all parse correctly.

Add a regression test covering four mixed-case schemes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@benjyw
Copy link
Copy Markdown
Owner

benjyw commented May 6, 2026

I've implemented this here: pantsbuild#23322

The env var part is basically copied from here (thanks) but the named index stuff is part of a larger change to support pointing specific requirements to named indexes.

Feel free to close this and comment there (and also merge that change in and try it out, before I upstream it...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants