Repository navigation
π inference: refresh rotated gateway keys and refuse ignored api_key_file paths - #18540
Conversation
β¦file paths The inference proxy resolved the LiteLLM/gateway key once, when an agent's route was set, so a rotated Secret kept 401ing until restart (#18532). Routes now carry an InferenceKeyResolver that re-stats the key file every few seconds (symlink target, mtime, size: a Secret update swaps ..data), re-resolves on change or every 30s, and on an upstream 401 re-resolves once and retries once with a changed key before counting an auth failure. The auth-failure WARN gains key_source and key_changed_since_start; no key material is logged. An api_key_file outside the allowed secret dirs was silently skipped (#18533). Dashboard/API saves now reject it with a message listing the allowed dirs and the mount remedy; boot warns (existing configs keep booting on their fallback key); runtime logs one WARN naming the skipped path and the effective source; the governor config API and gateway rows expose keySource, keySourceDescription and apiKeyFileAllowed. The audit N8 gate is unchanged. Fixes #18532 Fixes #18533 Signed-off-by: Andy Anderson <andy@clubanderson.com>
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
Keeps agent_runtime.go under the crux file-size ratchet baseline. Signed-off-by: Andy Anderson <andy@clubanderson.com>
Mutation testinggremlins found 1 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.
3 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the
Verdict: |
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
What
Two LiteLLM/gateway key bugs, fixed together because they share the key-resolution path.
#18532: the proxy cached the key at route-set time
proxy.InferenceKeyResolver(src/crux/proxy/inference_key_resolver.go). It re-stats the key file at most everyinferenceKeyRecheckTTL(3s). It re-resolves when the symlink target, mtime or size changes (a Secret update swaps..data), and does a full re-resolve everyinferenceKeyFullRefreshInterval(30s) so a new source is noticed too.:18444and the MITM reroute) use the current key on each request, on a copy of the route. The shared route is never mutated.inference backend auth failureWARN gainskey_source(file:<path>,env:<NAME>,static,none) andkey_changed_since_start. Key material is never logged.litellmand named-gateway routes incmd/hive. watsonx keeps its static minted IAM token.#18533: an api_key_file outside the allowed dirs was silently ignored
WARNING:line.configured api_key_file is outside the allowed secret dirs and was skipped), withskipped_pathandeffective_key_source.GET /api/config/governorand each gateway row now includekeySource,keySourceDescription(e.g.key from /secrets/litellm_api_key (configured path /secrets-cb/... not allowed)) andapiKeyFileAllowed. The LiteLLM section also includesallowedKeyFileDirs. The LiteLLM tab shows a red note when the configured path is ignored.SecretFilePathAllowed) and its allowed dirs are unchanged.Tests
..datasymlink swap) is picked up after the TTL, on both entry points, with no restart.key_sourceandkey_changed_since_start=true. An unchanged key is not retried.Docs:
src/docs/agent-configuration.mdanddocs/inference-backends.md. Changelog:changelog.d/fixed-18532-18533-litellm-key-resolution.md.Fixes #18532
Fixes #18533
cc @joerunde
π€ Generated with Claude Code