Grid credential inject

[nerdalert]

Dependency

⚠️ This PR depends on #339 (grid_route).

It is intentionally stacked on top of the grid_route work because grid_credential_inject consumes route metadata written by grid_route after candidate selection. This PR will remain a draft until #339 merges.

Summary

Adds native AI Grid per-candidate credential injection to the AI gateway.

This PR teaches the AI data plane how to carry a credential reference from grid_route and inject the correct outbound provider credential after route selection. The Grid overlay still contains only safe Secret locating information, never token bytes.

The request-time chain is:

grid_route
  -> selects the provider candidate
  -> writes grid.route.credential.* metadata when the selected candidate has a credential ref

grid_credential_inject
  -> reads grid.route.credential.* metadata
  -> matches the selected secretRef against configured credential entries
  -> reads the token from value/env/file source at filter construction
  -> injects Authorization: Bearer <token>

load_balancer
  -> forwards to the selected upstream cluster

Why

AI Grid needs to support API-backed and cloud-backed providers without requiring workloads to know or carry provider credentials.

A workload should authenticate to its local gateway with enterprise auth. The gateway should then:

authorize the workload,
route to the selected provider/capability,
inject the provider-side credential required for that upstream.
This keeps provider API keys out of clients, route overlays, route metadata, and logs.

• AI Grid needs to support API-backed and cloud-backed providers without requiring workloads to know or carry provider credentials.

A workload should authenticate to its local ingress gateway with enterprise auth. The Grid/Praxis path should then:

• authorize the workload,
• route to the selected provider/capability,
• forward to the appropriate egress gateway or backend,
• inject or sign with the provider-side credential only at the egress point that makes the final upstream call.

This keeps provider credential values out of clients, routing overlays, route metadata, ConfigMaps, and logs.

What changed

grid_route

grid_route now accepts an optional candidate credential reference:

candidates:
  - kind: inference_model
    name: model-z
    site: api-provider
    cluster: gateway-api-provider
    fresh: true
    credential:
      strategy: bearer_token
      secretRef:
        name: provider-api-token
        namespace: default
        key: token

The credential reference is carried through candidate validation and route selection.

When a credential-bearing candidate is selected, grid_route writes these in-process metadata keys:

  grid.route.credential.strategy
  grid.route.credential.name
  grid.route.credential.namespace
  grid.route.credential.key

These values contain only the credential reference, never the token.

New grid_credential_inject filter

Adds a new AI Grid filter:

- filter: grid_credential_inject
  credentials:
    - name: provider-api-token
      namespace: default
      key: token
      strategy: bearer_token
      file: /run/secrets/grid-credentials/provider-api-token/token

Supported token sources:

- value
- env_var
- file

Exactly one source must be configured per credential entry.

The file source is the production-oriented Kubernetes path: mount a Secret into the gateway pod and point file at the mounted token file. This keeps token bytes out of Praxis ConfigMaps.

Registry

Registers both Grid filters in the AI server registry:

• grid_route
• grid_credential_inject

Adds a registry smoke test to ensure the full registry includes both filters and does not hit duplicate grid_route registration.

Security properties

Token bytes do not appear in:

• Grid operator overlay JSON
• grid_route candidate YAML
• grid.route.* filter metadata
• tracing spans
• debug logs
• HTTP error response bodies

Token bytes may exist only in:

• the configured source (value, env_var, or mounted file)
• grid_credential_inject memory as Zeroizing
• the per-request outbound Authorization header
• the upstream provider request

The filter fails closed with HTTP 503 when:

• selected route metadata references a credential that is not configured
• selected route metadata uses an unsupported strategy
• the configured token source is missing, empty, or invalid at construction time

Validation

This PR adds validation for credential-bearing candidates:

• nested credential and secretRef structs use deny_unknown_fields
• credential fields are bounded and non-blank before being written to metadata
• overlong credential fields fail config validation instead of being silently dropped by metadata limits
• duplicate grid_credential_inject entries for the same (name, namespace, key) are rejected

Relationship to future auth work

This does not replace inbound caller authentication or authorization. It only handles outbound provider credential injection after Grid route selection.

• Expected boundary:

• Inbound auth/policy validates who the caller is and what they may access.

• AI Grid routing chooses the provider/capability.

• grid_credential_inject injects the credential required by the selected upstream provider.

This PR intentionally keeps the credential injector Grid-specific for now. It can be generalized later if the same route-selected credential pattern is needed outside Grid.

This unblocks the Grid operator work to project InferenceProvider.spec.auth.secretRef through the overlay and prove API-provider fallback without exposing provider tokens in routing data or ConfigMaps.

Validation

cargo test -p praxis-ai-filters grid
cargo test -p praxis-ai-proxy full_registry_includes_grid_credential_inject_without_duplicate_grid_route

• Companion Grid E2E validation:

cargo xtask env verify-api-fallback-native -c tests/env/operator-routing.toml
cargo xtask env verify-api-fallback -c tests/env/operator-routing.toml

[praxis-bot]

Review: Grid credential inject

Well-structured PR. The two-filter design (grid_route writes credential reference metadata, grid_credential_inject reads it and injects the bearer token) is clean separation of concerns. Security properties are solid: tokens stored in Zeroizing<String>, never written to metadata/traces/error bodies, fail-closed on mismatch. Config validation is thorough with deny_unknown_fields on all structs and bounded field lengths.

Findings

Severity	Count
Critical	0
Large	0
Medium	4

See inline comments for details. Summary:

1. [Medium] Doc table uses secret_ref (snake_case) but YAML config requires secretRef (camelCase) due to #[serde(rename_all = "camelCase")] on CandidateCredential. Will cause config parsing errors for users following the docs.
2. [Medium] No functional integration test for grid-route-mcp.yaml example config. Only inference routing has an integration test in tests/schema/tests/suite/examples/ai/grid_route.rs. Project conventions require functional integration tests for all example configs.
3. [Medium] No example config or functional integration test for grid_credential_inject. Unit tests are thorough, but project conventions require an example config in examples/configs/ and a corresponding integration test for new capabilities.
4. [Medium] Test credential_not_written_to_route_metadata has a misleading name and checks keys that would never be written. Credential references ARE written to route metadata by record_route_decision (under grid.route.credential.*). The test only checks non-existent key patterns (grid.route.credential, grid.route.strategy, grid.route.secret_ref, grid.route.token), which pass trivially. The companion test credential_reference_written_to_route_metadata does the meaningful positive assertion.