remote: support source-referenced secrets in services.json (env:/file:) instead of inline-only values #28

Problem

The remote package loads service definitions from services.json via Manager.LoadServices (remote/manager.go). Today, authentication secrets are embedded as literal values inside that file — auth.options.value (API key), token (bearer), and client_secret (OAuth2) are all plain strings.

This forces an all-or-nothing posture: because the secrets live inline, the entire config file becomes sensitive. That has real downsides for our deployment model:

• No clean ConfigMap/Secret split. Non-sensitive operational config (service IDs, URLs, timeouts, scopes, header names) can't live in a ConfigMap / be committed for GitOps, because it's interleaved with credentials.
• Coarse rotation. Rotating one credential means rewriting the whole blob.
• Poor fit for our secrets pipeline. We use HashiCorp Vault as the source of truth, synced into the cluster by the External Secrets Operator (ESO) via the nsw-vault-backend ClusterSecretStore, which generates native Kubernetes Secrets consumed through secretKeyRef / mounted volumes. The current "whole file is the secret" model can't consume those native Secrets by reference — it only accepts literal values.

In short: the config layer has no way to say "this field's value comes from somewhere else." Secrets and structure are fused.

Proposed Solution

Introduce a scheme-prefixed value reference for any secret-bearing field in services.json. A field value may be a literal, or a reference whose prefix names its source:

{
  "auth": {
    "type": "oauth2",
    "options": {
      "token_url": "https://idp.example.gov/token",
      "client_id": "my-client",
      "client_secret": "env:NPQS_CLIENT_SECRET",
      "scopes": ["npqs:submit"]
    }
  }
}

{
  "auth": {
    "type": "bearer",
    "options": { "token": "file:/var/run/secrets/npqs/token" }
  }
}

Supported sources initially:

• env:NAME — read from environment variable NAME (populated by ESO-generated Secret via secretKeyRef/envFrom).
• file:/path — read from a file (populated by an ESO-generated Secret mounted as a volume). Trailing newline trimmed.

Key design points:

• Per-field, not whole-file. Only fields explicitly marked with a known scheme are resolved; secret and non-secret fields coexist in the same options block (fixes OAuth2, which mixes both).
• Backward compatible. A value with no recognized scheme prefix is treated as a literal, exactly as today. Existing services.json files keep working unchanged — this is purely additive. An explicit literal: prefix is available as an escape hatch for the rare value that genuinely begins with a scheme name.
• Lazy resolution for rotation. file: references resolve inside the authenticator's Apply() (with a short TTL cache) rather than at client-construction time, so when ESO refreshes a mounted Secret the new value is picked up without a pod restart. env: resolves at startup (its natural lifecycle).
• Loud failures. An unset env var or unreadable/empty file errors at use (or at an eager validation pass on load) — it must never silently resolve to an empty string and send an unauthenticated request.

This lets non-sensitive config live in a ConfigMap (committable, auditable, GitOps-friendly) while only references point at ESO-generated native Secrets — aligning the SDK with the Vault → ESO → native Secret pipeline.

Scope / non-goals

• Initial sources are env: and file: only. A direct vault: source is intentionally out of scope — ESO already lands secrets as native Kubernetes Secrets, and a runtime Vault dependency in the SDK would defeat that. The scheme is extensible, so vault: (or others) can be added later without a breaking change.

Alternatives

• Whole-file ${VAR} interpolation (os.ExpandEnv over the file at load). Simpler (one line), but env-only, resolves once at load (no rotation without restart), and blindly rewrites the entire file — any literal containing $ is a footgun. Rejected in favor of the per-field, multi-source scheme.
• Flat top-level authSecretRef + fixed /secrets/<ref>/<key> mount convention. Clean for bearer/api_key but doesn't map onto OAuth2's mix of secret and non-secret fields, and couples the config to a single hard-coded mount layout. The per-field source:ref form is more flexible.
• Keep the whole file as a Kubernetes Secret. Makes all structure opaque, breaks GitOps/auditability, and forces a full-Secret rewrite to change a timeout. This is the status quo limitation we're trying to remove.
• Direct Vault integration in the SDK. Reintroduces the runtime coupling ESO exists to eliminate. Out of scope.