Skip to content

egress: add credential provider which reads from k8s secret - #1335

Open
yufan-su wants to merge 1 commit into
agent-substrate:mainfrom
yufan-su:cred-provider-plugin
Open

egress: add credential provider which reads from k8s secret#1335
yufan-su wants to merge 1 commit into
agent-substrate:mainfrom
yufan-su:cred-provider-plugin

Conversation

@yufan-su

@yufan-su yufan-su commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

First of a two-PR stack for atenet egress credential injection change. This PR adds the credential provider; the follow-up adds the egress injector that calls it.

What this adds

A credprovider gRPC service implementing the CredentialProvider.RequestSecret plugin API (pkg/proto/credproviderpb), backed by Kubernetes Secrets. It resolves substrate-secret://kubernetes.io/<provider>/<namespace>/<secret>[/<key>] URIs to a Secret value — brokering a read it's authorized to perform rather than persisting the secret itself.

Authorization

  • Default-deny atespace→namespace mapping (internal/proto/nsauthzpb); an atespace can only resolve secrets from its permitted namespaces.
  • The atespace is derived from the attested actor SPIFFE identity (internal/actorspiffe, shared with the injector in the next PR).
  • mTLS with optional client-CA verification for callers.

Included

  • cmd/credprovider (service + kubeprovider backend)
  • pkg/proto/credproviderpb (plugin API), internal/proto/nsauthzpb (authz mapping), internal/actorspiffe (SPIFFE parser)
  • Deploy manifest + namespace-policy / sample-secret fixtures

@EItanya

Eitan Yarmush (EItanya) commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

I think this PR is honestly a bit noisy. There's both the credential provider, and the implementation of yet another extproc server which uses it. Can we instead review the credential provider in isolation, especially given that there are still open questions in the doc, and then we can tackle the thing which calls it?

Doc since it's not linked anywhere: https://docs.google.com/document/d/1-zQKvu-oYJlva_kwL3xbAOaFoJJaqO0qEqmqRlt1r0c/edit?resourcekey=0-TDrWohfRuYolcXYO-QPVTQ&tab=t.0#heading=h.rx0okwgtp5yu

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed offline but for posterity, Eitan Yarmush (@EItanya) 's idea to split the pr sounds good.

Comment thread manifests/egress-credential-injection/README.md Outdated
@yufan-su
yufan-su force-pushed the cred-provider-plugin branch from 7e2e606 to 02a01c3 Compare September 1, 2026 03:35
@yufan-su
yufan-su force-pushed the cred-provider-plugin branch from 02a01c3 to d2731aa Compare September 1, 2026 03:59
@yufan-su yufan-su changed the title egress: add credential injector and credential provider egress: add credential provider which reads from k8s secret Sep 1, 2026
@yufan-su

yufan-su commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Split into two PRs #1335 and #1360 and this one only contains the credential provider change.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks yufan-su !

Comment on lines +40 to +41
// ProviderClass is the substrate-secret:// URI host this backend serves.
const ProviderClass = "kubernetes.io"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we removed "providerClass" concept from the doc and its now only provider name. the name is kubernetes.io

// loads from a textproto file at startup. It maps each atespace to the
// Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is
// default-deny: an atespace absent from this file can resolve nothing. This is a
// POC stand-in for a future authorization API.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment is very verbose, can we remove things that are POC related and make it shorter?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Lior Lieberman (@LiorLieberman) here, it's not clear we really need these proto types at all?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doubly so given that this never goes over the wire, it could just be golang directly

Comment on lines +18 to +19
# nothing. credprovider loads it once at startup, so editing this ConfigMap
# requires restarting the credprovider Deployment to take effect.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a TODO to make it dynamically reload?

namespace: ate-system
---
apiVersion: apps/v1
kind: Deployment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating YET another deployment for this feels like it spreads our deployment footprint out quite a bit. What do you think about running this as a sidecar for the egress policy, or from the control-plane somehow? I'm worried about:

  1. Latency
  2. deployment footprint

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to see if we consider this more of an example vs something that a lot of users will use.

Key things that would make some of the decisions:

  1. Would this be enabled by default (with opt-out)
  2. How we document the positioning of this (it's for demos vs it's for prod)

// loads from a textproto file at startup. It maps each atespace to the
// Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is
// default-deny: an atespace absent from this file can resolve nothing. This is a
// POC stand-in for a future authorization API.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Lior Lieberman (@LiorLieberman) here, it's not clear we really need these proto types at all?

// The attested actor identity on whose behalf the secret is fetched. Today
// this is the actor's SPIFFE URI as verified by the egress gateway; a
// verifiable Actor JWT is the intended future form.
string actor_identity = 1;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this secure? Wouldn't we need the actual actor identity document?

Comment on lines +27 to +32
# atespace "team-a" may resolve secrets in namespace "ns1" (matches the
# sample policy's substrate-secret://kubernetes.io/team-secrets/ns1/example-api).
policy {
atespace: "team-a"
allowed_namespaces: "ns1"
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this API been discussed at all? This seems like an important piece we need to come to agreement on which was not in the doc Bowei Du (@bowei) Lior Lieberman (@LiorLieberman)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's somewhat separate than the doc as this is specific to cred provider implementation.

I was thinking for this example, we do something very simple like a static YAML map of atespace(s) to k8s namespaces.

Nit: probably shouldn't use textproto, this is not a very popular format (outside of Google)

// loads from a textproto file at startup. It maps each atespace to the
// Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is
// default-deny: an atespace absent from this file can resolve nothing. This is a
// POC stand-in for a future authorization API.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doubly so given that this never goes over the wire, it could just be golang directly


// TrustDomain is the SPIFFE host an actor identity URI carries, matching the URI
// minted in cmd/ateapi/internal/actoridentity.
const TrustDomain = "substrate-actor.local"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This relates to my comment about the actual actor credential, I think this is a code smell from that

Comment thread cmd/credprovider/main.go
metricsAddr = pflag.String("metrics-address", ":9090", "Prometheus/health HTTP listen address")
serverBundle = pflag.String("server-cred-bundle", "", "credential bundle (PEM key+chain) presented for serving TLS; empty serves plaintext (dev only)")
clientCAFile = pflag.String("client-ca-file", "", "CA bundle that caller (injector) client certificates must chain to; empty accepts any client when TLS is on")
defaultKey = pflag.String("default-secret-key", "", "Secret data key used when a credential URI omits one; empty requires a single-key Secret")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a good idea

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants