egress: add credential provider which reads from k8s secret - #1335
egress: add credential provider which reads from k8s secret#1335yufan-su wants to merge 1 commit into
Conversation
|
I think this PR is honestly a bit noisy. There's both the credential provider, and the implementation of yet another Doc since it's not linked anywhere: https://docs.google.com/document/d/1-zQKvu-oYJlva_kwL3xbAOaFoJJaqO0qEqmqRlt1r0c/edit?resourcekey=0-TDrWohfRuYolcXYO-QPVTQ&tab=t.0#heading=h.rx0okwgtp5yu |
Lior Lieberman (LiorLieberman)
left a comment
There was a problem hiding this comment.
We discussed offline but for posterity, Eitan Yarmush (@EItanya) 's idea to split the pr sounds good.
7e2e606 to
02a01c3
Compare
02a01c3 to
d2731aa
Compare
Lior Lieberman (LiorLieberman)
left a comment
There was a problem hiding this comment.
Thanks yufan-su !
| // ProviderClass is the substrate-secret:// URI host this backend serves. | ||
| const ProviderClass = "kubernetes.io" |
There was a problem hiding this comment.
nit: we removed "providerClass" concept from the doc and its now only provider name. the name is kubernetes.io
| // loads from a textproto file at startup. It maps each atespace to the | ||
| // Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is | ||
| // default-deny: an atespace absent from this file can resolve nothing. This is a | ||
| // POC stand-in for a future authorization API. |
There was a problem hiding this comment.
comment is very verbose, can we remove things that are POC related and make it shorter?
There was a problem hiding this comment.
I agree with Lior Lieberman (@LiorLieberman) here, it's not clear we really need these proto types at all?
There was a problem hiding this comment.
Doubly so given that this never goes over the wire, it could just be golang directly
| # nothing. credprovider loads it once at startup, so editing this ConfigMap | ||
| # requires restarting the credprovider Deployment to take effect. |
There was a problem hiding this comment.
should we add a TODO to make it dynamically reload?
| namespace: ate-system | ||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: Deployment |
There was a problem hiding this comment.
Creating YET another deployment for this feels like it spreads our deployment footprint out quite a bit. What do you think about running this as a sidecar for the egress policy, or from the control-plane somehow? I'm worried about:
- Latency
- deployment footprint
There was a problem hiding this comment.
I think we need to see if we consider this more of an example vs something that a lot of users will use.
Key things that would make some of the decisions:
- Would this be enabled by default (with opt-out)
- How we document the positioning of this (it's for demos vs it's for prod)
| // loads from a textproto file at startup. It maps each atespace to the | ||
| // Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is | ||
| // default-deny: an atespace absent from this file can resolve nothing. This is a | ||
| // POC stand-in for a future authorization API. |
There was a problem hiding this comment.
I agree with Lior Lieberman (@LiorLieberman) here, it's not clear we really need these proto types at all?
| // The attested actor identity on whose behalf the secret is fetched. Today | ||
| // this is the actor's SPIFFE URI as verified by the egress gateway; a | ||
| // verifiable Actor JWT is the intended future form. | ||
| string actor_identity = 1; |
There was a problem hiding this comment.
Is this secure? Wouldn't we need the actual actor identity document?
| # atespace "team-a" may resolve secrets in namespace "ns1" (matches the | ||
| # sample policy's substrate-secret://kubernetes.io/team-secrets/ns1/example-api). | ||
| policy { | ||
| atespace: "team-a" | ||
| allowed_namespaces: "ns1" | ||
| } |
There was a problem hiding this comment.
Has this API been discussed at all? This seems like an important piece we need to come to agreement on which was not in the doc Bowei Du (@bowei) Lior Lieberman (@LiorLieberman)
There was a problem hiding this comment.
It's somewhat separate than the doc as this is specific to cred provider implementation.
I was thinking for this example, we do something very simple like a static YAML map of atespace(s) to k8s namespaces.
Nit: probably shouldn't use textproto, this is not a very popular format (outside of Google)
| // loads from a textproto file at startup. It maps each atespace to the | ||
| // Kubernetes namespaces whose Secrets that atespace may resolve. Enforcement is | ||
| // default-deny: an atespace absent from this file can resolve nothing. This is a | ||
| // POC stand-in for a future authorization API. |
There was a problem hiding this comment.
Doubly so given that this never goes over the wire, it could just be golang directly
|
|
||
| // TrustDomain is the SPIFFE host an actor identity URI carries, matching the URI | ||
| // minted in cmd/ateapi/internal/actoridentity. | ||
| const TrustDomain = "substrate-actor.local" |
There was a problem hiding this comment.
This relates to my comment about the actual actor credential, I think this is a code smell from that
| metricsAddr = pflag.String("metrics-address", ":9090", "Prometheus/health HTTP listen address") | ||
| serverBundle = pflag.String("server-cred-bundle", "", "credential bundle (PEM key+chain) presented for serving TLS; empty serves plaintext (dev only)") | ||
| clientCAFile = pflag.String("client-ca-file", "", "CA bundle that caller (injector) client certificates must chain to; empty accepts any client when TLS is on") | ||
| defaultKey = pflag.String("default-secret-key", "", "Secret data key used when a credential URI omits one; empty requires a single-key Secret") |
There was a problem hiding this comment.
I don't think this is a good idea
First of a two-PR stack for atenet egress credential injection change. This PR adds the credential provider; the follow-up adds the egress injector that calls it.
What this adds
A
credprovidergRPC service implementing theCredentialProvider.RequestSecretplugin API (pkg/proto/credproviderpb), backed by Kubernetes Secrets. It resolvessubstrate-secret://kubernetes.io/<provider>/<namespace>/<secret>[/<key>]URIs to a Secret value — brokering a read it's authorized to perform rather than persisting the secret itself.Authorization
Included
cmd/credprovider(service + kubeprovider backend)pkg/proto/credproviderpb(plugin API),internal/proto/nsauthzpb(authz mapping),internal/actorspiffe(SPIFFE parser)namespace-policy/sample-secretfixtures