Implement admission, credential expiry and access revocation #977

Activity

1. privateip

   added

   enhancementNew feature or requestNew feature or request

   on Oct 22, 2026
2. privateip

   added a parent issue on Oct 22, 2026

   • MASQUE Connector for gVPC #970
3. privateip

   changed the title [-]MASQUE: implement admission, credential expiry and access revocation[/-] [+]Implement admission, credential expiry and access revocation[/+] on Oct 22, 2026
4. privateip

   added the

   Task

   issue type on Oct 22, 2026
5. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Verdict: valid. The capability is missing on main at 69640a5.

   How this was confirmed. This is a missing component rather than a misbehaving one, so the check was a source search, not a failing test.

   • internal/edge/masque/grant.go defines the Authorizer interface (Authorize, Refresh, Retire) that the adapter calls on admission, on every refresh tick and on replacement. Its doc comment states that authentication, /96 allocation and EdgeService issuance are not the adapter's job.
   • The only implementation is the conformance fixture in tests/edgeconformance/masqueprovider/config.go. It compares plaintext credentials from a config file, issues a one-hour deadline with no credential expiry, refers to EdgeServices the deployer created by hand, and forgets retirements when it restarts.
   • Nothing in the repository implements what the MASQUE contract assigns to this issue: digest-only credential storage, a separate grant authority that writes one finite EdgeService per authorization incarnation, the validUntil = min(credential expiry, read start + 30 s − 2 s) renewal, pod-bound provider identity on an authenticated TLS channel, revocation and rotation, and the operator tool for test connectors.

   What is already in place.

   • The foundation enforces EdgeService.spec.validUntil minus MaxSkew through admission, the attachment service controller and the route checks.
   • The adapter already ends a single connector's request when Refresh returns unauthenticated or forbidden, or when its deadline passes, without touching other connectors. Integrate connector ownership with Edge claims and independent expiry #976 added this.
   • internal/edge/admission.IdentityFromUserInfo turns a TokenReview result into a pod-bound identity, so the authority can reuse it.

   Findings beyond the issue text.

   • masque.Grant has no authorization ID or credential revision. The contract's Refresh takes both, so the grant type needs them, and sameAuthority has to compare them.
   • The HTTP/2 adapter (Implement HTTP/2 CONNECT-IP fallback over TLS #981) does not exist yet. Both transports use the same connectip.Admission, so this component serves both by construction. Running the HTTP/2 rows themselves waits on Implement HTTP/2 CONNECT-IP fallback over TLS #981.
   • The connector /96 ledger is Implement /96 connector addressing and aggregate return routing #978's deliverable. This issue needs an allocation to put into each grant, so the plan below has to say where that boundary sits.
   • Sending no probe before admission (CONNECT-IP server lacks overload limits and tenant isolation checks #991, item 2) is a transport-layer change for Implement the HTTP/3 CONNECT-IP edge service #980/Implement HTTP/2 CONNECT-IP fallback over TLS #981. This component cannot fix it on its own.
6. privateip

   self-assigned this

   on Oct 28, 2026
7. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Approved plan.

   Root cause. masque.Authorizer has no production implementation. The only one is the conformance fixture's static config.

   Fix, on branch fix/issue-977:

   1. internal/edge/masque/authority, the core.
      • Grant records are one restricted Secret per connector. Each holds the connector UUID, the VPC namespace, name and UID, the cell, the credential's SHA-256 digest and revision, an expiry of at most 24 h, an enabled flag, the allowed destinations, the policy references, and the allocation.
      • Authorize compares digests in constant time. It reads the record and the VPC uncached and requires the VPC UID to match. It then creates one EdgeService per authorization incarnation, with PacketAccess and PrefixReachability, the /96 as its only prefix, the /120 pool as clientPool, and validUntil = min(credential expiry, read start + 28 s).
      • Annotations on the service record the authorization ID, pod UID, revision and nonce. The authority keeps no other state, so a new leader rereads everything, and a retry with the same nonce returns the same incarnation.
      • Refresh rereads the record and renews validUntil with UID and resourceVersion preconditions. It refuses a disabled or rotated revision, a retired incarnation and the wrong pod. When the authority can't read the record, it returns unavailable and never renews from a cache.
      • Retire and a revocation sweep set validUntil in the past. A retired incarnation is never renewed.
      • masque.Grant gains an authorization ID and a credential revision, and sameAuthority compares both.
   2. Ledger. One ConfigMap per VPC and cell holds at most 256 entries. Every insert or retirement uses UID and resourceVersion preconditions. Retired /96s and pools are tombstoned and never reassigned.
   3. Channel and client.
      • The channel is HTTPS with JSON over TLS 1.3. The operator mounts the server certificate and the CA.
      • The provider sends a projected ServiceAccount token for audience masque-authority. The authority checks it with a TokenReview and admission.IdentityFromUserInfo.
      • authority.Client implements masque.Authorizer for the adapter.
   4. Command. cmd/masque-authority has a serve command and grant create|rotate|disable|delete commands. Each grant command validates the VPC UID and the reserved pools, prints the bearer once and stores only its digest.
   5. Fixture and docs. The conformance fixture gets an authority mode, and the provider guide and the contract's status are updated.

   Tests.

   • Admission tests cover:
     • a valid request;
     • a wrong credential;
     • a wrong VPC selector;
     • a recreated VPC with a new UID;
     • an expired credential;
     • a disabled or rotated revision.
   • Refresh tests cover a stale revision or service UID, a request from another pod, a replayed nonce and a retired incarnation.
   • Store tests cover an unavailable store, which must issue no grant.
   • Ledger tests cover conflicts and tombstones.
   • An adapter-level test revokes one connector while another keeps the shared instance renewing. The first stops within its bound and the other keeps forwarding.
   • A test checks that no credential appears in logs.

   Risk. Almost all of the code is new. The only change to an existing API is the two new masque.Grant fields, which touches the fixture and the adapter tests. The foundation is unchanged.

   Out of scope.

   • The HTTP/2 rows wait on Implement HTTP/2 CONNECT-IP fallback over TLS #981.
   • Removing the probe that runs before admission is Implement the HTTP/3 CONNECT-IP edge service #980/Implement HTTP/2 CONNECT-IP fallback over TLS #981 work.
   • The Dockerfile, the publish entry and the config/ manifests ship with Implement the HTTP/3 CONNECT-IP edge service #980.
   • The lab measurements for revocation and outage belong to Add reproducible end-to-end feature acceptance tests #985.
8. privateip commented on Oct 28, 2026

   

   privateip

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Three changes to the approved plan. Each came from a test that failed while I implemented it.

   1. Authorize carries the time the call was first sent, and the authority keeps a retired incarnation for 10 s before deleting it. Without this, a late retry could create a second incarnation. The sequence that showed it:

      1. Another pod's takeover retired the old incarnation and deleted it.
      2. The predecessor then retried with its old nonce.
      3. Nothing in the API still carried that nonce, so the authority treated the retry as a new call.

      Now a retry either finds its retired incarnation and is refused, or arrives too late and is refused as stale. The cost is up to 10 s more on a cross-pod takeover, on top of waiting for core to release the old attachments.

   2. Incarnations are named masque-<connector>-r<revision>, not after the authorization ID. A second test raced three pods for one connector. One check after each create could not settle the race: an issuer that checked before its rival existed could lose on name order, and both incarnations stayed live. The authority already issues only after every other incarnation of the connector is gone, so racing issuers now contend for one name. The API server accepts exactly one create. The UID tells successive incarnations apart, and the authorization ID still includes the nonce.

   3. Revocation push notifications are not built. The provider learns of a revocation from its next Refresh, at most 5 s later. The issued validUntil stays the backstop.

   Renewing in place works against core. A new adapter check renews a live service's validUntil past its first deadline, through several new generations. The attachment, route and claim epoch stay unchanged.
9. privateip

   added 2 commits that reference this issue on Oct 28, 2026

   feat(edge): Add the MASQUE grant authority

   ...

   55461bd

   feat(edge): Add the MASQUE grant authority

   ...

   f4c6beb
10. privateip

    closed this as completedin #1176on Oct 28, 2026