Skip to content

control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/7) - #3342

Open
bbartman wants to merge 2 commits into
bmb/2781-stack-1-snapshot-authzfrom
bmb/2781-stack-2-inprocess-authz
Open

control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/7)#3342
bbartman wants to merge 2 commits into
bmb/2781-stack-1-snapshot-authzfrom
bmb/2781-stack-2-inprocess-authz

Conversation

@bbartman

@bbartman bbartman commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

What

Split 2 of 4 of #3155 (issue #2781): every catalog authorization decision of
publications and discovers moves from SQL (internal.user_roles() /
computed user_capability + spec_capabilities columns) to in-process
evaluation against one Snapshot pinned per operation.

  • live_specs: fetch_live_specs / fetch_expanded_live_specs lose
    their authorization parameters and computed-capability columns; the queries
    become plain spec fetches. get_live_specs / get_connected_live_specs
    gain a &Snapshot (and started_at anchor) and filter by
    models::authz::CapabilitySet in-process. A denial under a Snapshot that is
    stale for the operation's anchor raises the retryable
    AuthorizationSnapshotStale instead of silently dropping the spec;
    expansion (get_connected_live_specs) keeps denials as final silent
    omissions.
  • publications: resolve_live_specs makes all four authorization
    decisions (drafter SpecEdit; spec read/write authorization to
    sources/targets; user CatalogRead on referenced specs) against the pinned
    Snapshot, with the same staleness policy. Data-plane-name authorization
    moves off internal.user_roles() onto Snapshot::user_authorization.
    DraftPublication gains snapshot and started_at, threaded through
    Publisher::build and the Initialize trait; ExpandDraft's filter
    switches from legacy admin to SpecEdit
    (filter_user_has_adminfilter_user_authz).
  • discovers: Discover pins snapshot / started_at across the
    whole operation (prechecks → connector RPC → merge); the live-capture
    fetch filters by CatalogRead via the Snapshot.
  • Executors and call sites (agent): both executors pin one Snapshot per
    poll and thread it through; main.rs wires snapshot_watch into them.
    Controller/system paths (controlplane.rs, create_data_plane,
    update_l2_reporting) pass started_at: None, anchoring staleness to each
    spec's own last_pub_id — preserving today's semantics of evaluating
    against current-enough state.
  • Test harness: the authorization Snapshot is now backed by a
    manually-driven watch with explicit refresh_snapshot() points (documented
    as the "Snapshot testing model"), and truncate_tables clears
    storage_mappings / data_planes so tests are deterministic next to a
    running local stack.

Behavior note (interim, until splits 3–4): a denial under a genuinely stale
Snapshot surfaces as a terminal task failure carrying the "please retry"
error, rather than deferring automatically. Splits 3 and 4 teach the two
executors to reschedule instead.

Tests

  • live_specs::tests (new, #[sqlx::test] on the new authz_specs
    fixture): unfiltered fetches never stale; authorized specs included;
    authoritative denials dropped; stale denials retryable; connected-spec
    expansion filtering.
  • publications::specs::resolve_tests (new): each of the four in-process
    authorization branches, a fully-authorized positive control, the
    started_at: None / zero-last_pub_id terminal-denial fallback, and
    spec-to-spec authorization under verify_user_authz: false.
  • Existing integration suites run under the new threading (mechanical
    signature updates in locking_retries, source_captures,
    unknown_connectors).

Stack

  1. control-plane: Snapshot authorization model and retryable staleness error (1/7) #3341 — Snapshot authorization model + retryable staleness error
  2. → this PR — move authorization in-process against a pinned Snapshot
  3. agent: publications executor defers on stale authorization snapshots (3/7) #3343 — publications executor defers on stale snapshots
  4. agent: discovers executor authorizes data-planes via Snapshot and defers on staleness (4/7) #3344 — discovers executor authorizes via Snapshot and defers on staleness

… pinned Snapshot

Every catalog authorization decision of publications and discovers moves from
SQL (internal.user_roles() and the computed user_capability /
spec_capabilities columns) to in-process evaluation against one Snapshot
pinned per operation:

- live_specs: fetch queries lose their authorization parameters;
  get_live_specs / get_connected_live_specs take a &Snapshot and filter by
  models::authz::CapabilitySet in-process. A denial under a Snapshot that is
  stale for the operation's freshness anchor raises the retryable
  AuthorizationSnapshotStale rather than silently dropping the spec.
- publications: resolve_live_specs makes all four authorization decisions
  against the pinned Snapshot; data-plane-name authorization moves off
  internal.user_roles(). DraftPublication gains snapshot / started_at,
  threaded through Publisher::build and the Initialize trait. ExpandDraft
  filters by SpecEdit instead of legacy admin.
- discovers: Discover pins snapshot / started_at across prechecks, the
  connector RPC, and the merge; live-capture reads filter by CatalogRead.
- Executors pin one Snapshot per poll; controller and server paths anchor
  with started_at: None, preserving spec-relative staleness semantics.
- Test harness: the Snapshot is backed by a manually-driven watch with
  explicit refresh points, and truncate_tables clears storage_mappings and
  data_planes for determinism next to a running local stack.

Until the follow-up splits land, a denial under a genuinely stale Snapshot
surfaces as a terminal failure carrying the retryable error message, rather
than deferring automatically.

Split 2 of 4 from #3155.
@bbartman
bbartman force-pushed the bmb/2781-stack-2-inprocess-authz branch from aba3aac to 6663bed Compare August 11, 2026 18:23
@bbartman bbartman changed the title control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/4) control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/7) Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant