Skip to content

agent: discovers executor authorizes data-planes via Snapshot and defers on staleness (4/7) - #3344

Open
bbartman wants to merge 2 commits into
bmb/2781-stack-3-publications-stalenessfrom
bmb/2781-stack-4-discovers-staleness
Open

agent: discovers executor authorizes data-planes via Snapshot and defers on staleness (4/7)#3344
bbartman wants to merge 2 commits into
bmb/2781-stack-3-publications-stalenessfrom
bmb/2781-stack-4-discovers-staleness

Conversation

@bbartman

@bbartman bbartman commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

What

Split 4 of 4 of #3155 (issue #2781): the discovers executor moves its
data-plane authorization onto the Snapshot and gains the same
defer-on-stale behavior as publications, completing the removal of
internal.user_roles() from the discover path.

  • Data-plane authorization via Snapshot: the inline SQL query joining
    internal.user_roles() is deleted (along with its .sqlx cache entry) in
    favor of Snapshot::user_authorization + data_plane_by_catalog_name.
    This finally distinguishes the three cases the SQL conflated:
    • authoritative denial → new terminal JobStatus::NotAuthorized;
    • denial under a Snapshot predating the discover → provisional: request an
      early refresh and retry;
    • plane genuinely absent from an authoritative Snapshot →
      JobStatus::NoDataPlane (with a refresh-and-recheck grace for planes
      registered after the Snapshot was taken).
  • Request-relative staleness + deferral: started_at anchors to the
    queued discover row; AuthorizationSnapshotStale from live-spec reads
    (capture fetch or merge-phase collection reads) reschedules via
    DiscoverState::awaiting_snapshot_after + STALE_RETRY_WAKE instead of
    reporting a spurious failure. DiscoverOutcome becomes an enum
    (Resolved / RetryStale).
  • Test harness: add_data_plane (a second plane, so denial ≠ missing),
    queue_discover (caller-chosen plane, not auto-run), discover_job_status,
    and SnapshotRefresher — an owned handle letting 'static connector
    fixtures refresh the Snapshot mid-RPC.

Tests

  • user_discovers (new integration tests):
    • data-plane authorization: test_discover_reschedules_on_stale_data_plane_authz,
      test_discover_unauthorized_data_plane_is_terminal,
      test_discover_defers_polls_until_authoritative_snapshot,
      test_job_status_not_authorized_serde
    • plane existence vs freshness:
      test_discover_succeeds_after_late_data_plane_registration,
      test_discover_missing_data_plane_is_terminal_after_refresh
    • request-anchored live-spec authorization:
      test_discover_reschedules_on_stale_live_spec_authz,
      test_discover_reschedules_on_stale_collection_authz,
      test_discover_preserves_live_capture_after_late_grant — the end-to-end
      regression: a re-discover queued just after a grant lands must preserve
      the live capture (nonzero expect_pub_id, kept bindings/interval, no
      injected auto_discover) instead of silently re-drafting a starter spec
    • snapshot pinning: test_discover_uses_one_snapshot_across_connector_rpc
      a grant revoked during the connector RPC doesn't flip decisions within
      one discover.

After this PR the stack's tree is identical to #3155.

Stack

  1. control-plane: Snapshot authorization model and retryable staleness error (1/7) #3341 — Snapshot authorization model + retryable staleness error
  2. control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/7) #3342 — move authorization in-process against a pinned Snapshot
  3. agent: publications executor defers on stale authorization snapshots (3/7) #3343 — publications executor defers on stale snapshots
  4. → this PR — discovers executor authorizes via Snapshot and defers on staleness

…ers on staleness

The discovers executor moves its data-plane authorization onto the Snapshot
and gains the same defer-on-stale behavior as publications, completing the
removal of internal.user_roles() from the discover path:

- The inline SQL data-plane query is replaced by Snapshot::user_authorization
  plus data_plane_by_catalog_name, distinguishing the three cases the SQL
  conflated: an authoritative denial is the new terminal
  JobStatus::NotAuthorized; a denial under a Snapshot predating the discover
  requests an early refresh and retries; and a plane genuinely absent from an
  authoritative Snapshot remains NoDataPlane, with a refresh-and-recheck
  grace for planes registered after the Snapshot was taken.
- started_at anchors staleness to the queued discover row, and
  AuthorizationSnapshotStale from live-spec reads reschedules via
  DiscoverState::awaiting_snapshot_after instead of reporting a spurious
  failure. DiscoverOutcome becomes Resolved / RetryStale.
- Harness: add_data_plane, queue_discover, discover_job_status, and
  SnapshotRefresher for refreshing from 'static connector fixtures.

After this change the stacked tree is identical to #3155.

Split 4 of 4 from #3155.
@bbartman
bbartman force-pushed the bmb/2781-stack-4-discovers-staleness branch from b880581 to e46072c Compare August 11, 2026 18:23
@bbartman bbartman changed the title agent: discovers executor authorizes data-planes via Snapshot and defers on staleness (4/4) agent: discovers executor authorizes data-planes via Snapshot and defers on staleness (4/7) Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant