Skip to content

control-plane: connector_tags.default_capture_interval is never applied to capture specs #3314

Description

@Alex-Bair

Summary

The connector_tags.default_capture_interval column exists to let each connector tag declare its own default polling interval, but nothing in the control plane ever applies it. Captures that don't set an explicit interval always fall back to the global 5-minute default, regardless of what the connector's tag says.

Intent

The column was added in f72c01e (2024-08-27) and its schema comment (supabase/migrations/01_compacted.sql:4968) states:

▎ The default value for the interval property for a Capture. This is normally used for non-streaming connectors.

This matters for capture connectors that can't tail their source continuously - they start, produce a batch of output, and exit, and Flow re-invokes them on a schedule. A reasonable cadence varies widely by connector, which is what this per-tag default is for.

Expected precedence

  1. Explicit interval in the capture spec
  2. connector_tags.default_capture_interval for the capture's connector tag
  3. Global 5-minute fallback

Actual precedence

  1. Explicit interval in the capture spec
  2. Global 5-minute fallback

Step 2 is entirely absent. The only readers of the column anywhere in the repo are the GraphQL resolver that exposes it as defaultCaptureInterval (crates/control-plane-api/src/server/public/graphql/connectors/tags.rs:31,58,85, added in 1c72c14), plus fixtures and a snapshot. Nothing in the build, publish, or discover path touches it.

Where the gap is

Two control-plane sites decide a capture's interval, and neither consults the column:

  • Build/publish — crates/validation/src/capture.rs:443 copies the model's interval straight into the built spec: interval_seconds: interval.as_secs() as u32.
  • Discover — crates/agent/src/discovers.rs:330 hardcodes models::CaptureDef::default_interval() into the starter spec for a brand-new capture, despite already having the connector image in scope.

Relatedly, fetch_connector_spec (crates/control-plane-api/src/connector_tags.rs:180-200) selects auto_discover_interval but not default_capture_interval, so even callers that do load a tag's spec don't see it.

Why this isn't just a missing query

CaptureDef.interval is a plain Duration with a serde default (crates/models/src/captures.rs:32-38):

  #[serde(
      default = "CaptureDef::default_interval",
      with = "humantime_serde",
      skip_serializing_if = "CaptureDef::is_default_interval"
  )]
  pub interval: Duration,

The 5-minute default is applied at deserialization, before anything knows which connector image the capture uses. By the time a connector default could be applied, "user omitted interval" and "user explicitly wrote 5m" are the same in-memory value. Closing the gap therefore requires a model or lifecycle change, not only a database read.

One related observation: because of skip_serializing_if, a 5-minute interval is omitted when the model serializes, so stored live_specs JSON does still distinguish "no interval key" from "explicit non-default value." The only case that collapses is a user who deliberately typed 5m.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions