Skip to content

NBK-164: streamline and instrument DDSQL async polling - #790

Closed
nmuldavin wants to merge 1 commit into
DataDog:mainfrom
nmuldavin:noah.muldavin/nbk-164-pup-remove-the-redundant-poll-sleep-bound-and-in
Closed

NBK-164: streamline and instrument DDSQL async polling#790
nmuldavin wants to merge 1 commit into
DataDog:mainfrom
nmuldavin:noah.muldavin/nbk-164-pup-remove-the-redundant-poll-sleep-bound-and-in

Conversation

@nmuldavin

@nmuldavin nmuldavin commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Linear

NBK-164 — https://linear.app/datadoghq/issue/NBK-164/pup-remove-the-redundant-ddsql-poll-sleep-and-instrument-the-async

Outcome

DDSQL async queries poll without redundant success-path sleeps until AQA returns a terminal state, while individual HTTP and rate-limit waits remain bounded and long waits are reported without contaminating command output.

Scope

  • Share polling semantics across public v2 table/time-series execution and the existing findings-analyze path.
  • Bound submissions to 60 seconds and fetches to 15 seconds without imposing a client-wide query deadline.
  • Honor Retry-After and exponentially retry 429s within a 20-second total retry window, below AOP's 30-second polling-liveness timeout.
  • Emit staged progress only on interactive stdout/stderr; piped and agent-mode output remains quiet.
  • Non-goal: add a user-configurable --timeout flag.

This was developed on top of #785 (NBK-161) and is rebased onto main after that dependency merged.

Acceptance

  • Successful polls have no client-side sleep.
  • There is no client-wide elapsed-time deadline; AQA terminal states or caller cancellation end polling.
  • Initial and fetch requests have independent HTTP timeouts.
  • Initial submissions and fetches retry 429 / Retry-After with exponential backoff inside a 20-second total retry window.
  • Fetch HTTP, rate-limit, and response-state failures include the query ID when available.
  • Interactive progress starts around 10 seconds and escalates around 30 seconds with narrowing / LIMIT guidance.
  • Progress is stderr-only and suppressed for pipes and agent mode, preserving JSON stdout.
  • Table, time-series, and findings-analyze use the shared polling implementation.

Validation

  • cargo test commands::ddsql::tests --no-fail-fast — 49 DDSQL tests passed, including no-success-sleep, no overall deadline, initial/fetch 429, request timeout, and query-ID diagnostics.
  • cargo clippy --all-targets -- -D warnings — passed.
  • cargo test --no-fail-fast -- --test-threads=1 — 1,914 tests passed.
  • cargo fmt --check — passed.
  • git diff --check — passed.
  • Fresh-context final review — no blocker, high, or medium findings.
  • Draft CI — all five real checks passed, including Linux/macOS/Windows cross-compilation and WASM.

Not run / automation gaps

  • cargo audit — unavailable because cargo-audit is not installed; this change adds no dependencies.
  • Cross-platform compilation and live long-running DDSQL verification are left to CI / manual verification.

Material deviations and decisions

  • Removed the previously implemented two-minute client deadline after confirming AQA has server-side execution and abandoned-client controls.
  • Kept request and rate-limit waits bounded so network failures cannot hang one HTTP operation indefinitely.
  • Kept --timeout out of scope.

Remaining work or conditions

  • Human approval is required before marking the PR ready for review.

@nmuldavin
nmuldavin force-pushed the noah.muldavin/nbk-164-pup-remove-the-redundant-poll-sleep-bound-and-in branch from 8cd5c64 to 1c4452c Compare September 1, 2026 21:57
Remove successful-poll sleeps, let AQA terminal states govern total runtime, bound individual HTTP and rate-limit retry waits below server liveness, and emit TTY-only progress on stderr.
@nmuldavin
nmuldavin force-pushed the noah.muldavin/nbk-164-pup-remove-the-redundant-poll-sleep-bound-and-in branch from 1c4452c to b5fc943 Compare September 2, 2026 19:16
@nmuldavin nmuldavin changed the title NBK-164: bound and instrument DDSQL async polling NBK-164: streamline and instrument DDSQL async polling Sep 2, 2026
@nmuldavin

Copy link
Copy Markdown
Contributor Author

Closing this draft because NBK-164 was re-scoped and this implementation no longer matches its requirements. The issue remains active and will be reimplemented separately.

@nmuldavin nmuldavin closed this Sep 3, 2026
@nmuldavin
nmuldavin deleted the noah.muldavin/nbk-164-pup-remove-the-redundant-poll-sleep-bound-and-in branch September 3, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant