Skip to content

NBK-164: make DDSQL async queries resilient - #795

Draft
nmuldavin wants to merge 1 commit into
DataDog:mainfrom
nmuldavin:noah.muldavin/nbk-164-implement-more-robust-async-query-client-for-dds
Draft

NBK-164: make DDSQL async queries resilient#795
nmuldavin wants to merge 1 commit into
DataDog:mainfrom
nmuldavin:noah.muldavin/nbk-164-implement-more-robust-async-query-client-for-dds

Conversation

@nmuldavin

@nmuldavin nmuldavin commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Linear

NBK-164 — https://linear.app/datadoghq/issue/NBK-164/implement-more-robust-async-query-client-for-ddsql-queries

Outcome

All DDSQL async query paths share a browser-aligned client for immediate polling, transient fetch retries, progress reporting, cancellation, and query-ID error context.

Acceptance

  • A successful running response is followed immediately by the next fetch, with no client-side inter-poll delay, matching the browser.
  • Initial query-submission failures are not retried. Fetch responses with status 429, 500, 502, 503, or 504 are retried; other HTTP failures surface immediately.
  • Fetch retries match the advanced query browser client's default: 429 retries five times with exponential base delays of 2s, 4s, 8s, 16s, and 30s (the final 32s base capped at 30s); 500/502/503/504 retry twice with 1s and 2s base delays. Every retry adds independently sampled uniform 0–999ms positive jitter, then caps the complete delay at 30s; jitter does not affect the next exponential base.
  • X-RateLimit-Remaining and X-RateLimit-Reset affect 429 delays using the browser's rules: comma-separated values are paired, malformed or incomplete pairs are ignored, only exhausted limits (remaining <= 0) count, and the greatest applicable reset delay is used. A header-selected delay above the 30s per-delay cap stops retries rather than retrying early.
  • Long-running queries emit progress to stderr after 10 seconds and again after 30 seconds, without writing progress to stdout.
  • Fetch errors include the query ID when available.
  • The client has no overall elapsed-time deadline; polling ends when AQA returns a terminal state or the caller cancels.

Validation

  • cargo test --all --no-fail-fast -- --test-threads=1 — 1,937 tests passed; serial execution avoids unrelated shared-environment test races.
  • cargo test commands::advanced_query::tests --no-fail-fast — 24 lifecycle tests passed.
  • cargo test commands::ddsql::tests --no-fail-fast — 43 DDSQL integration/unit tests passed.
  • cargo clippy --all-targets -- -D warnings — passed.
  • cargo fmt --check — passed.
  • cargo check --target wasm32-wasip1 --no-default-features --features wasi — passed, including the target-specific non-Send transport future.
  • git diff --check — passed.

Not run / automation gaps

  • cargo auditcargo-audit is not installed locally; no dependencies were added.
  • Remaining CI cross-platform builds are delegated to the repository workflows.

Material deviations and decisions

  • None.

Remaining work or conditions

  • CI stabilization and human review. Keep this PR in draft until explicit human approval to mark it ready.

Share browser-aligned polling, fetch retry, rate-limit delay, progress,
and error-context behavior across DDSQL async query call sites.

Add deterministic coverage for retry schedules, header parsing, progress,
cancellation, and unbounded query lifetime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant