Skip to content

refactor(mcp): share execution timeout handling across tools - #3338

Merged
kodiakhq[bot] merged 5 commits into
mainfrom
brandon/mcp-dedupe
Oct 9, 2026
Merged

kodiakhq[bot] merged 5 commits into
mainfrom
brandon/mcp-dedupe

Conversation

@brandon-pereira

Copy link
Copy Markdown
Member

Summary

MCP tools each hand-rolled their own wall-clock timeout: an AbortController, one or two setTimeouts, a Promise.race, and either a marker error (DESCRIBE_METRIC_TIMEOUT, LIST_METRICS_TIMEOUT) or a sentinel symbol to detect the timeout. They now share runWithTimeout in mcp/utils/timeout.ts, and the 30s ClickHouse cap comes from one constant instead of literal 30s scattered across tools. Each tool keeps its own budget.

Resolves HDX-5606. Follow-up to #3260 (comment).

runWithTimeout(work, { timeoutMs, graceMs }):

  • aborts the signal passed to work at timeoutMs, and stops waiting graceMs later — work that returns partial results after the abort keeps them (what describe_source needs)
  • a rejection after the abort counts as a timeout; one before it is rethrown
  • always aborts the signal on exit, so no query outlives the call
Caller Budget Grace
clickstack_describe_source 30s 2s
clickstack_describe_metric 30s –
clickstack_list_metrics 30s –
query_tiles withDeadline (per tile, shared batch deadline) remaining of 60s –
list_sources metric-name preview 3s –

MCP_CLICKHOUSE_SETTINGS, MCP_REQUEST_TIMEOUT (30s + 2s), and the max_execution_time in denoise, runEventPatterns, and eventDeltas now derive from the shared constants.

Not converted: scanKindsForPage (its deadline finalizes a page rather than cancelling work) and the per-query ClickHouse caps (14s/28s/3s/2.5s), which are tuned per tool.

Behavior change: describe_source, describe_metric, list_metrics, and the list_sources preview now abort their signal on every exit, not only on timeout. withDeadline already did this.

Test plan

  • New timeout.test.ts covers success, abort at the deadline, partial results in the grace window, the backstop, and rejections before/after the abort
  • Existing describeSourceTimeout, withDeadline, and listMetrics* unit tests pass unchanged
  • sources, queryTiles, and deltasTool integration suites pass

@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Oct 9, 2026 7:17pm UTC
hyperdx-storybook Ready Ready Preview Oct 9, 2026 7:17pm UTC

Request Review

@changeset-bot

changeset-bot Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 042499c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 370 production lines changed (Tier 2 max: < 250)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 10
  • Production lines changed: 370 (+ 119 in test files, excluded from tier calculation)
  • Branch: brandon/mcp-dedupe
  • Author: brandon-pereira

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 125 in total (1.75%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

1 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 1 minor

None could be anchored to changed lines; they are listed below.

Findings outside the changed lines

1 minor
  • 🔵 packages/api/src/mcp/tools/sources/describeSource.ts:452 — Tool description reads the ClickHouse query cap, but the sampling cutoff is set by DESCRIBE_TIMEOUT_MS → Sampling is cut off by DESCRIBE_TIMEOUT_MS (the timeoutMs passed to runWithTimeout at line ~619), but the agent-facing description now prints MCP_QUERY_MAX_EXECUTION_SEC. The two only match today because MCP_TOOL_TIMEOUT_MS is defined as the cap × 1000. If the tool budget changes, the description will be wrong. Restore ${DESCRIBE_TIMEOUT_MS / 1000} so the description and the cutoff come from the same constant.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues remain.

Summary

MCP tools now share runWithTimeout and timeout constants while keeping their existing budgets.

  • MCP tools share timeout handling while keeping their own time budgets.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start work with an abort signal] --> B{Work finishes before deadline?}
  B -->|Yes| C[Return value or throw error]
  B -->|No| D{Grace enabled?}
  D -->|No| E[Settle timeout, then abort]
  D -->|Yes| F[Abort and wait through grace]
  F --> G{Work returns during grace?}
  G -->|Yes| H[Keep partial result]
  G -->|No or rejects| I[Report timeout]
  C --> J[Clear timer and abort on exit]
  E --> J
  H --> J
  I --> J
Loading

Reviews (5) · Last reviewed commit: "Merge branch 'main' into brandon/mcp-ded..." · Reviewed by Greptile

Comment thread packages/api/src/mcp/utils/timeout.ts Resolved
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 406 passed • 1 skipped • 1308s

Status Count
✅ Passed 406
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

Refactor consolidating five hand-rolled MCP timeout patterns into a shared runWithTimeout helper plus centralized ClickHouse cap constants. The correctness, adversarial, and reliability reviewers each traced all five call sites and the new primitive and found no behavior regressions: the zero-grace path settles TIMED_OUT before controller.abort() (so an abort-listener result can't masquerade as success), the graceMs>0 path preserves the old 30s+2s backstop semantics, MCP_REQUEST_TIMEOUT still resolves to exactly 32_000, and withDeadline already used a per-tile controller so cross-tile cancellation is unchanged. The prior greptile P2 ("deadline reported as success") is confirmed resolved with a matching test.

🟡 P2 -- recommended

  • packages/api/src/mcp/tools/sources/describeMetric.ts:746 -- The handler's timeout branch and the new abort-signal-on-every-exit behavior have no handler-level test; only describeSource has a dedicated timeout test.
    • Fix: Add a fake-timer test asserting the timed-out call returns the mcpServerError, logs the warning, and aborts the signal on both the timeout and success exits.
  • packages/api/src/mcp/tools/sources/listMetrics.ts:82 -- Same gap: the runWithTimeout timeout branch returning the mcpServerError and the abort-on-success exit are exercised only indirectly through the helper's own unit tests.
    • Fix: Add a fake-timer handler test asserting the timeout response and that the signal passed to listMetricsImpl aborts on both exits.
🔵 P3 nitpicks (3)
  • packages/api/src/mcp/tools/sources/describeSource.ts:572 -- The user-facing "Value sampling stops after N seconds" message now interpolates MCP_QUERY_MAX_EXECUTION_SEC (the per-query ClickHouse cap) rather than the wall-clock budget that actually stops sampling; the two are equal only by today's derivation.
    • Fix: Interpolate DESCRIBE_TIMEOUT_MS / 1000 so the message tracks the real deadline if the constants ever diverge.
    • kieran-typescript, maintainability
  • packages/api/src/mcp/utils/timeout.ts:5 -- MCP_TOOL_TIMEOUT_MS (whole-call wall-clock budget) is derived from MCP_QUERY_MAX_EXECUTION_SEC (single-query cap), so raising the query cap silently enlarges every tool's budget and MCP_REQUEST_TIMEOUT.
    • Fix: Make MCP_TOOL_TIMEOUT_MS a standalone literal, or add a comment stating the budget is intentionally pinned to the query cap.
  • packages/api/src/mcp/__tests__/timeout.test.ts:82 -- The graceMs>0 cases assert the result but never expect(jest.getTimerCount()).toBe(0), so a regression that failed to clear the reassigned grace timer in finally would go uncaught.
    • Fix: Add a timer-count assertion after each grace-path case resolves.

Reviewers (7): correctness, adversarial, reliability, kieran-typescript, testing, maintainability, previous-comments.

Testing gaps:

  • No handler-level fake-timer tests for the describeMetric / listMetrics timeout paths (only describeSource has one).
  • runWithTimeout grace-path cases lack timer-leak assertions.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

Comment thread packages/api/src/mcp/utils/timeout.ts Resolved

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice refactor

@kodiakhq
kodiakhq Bot merged commit 7aeb621 into main Oct 9, 2026
29 checks passed
@kodiakhq
kodiakhq Bot deleted the brandon/mcp-dedupe branch October 9, 2026 19:23

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 042499cd Deployed Oct 9, 2026 by vercel[bot]
Preview – hyperdx-storybook — 042499cd Deployed Oct 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants