Repository navigation
perf(mcp): speed up list_metrics and give metric discovery tools a 30s budget - #3260
Conversation
…s budget list_metrics timed out on 19% of production calls at its 10s cap. The per-kind query aggregated MetricUnit/MetricDescription across every point in the window, and kinds ran one after another. - Scan names only (MetricName + TimeUnix), then fetch unit/description for the returned page with LIMIT 1 BY + LIMIT so ClickHouse stops early. - Scan kinds in parallel; resolve once the page is decided and abort the rest. Kinds still running at the deadline are reported in partialFailure and finished kinds are returned. - Emit a cursor when a kind exactly fills the page and later kinds have names (previously the remaining kinds were dropped). - Raise list_metrics and describe_metric wall clocks to 30s to match the other MCP query tools, scaling per-query caps (break mode) to fit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 859c996 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🟡 Tier 3 — StandardIntroduces new logic, modifies core functionality, or touches areas with non-trivial risk. Why this tier:
Review process: Full human review — logic, architecture, edge cases. Stats
|
Duplicate code analysisNo new duplicate code introduced by this PR. jscpd compared this branch against |
PR Review
3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor 1 posted as inline comment(s) on the changed lines. 2 listed below. Findings outside the changed lines2 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
|
Deep Review✅ No critical issues found. No P0/P1 ship-blockers survived verification. The pagination, timeout/abort, and enrichment logic is sound, and all five prior bot review comments (timed-out-kind cursor, full-page cursor, large-page enrichment, schema-lookup discard, file size) were confirmed addressed by the current code. The findings below are recommended hardening and test-coverage items. 🟡 P2 — recommended
🔵 P3 nitpicks (6)
Reviewers (10): correctness, adversarial, reliability, performance, security, kieran-typescript, testing, maintainability, project-standards, previous-comments. Notes: Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
E2E Test Results✅ All tests passed • 407 passed • 1 skipped • 1505s
Tests ran across 4 shards in parallel. |
… pages - End the page at a timed-out kind and resume the cursor there (greptile P1) - Return a full page immediately with a cursor to the next non-empty kind (greptile P1) - Race unit/description enrichment, including getColumns, against the deadline (greptile P1) - Inline page names as escaped literals instead of one bind param each (greptile P2) - Resolve immediately when there are no kinds to scan (deep-review P0) - Bring api lint warnings back under the ceiling; add changeset
Keeps listMetricsPage.ts under the 300-line limit (greptile P2) and shares one ClickHouse query helper between the name scan and enrichment queries, removing the duplicated block jscpd flagged.
| signal: AbortSignal; | ||
| }): Promise<Map<string, { unit?: string; description?: string }>> { | ||
| const enrichments = new Map< | ||
| string, |
There was a problem hiding this comment.
🔵 minor — Batch unit/description lookup duplicates describeMetric's fetchUnitAndDescription; error cleanup duplicates sanitizeFetchError
describeMetric.ts:171 fetchUnitAndDescription builds the same hasUnit/hasDescription projection over the same bounded scan for a single name. describeMetric.ts:143 sanitizeFetchError is the same replace(/\s+/g,' ').trim().slice(0,200) that listMetricsPage.ts:142 inlines. Have describeMetric call fetchMetricUnitsAndDescriptions({ names: [metricName] }), and move sanitizeFetchError into a shared module that both files import.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Keeps listMetricsPage free of tool-module imports (review bot minor).
…meRange - Move the input schema and cursor helpers to listMetricsSchema.ts - Move enrichEntries next to the queries it runs in listMetricsQueries.ts - Share parseTimeRange between list_metrics and describe_metric (jscpd clone) Addresses the greptile file-size finding.
…cated The names scan used timeout_overflow_mode 'break' with a 24s cap, below the 26s scan deadline, so ClickHouse could return a partial name set that looked complete and the cursor skipped names never read. Raise the cap above the deadline and throw on overflow so the kind is reported in partialFailure and the cursor resumes at it (review bot major).
karl-power
left a comment
There was a problem hiding this comment.
Looks good. Just had one question here
…mes out End the page at a timed-out kind only when it already has entries; otherwise record it in partialFailure and continue, so a call never returns the cursor it was given (review bot major). Drop ASCII section dividers (greptile P2).
|
|
||
| const projections = [ | ||
| chSql`MetricName`, | ||
| ...(hasUnit ? [chSql`${{ Identifier: 'MetricUnit' }}`] : []), |
There was a problem hiding this comment.
🔵 minor — Unit/description lookup now exists twice, with different query strategies
fetchUnitAndDescription in describeMetric.ts:147 still uses a 100k-row anyLast subquery for the same columns. The PR measured that approach as slower than the new LIMIT 1 BY query. Export fetchMetricUnitsAndDescriptions and have describeMetric call it with names: [metricName], so there is one implementation.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
…int ceiling - Reuse parseTimeRange from mcp/tools/query/helpers with a defaultLookbackMs argument instead of a second copy (review bot minor) - Share a NumberMetricPoint type between the gauge and sum fixtures (jscpd) - Type fetchMetricNames' client as Pick<ClickhouseClient, 'query'> so its test needs no cast, keeping api lint at the 302-warning ceiling after merging main
| connectionId: string; | ||
| names: string[]; | ||
| startDate: Date; | ||
| endDate: Date; |
There was a problem hiding this comment.
🔵 minor — New unit/description lookup duplicates fetchUnitAndDescription in describeMetric.ts
describeMetric.ts:150 (fetchUnitAndDescription) already does this lookup for one name: same projections built from hasUnit/hasDescription, same time filter, same empty-string stripping. Export fetchMetricUnitsAndDescriptions and have describeMetric call it with [input.metricName], so the two tools share one query and can't drift apart. Today they pick their row differently (LIMIT 1 BY here vs anyLast over a 100k-row sample there).
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| logger.warn( | ||
| { kinds }, | ||
| 'Timed out fetching metric unit/description; returning names only', | ||
| ); |
There was a problem hiding this comment.
🔵 minor — No test for enrichEntries' best-effort fallback paths
Nothing tests the new fallback logic: the race against the abort signal when metadata.getColumns is slow, the per-kind catch that returns an empty map, and the merge by kind. The only test that touches enrichment is the happy-path integration test. Add unit tests with a stub metadata.getColumns that never resolves (or a pre-aborted signal), plus a failing query. Assert that the entries come back unchanged and the call doesn't hang.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| maxExecutionSeconds: number; | ||
| signal: AbortSignal; | ||
| }): Promise<Map<string, { unit?: string; description?: string }>> { | ||
| const enrichments = new Map< |
There was a problem hiding this comment.
🔵 minor — Batch unit/description lookup duplicates describeMetric's fetchUnitAndDescription
fetchMetricUnitsAndDescriptions now does the same lookup as fetchUnitAndDescription in packages/api/src/mcp/tools/sources/describeMetric.ts:150: same column-presence flags, same WHERE clause, and the same filtering of empty unit and description values. The only difference is the query shape (LIMIT 1 BY here, an anyLast over a 100k-row sample there). Export the batch helper and have describeMetric call it with names: [metricName]. That leaves one implementation, and describeMetric also gets the cheaper query.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
## Summary MCP tools each hand-rolled their own wall-clock timeout: an `AbortController`, one or two `setTimeout`s, a `Promise.race`, and either a marker error (`DESCRIBE_METRIC_TIMEOUT`, `LIST_METRICS_TIMEOUT`) or a sentinel symbol to detect the timeout. They now share `runWithTimeout` in `mcp/utils/timeout.ts`, and the 30s ClickHouse cap comes from one constant instead of literal `30`s scattered across tools. Each tool keeps its own budget. Resolves HDX-5606. Follow-up to #3260 (comment). `runWithTimeout(work, { timeoutMs, graceMs })`: - aborts the signal passed to `work` at `timeoutMs`, and stops waiting `graceMs` later — work that returns partial results after the abort keeps them (what `describe_source` needs) - a rejection after the abort counts as a timeout; one before it is rethrown - always aborts the signal on exit, so no query outlives the call | Caller | Budget | Grace | |---|---|---| | `clickstack_describe_source` | 30s | 2s | | `clickstack_describe_metric` | 30s | – | | `clickstack_list_metrics` | 30s | – | | `query_tiles` `withDeadline` (per tile, shared batch deadline) | remaining of 60s | – | | `list_sources` metric-name preview | 3s | – | `MCP_CLICKHOUSE_SETTINGS`, `MCP_REQUEST_TIMEOUT` (30s + 2s), and the `max_execution_time` in `denoise`, `runEventPatterns`, and `eventDeltas` now derive from the shared constants. Not converted: `scanKindsForPage` (its deadline finalizes a page rather than cancelling work) and the per-query ClickHouse caps (14s/28s/3s/2.5s), which are tuned per tool. **Behavior change:** `describe_source`, `describe_metric`, `list_metrics`, and the `list_sources` preview now abort their signal on every exit, not only on timeout. `withDeadline` already did this. ## Test plan - New `timeout.test.ts` covers success, abort at the deadline, partial results in the grace window, the backstop, and rejections before/after the abort - Existing `describeSourceTimeout`, `withDeadline`, and `listMetrics*` unit tests pass unchanged - `sources`, `queryTiles`, and `deltasTool` integration suites pass
Why
clickstack_list_metricsoften hit its 10s wall clock and returned an error with no results.clickstack_describe_metrichad the same 10s cap. The other MCP query tools allow 30s (max_execution_time: 30,MCP_REQUEST_TIMEOUT = 32_000).What changed
list_metrics
anyLast(MetricUnit)andanyLast(MetricDescription)over every point in the 24h window. Reading those String columns was most of the bytes read. The scan now reads onlyMetricNameandTimeUnix.WHERE MetricName IN (page) … LIMIT 1 BY MetricName LIMIT n. The outerLIMITlets ClickHouse stop as soon as every name has a row. This step is best effort: if it fails or times out, names come back without unit/description. The row it picks is arbitrary, which matches theanyLastbehaviour it replaces.cancel_http_readonly_queries_on_client_closeis already on by default). Before, kinds ran one after another.partialFailureas timed out, and the kinds that finished are returned. Before, the whole call returned an error.nextCursorwas returned, so those kinds were dropped. A cursor is now returned in that case.timeout_overflow_mode: 'break'.mcp.list_metrics.kinds_scanned,kinds_failed,kinds_timed_outandresult_count.describe_metric
breakmode before the wall clock fires. The old cap was already too high: two 8s stages added up to more than the 10s limit.Measurements (local, ClickHouse 26.8)
The local dev data is too small to show a difference, so I measured on a synthetic gauge table with the OTel schema: 60M rows, 3,000 metric names, 200 services, 48h of data. The query used a 24h window and the default page size (51 rows). Numbers are the fastest of 3 runs, taken from
system.query_log.anyLastunit + description)LIMIT 1 BY+LIMIT)argMaxinstead (rejected)namePatternnamePatternThat is about 9x fewer bytes read per kind and about 3.5x less ClickHouse time. Parallel kinds help further when the first kinds are small or empty, for example sparse
namePatternmatches or empty exponential histogram or summary tables.agent_docs/query_optimizations.md, which the task pointed to, does not exist in the repo, so it was not consulted.Trade-offs
breakmode on the names scan can still return a partial name set when the cap is hit. This was already true before this change.Tests
listMetricsPage.test.tscover page assembly, cursors (including the exact-fill case), kinds running in parallel, early abort, and partial results at the deadline.sources.int.test.tscover paging through every kind withlimit: 1without repeats or gaps, and unit/description coming back in the response.make dev-int FILE=sources.intpasses (142 tests), andyarn ci:unit src/mcppasses (379 tests).🤖 Generated with Claude Code