Skip to content

fix(mcp): tighten tool and input descriptions - #3396

Merged
kodiakhq[bot] merged 14 commits into
mainfrom
agent/mcp-tool-descriptions
Oct 19, 2026
Merged

kodiakhq[bot] merged 14 commits into
mainfrom
agent/mcp-tool-descriptions

Conversation

@brandon-pereira

@brandon-pereira brandon-pereira commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Claude Code truncates MCP tool descriptions longer than 2048 UTF-8 bytes. A tool-doc review also found inputs with no description, descriptions that repeat schema constraints (defaults, ranges, enum lists), ID fields that don't say where the ID comes from, and tools that don't describe their mutations or return values.

Nearly all changes are text only:

  • Shortened four tool descriptions that were over the limit and moved per-field detail into the field descriptions: clickstack_event_deltas (3318 → 1796 bytes), trace_top_time_consuming_operations (2585 → 1436), trace_waterfall (2199 → 1244), table (2082 → 1578).
  • Added descriptions to every input that lacked one: save_source fields, dashboard tile config/containers/colorRules/backgroundChart, alert chart-level whereLanguage, and the event_deltas baseline fields.
  • Removed descriptions that only repeated schema constraints. ID fields now say which tool returns the ID.
  • save_alert, save_webhook, save_saved_search and patch_dashboard now describe how updates work and what side effects they have. For example, patch_dashboard deletes the alerts on a tile when the tile's new config can't hold alerts. get_dashboard_tile and query_tile now describe what they return.
  • Fixed wrong descriptions: the percent number format multiplies by 100, and heatmapScaleType sets the Y-axis scale.

Non-text changes. Validation behavior doesn't change:

  • Field descriptions for the number-tile color rules, BackgroundChartSchema, and DashboardContainerSchema / DashboardContainerTabSchema are added in common-utils/src/types.ts, so the MCP tools use those schemas unchanged. Other consumers get the descriptions too.
  • query/eventDeltas builds the target and baseline groups from a small factory so each group gets its own descriptions. Types and defaults are unchanged.

Not done:

  • outputSchema, because tools return JSON as text content, not structuredContent.
  • The remaining "missing description" findings on $ref dedupe pointers. These come from the scanner, not from missing descriptions.

packages/api/src/mcp/__tests__/toolDescriptions.test.ts fails if any tool description or the server instructions are empty or go over 2048 bytes, or if any input property lacks a description (bare $ref pointers excepted).

How to reproduce

Starting point: local yarn dev.

On main:

  1. Run yarn dev.
  2. Connect npx @modelcontextprotocol/inspector to the local MCP endpoint (see MCP.md).
  3. Look at the initialize and tools/list responses. clickstack_event_deltas and the trace tools have descriptions over 2048 bytes, and several inputs have no description.

On this branch:

  1. Repeat the steps above. Every tool description is under 2048 bytes and every input has a description.

References

  • Related PRs: independent of agent/mcp-tool-annotations

@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Oct 19, 2026 4:54pm UTC
hyperdx-storybook Ready Ready Preview Oct 19, 2026 4:54pm UTC

Request Review

@changeset-bot

changeset-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2cb13fd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/api Patch
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 124 in total (1.67%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge with the saved-search default fix in dependent PR #3404.

Summary

Tightens MCP tool descriptions, fills missing input descriptions, and adds checks for empty or oversized metadata.

  • Event comparisons and trace tools explain what they compare and return.
  • Dashboard tools explain tile updates, alerts, and query results.
  • Alert tools explain update effects and webhook destinations.
  • Query tools explain filters, chart settings, and metric inputs.
  • Saved-search tools explain which fields updates replace or keep.
  • Source setup fields explain the telemetry settings they describe.

Reviews (13) · Last reviewed commit: "Merge branch 'main' into agent/mcp-tool-..." · Reviewed by Greptile

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 436 passed • 1 skipped • 1765s

Status Count
✅ Passed 436
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@brandon-pereira
brandon-pereira marked this pull request as ready for review October 16, 2026 19:44
@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Oct 16, 2026
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

  • Large diff: 2328 production lines changed (threshold: 1000)
  • Cross-layer change: touches backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 31
  • Production lines changed: 2328 (+ 89 in test files, excluded from tier calculation)
  • Branch: agent/mcp-tool-descriptions
  • Author: brandon-pereira

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

2 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 1 minor

2 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

Scope: PR #3396 fix(mcp): tighten tool and input descriptions against base d3e6f96. 34 files, ~1507/-916 lines — almost entirely Zod .describe() text in the MCP tool layer, plus two structural changes: a makeGroupSchema factory in eventDeltas.ts and description-only field additions in common-utils/src/types.ts.

Independent verification performed on the behavior-bearing parts of the diff:

  • makeGroupSchema factory (packages/api/src/mcp/tools/query/eventDeltas.ts) produces a schema structurally identical to the removed groupSchema — same keys, types, .optional()/.default('') on where, and the whereLanguage enum with .default('lucene'). No contract change to the target/baseline input shapes.
  • common-utils/src/types.ts changes are description-only; no .optional()/.default()/.min/.max/.length/enum members were altered on the color-rule, BackgroundChartSchema, or DashboardContainerSchema/Tab fields. Shared consumers are unaffected.
  • packages/api/src/utils/zod.ts drops the re-exports of MAX_TAG_LENGTH/MAX_TAGS; no code imports those symbols via @/utils/zod (all consumers import from @hyperdx/common-utils). Safe.
  • Prior review feedback is addressed: the histogram level description now scopes the requirement to quantile; the event_deltas score description now names the 0.1 tie-breaking boost; the color-rule wording now says the raw value is compared; exponential-histogram support is reflected; and saveAlert now documents that clearing scheduleStartAt without an offset resets the offset to 0.

✅ No critical issues found.

No P0/P1 defects. Validation behavior is unchanged, the structural refactor preserves the input contract, and the new description-coverage/byte-limit tests are sound.

🟡 P2 — recommended

  • packages/common-utils/src/types.ts:2251 — the collapsed description asserts it is "ignored when collapsible is false," a runtime-interaction claim that is not verified by any test and, if inaccurate, would mislead an agent about a container's initial state.
    • Fix: Confirm against the dashboard container rendering code that an initial collapsed: true is genuinely suppressed when collapsible is false, and reword if the interaction differs.
🔵 P3 nitpicks (1)
  • packages/api/src/mcp/__tests__/toolSchemas.test.ts:90 — the new makeGroupSchema factory relies on the generic input-schema walker for structural validity but has no targeted assertion that target and baseline still expose identical property sets after the refactor.
    • Fix: Add a focused assertion comparing the generated target/baseline input schemas (keys, defaults, enum) so a future factory edit that diverges the two groups is caught.

Reviewers (5): correctness, api-contract, kieran-typescript, testing, previous-comments.

Testing gaps: No regression test pins the makeGroupSchema factory output to the pre-refactor groupSchema shape; coverage currently relies on the general input-schema validity walker in toolSchemas.test.ts.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

pulpdrew
pulpdrew previously approved these changes Oct 19, 2026

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, a couple of potential suggestions to be slightly more accurate.

Comment thread packages/common-utils/src/types.ts Resolved

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -14,34 +9,50 @@ import {
const mcpFilterSchema = z

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Descriptions added to a hand-copied twin of common-utils FilterSchema instead of the canonical schema

mcpFilterSchema copies FilterSchema / ConditionFilterSchema / SqlAstFilterSchema field for field (packages/common-utils/src/types.ts:1254-1273). Elsewhere this PR moves field descriptions into common-utils so the MCP tools reuse those schemas unchanged (color rules, containers). Do the same here: put the new type/condition/operator/left/right descriptions on the common-utils schemas, then build this schema as FilterSchema.describe(...). That removes a second copy that has to be kept in sync by hand, which the DRY rule (REQUIRED) forbids.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

.optional()
.describe('Language for the where filter. Default: lucene.'),
.describe(
'Language for the where filter. Defaults to lucene on create; on update, ' +

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 major — whereLanguage says it defaults to lucene on create, but nothing sets that default and alerts read a missing value as SQL

On create, saveSavedSearch.ts:62 passes input.whereLanguage through unchanged, and the SavedSearch model has no default for it, so the field is saved as undefined. The alert task builds its query with buildSearchChartConfig (common-utils/src/core/searchChartConfig.ts:165, whereLanguage: input.whereLanguage ?? 'sql'). That means an agent that follows this description, sends where: "level:error" and leaves out whereLanguage gets an alert that runs the filter as SQL. Fix it in the handler: on create, default to 'lucene' (whereLanguage: input.whereLanguage ?? (isUpdate ? undefined : 'lucene')). Otherwise, change the description to say that omitting whereLanguage stores no language and alerts treat it as SQL.

Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond

'Omitted whereLanguage, orderBy, and filters keep their stored values. To ' +
'change one field, read the saved search with clickstack_get_saved_search ' +
'and resend everything you want to keep.\n\n' +
'Alerts can be attached to a saved search and evaluate its current query, ' +

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Description says changing select changes what attached alerts check, but alerts ignore select and do use filters

Saved-search alerts always query with select: ALERT_COUNT_DEFAULT_SELECT (tasks/checkAlerts/index.ts:805), so the saved search's select has no effect on them. The alert query does use where, whereLanguage and filters (lines 801-803). Change the sentence to "changing where, whereLanguage, filters, or sourceId changes what those alerts check".

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@kodiakhq
kodiakhq Bot merged commit 03f9fd5 into main Oct 19, 2026
29 checks passed
@kodiakhq
kodiakhq Bot deleted the agent/mcp-tool-descriptions branch October 19, 2026 17:02
kodiakhq Bot pushed a commit that referenced this pull request Oct 19, 2026
#3404)

## Summary

`clickstack_save_saved_search` stores a new saved search without `whereLanguage` when the caller omits it. The app reads a missing value as Lucene, but alerts read it as SQL (`buildSearchChartConfig` falls back to `'sql'`). So a Lucene filter such as `level:error` looks fine in the search page while the search's alerts evaluate it as SQL and break. Found by @pulpdrew in review of #3396.

- On create, an omitted `whereLanguage` is now stored as `lucene`, which matches the app and the tool's description.
- On update, an omitted `whereLanguage` still keeps the stored value. A schema-level `.default('lucene')` would have overwritten a stored `sql` value on every update that left the field out, so the default lives on the create path only.

Stacked on #3396, which kept its changes to descriptions only.

### How to reproduce

**Starting point:** local `yarn dev`, with an MCP client connected (see `MCP.md`).

**On `main`:**

1. Call `clickstack_save_saved_search` with a `name`, a `sourceId`, and `where: "level:error"`, and no `whereLanguage`.
2. Call `clickstack_get_saved_search` with the returned id. `whereLanguage` is missing.

**On this branch:**

1. Repeat the steps above. `whereLanguage` is `lucene`.
2. Update the same search with `whereLanguage: "sql"`, then update it again without `whereLanguage`. It stays `sql`.

`packages/api/src/mcp/__tests__/savedSearches.int.test.ts` covers both cases.

### References

- Related PRs: #3396 (base)

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 2cb13fdc Deployed Oct 19, 2026 by vercel[bot]
Preview – hyperdx-storybook — 2cb13fdc Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants