Repository navigation
fix(mcp): tighten tool and input descriptions - #3396
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 2cb13fd The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Duplicate code analysisNo new duplicate code introduced by this PR. jscpd compared this branch against |
[Medium impact] This PR appears safe to merge with the saved-search default fix in dependent PR #3404. SummaryTightens MCP tool descriptions, fills missing input descriptions, and adds checks for empty or oversized metadata.
Reviews (13) · Last reviewed commit: "Merge branch 'main' into agent/mcp-tool-..." · Reviewed by Greptile |
E2E Test Results✅ All tests passed • 436 passed • 1 skipped • 1765s
Tests ran across 4 shards in parallel. |
🔴 Tier 4 — CriticalTouches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI. Why this tier:
Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
PR Review
2 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 1 minor 2 posted as inline comment(s) on the changed lines. Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Deep ReviewScope: PR #3396 Independent verification performed on the behavior-bearing parts of the diff:
✅ No critical issues found. No P0/P1 defects. Validation behavior is unchanged, the structural refactor preserves the input contract, and the new description-coverage/byte-limit tests are sound. 🟡 P2 — recommended
🔵 P3 nitpicks (1)
Reviewers (5): correctness, api-contract, kieran-typescript, testing, previous-comments. Testing gaps: No regression test pins the No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
…ls and test input descriptions
…s; share schema walker in tests
pulpdrew
left a comment
There was a problem hiding this comment.
LGTM, a couple of potential suggestions to be slightly more accurate.
| @@ -14,34 +9,50 @@ import { | |||
| const mcpFilterSchema = z | |||
There was a problem hiding this comment.
🔵 minor — Descriptions added to a hand-copied twin of common-utils FilterSchema instead of the canonical schema
mcpFilterSchema copies FilterSchema / ConditionFilterSchema / SqlAstFilterSchema field for field (packages/common-utils/src/types.ts:1254-1273). Elsewhere this PR moves field descriptions into common-utils so the MCP tools reuse those schemas unchanged (color rules, containers). Do the same here: put the new type/condition/operator/left/right descriptions on the common-utils schemas, then build this schema as FilterSchema.describe(...). That removes a second copy that has to be kept in sync by hand, which the DRY rule (REQUIRED) forbids.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| .optional() | ||
| .describe('Language for the where filter. Default: lucene.'), | ||
| .describe( | ||
| 'Language for the where filter. Defaults to lucene on create; on update, ' + |
There was a problem hiding this comment.
🟠 major — whereLanguage says it defaults to lucene on create, but nothing sets that default and alerts read a missing value as SQL
On create, saveSavedSearch.ts:62 passes input.whereLanguage through unchanged, and the SavedSearch model has no default for it, so the field is saved as undefined. The alert task builds its query with buildSearchChartConfig (common-utils/src/core/searchChartConfig.ts:165, whereLanguage: input.whereLanguage ?? 'sql'). That means an agent that follows this description, sends where: "level:error" and leaves out whereLanguage gets an alert that runs the filter as SQL. Fix it in the handler: on create, default to 'lucene' (whereLanguage: input.whereLanguage ?? (isUpdate ? undefined : 'lucene')). Otherwise, change the description to say that omitting whereLanguage stores no language and alerts treat it as SQL.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
| 'Omitted whereLanguage, orderBy, and filters keep their stored values. To ' + | ||
| 'change one field, read the saved search with clickstack_get_saved_search ' + | ||
| 'and resend everything you want to keep.\n\n' + | ||
| 'Alerts can be attached to a saved search and evaluate its current query, ' + |
There was a problem hiding this comment.
🔵 minor — Description says changing select changes what attached alerts check, but alerts ignore select and do use filters
Saved-search alerts always query with select: ALERT_COUNT_DEFAULT_SELECT (tasks/checkAlerts/index.ts:805), so the saved search's select has no effect on them. The alert query does use where, whereLanguage and filters (lines 801-803). Change the sentence to "changing where, whereLanguage, filters, or sourceId changes what those alerts check".
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
#3404) ## Summary `clickstack_save_saved_search` stores a new saved search without `whereLanguage` when the caller omits it. The app reads a missing value as Lucene, but alerts read it as SQL (`buildSearchChartConfig` falls back to `'sql'`). So a Lucene filter such as `level:error` looks fine in the search page while the search's alerts evaluate it as SQL and break. Found by @pulpdrew in review of #3396. - On create, an omitted `whereLanguage` is now stored as `lucene`, which matches the app and the tool's description. - On update, an omitted `whereLanguage` still keeps the stored value. A schema-level `.default('lucene')` would have overwritten a stored `sql` value on every update that left the field out, so the default lives on the create path only. Stacked on #3396, which kept its changes to descriptions only. ### How to reproduce **Starting point:** local `yarn dev`, with an MCP client connected (see `MCP.md`). **On `main`:** 1. Call `clickstack_save_saved_search` with a `name`, a `sourceId`, and `where: "level:error"`, and no `whereLanguage`. 2. Call `clickstack_get_saved_search` with the returned id. `whereLanguage` is missing. **On this branch:** 1. Repeat the steps above. `whereLanguage` is `lucene`. 2. Update the same search with `whereLanguage: "sql"`, then update it again without `whereLanguage`. It stays `sql`. `packages/api/src/mcp/__tests__/savedSearches.int.test.ts` covers both cases. ### References - Related PRs: #3396 (base)
Summary
Claude Code truncates MCP tool descriptions longer than 2048 UTF-8 bytes. A tool-doc review also found inputs with no description, descriptions that repeat schema constraints (defaults, ranges, enum lists), ID fields that don't say where the ID comes from, and tools that don't describe their mutations or return values.
Nearly all changes are text only:
clickstack_event_deltas(3318 → 1796 bytes),trace_top_time_consuming_operations(2585 → 1436),trace_waterfall(2199 → 1244),table(2082 → 1578).save_sourcefields, dashboard tileconfig/containers/colorRules/backgroundChart, alert chart-levelwhereLanguage, and theevent_deltasbaseline fields.save_alert,save_webhook,save_saved_searchandpatch_dashboardnow describe how updates work and what side effects they have. For example,patch_dashboarddeletes the alerts on a tile when the tile's new config can't hold alerts.get_dashboard_tileandquery_tilenow describe what they return.percentnumber format multiplies by 100, andheatmapScaleTypesets the Y-axis scale.Non-text changes. Validation behavior doesn't change:
BackgroundChartSchema, andDashboardContainerSchema/DashboardContainerTabSchemaare added incommon-utils/src/types.ts, so the MCP tools use those schemas unchanged. Other consumers get the descriptions too.query/eventDeltasbuilds the target and baseline groups from a small factory so each group gets its own descriptions. Types and defaults are unchanged.Not done:
outputSchema, because tools return JSON as text content, notstructuredContent.$refdedupe pointers. These come from the scanner, not from missing descriptions.packages/api/src/mcp/__tests__/toolDescriptions.test.tsfails if any tool description or the server instructions are empty or go over 2048 bytes, or if any input property lacks a description (bare$refpointers excepted).How to reproduce
Starting point: local
yarn dev.On
main:yarn dev.npx @modelcontextprotocol/inspectorto the local MCP endpoint (seeMCP.md).initializeandtools/listresponses.clickstack_event_deltasand the trace tools have descriptions over 2048 bytes, and several inputs have no description.On this branch:
References
agent/mcp-tool-annotations