Skip to content

fix(mcp): stop clickstack_sql callers guessing table and timestamp names - #3354

Merged
kodiakhq[bot] merged 6 commits into
ClickHouse:mainfrom
squaredg:claude/hdx-5683-sql-table-guessing
Oct 13, 2026
Merged

kodiakhq[bot] merged 6 commits into
ClickHouse:mainfrom
squaredg:claude/hdx-5683-sql-table-guessing

Conversation

@squaredg

@squaredg squaredg commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Agents using clickstack_sql have no tool that tells them which table a source reads from, so they guess. The tool's own examples make the guess for them: FROM otel_logs and TimestampTime.

  • An unqualified otel_logs only resolves when the tables are in the connection's default database. That's true for the default install and false for most other deployments.
  • The seed otel_logs schema has no TimestampTime column, so the time-series example fails even on a default install.
  • clickstack_list_sources returns each source's timestampColumn and key columns, but not its database or table.
  • Unknown-table errors come back with no hint.

Changes:

  • clickstack_list_sources returns table (<database>.<table>) for every source with a table, and database for metric sources, whose tables are already listed in metricTables.
  • The sql parameter examples use <table> and <timestampColumn> placeholders, with one line saying where both come from.
  • Unknown-table and unknown-database errors get a hint pointing at the table from clickstack_list_sources. Unknown-column errors keep the clickstack_describe_source hint.

Tests (all failed before the change):

  • list_sources includes the database-qualified table for log and trace sources, and the database for metric sources.
  • The sql examples use only placeholders, and run once those are filled from a source.
  • Real ClickHouse unknown-table messages produce the new hint, both from errorHint and end to end through clickstack_sql.

Not in this PR:

  • Dashboard prompt and schema examples (prompts/dashboards/content.ts, tools/dashboards/schemas.ts) still hard-code otel_traces and TimestampTime. Raw SQL tiles can use $__sourceTable there, so that's a separate change.
  • Adding sourceId and $__sourceTable to clickstack_sql itself. I'll propose that separately.

How to test on Vercel preview

N/A — non-UI change.

References

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1efd7d2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 12, 2026

Copy link
Copy Markdown

@squaredg is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps

greptile-apps Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

This PR gives clickstack_sql callers source table names instead of making them guess.

  • Source listings show the table or database each source uses.
  • SQL examples use placeholders for each source's table and timestamp.
  • Raw SQL table errors point callers to the source list.

Reviews (5) · Last reviewed commit: "Merge branch 'main' into claude/hdx-5683..." · Reviewed by Greptile

Agents writing raw SQL had no tool that named a source's table, and the sql
examples hard-coded otel_logs and TimestampTime. Unqualified otel_logs only
resolves when tables sit in the connection's default database, and the seed
otel_logs schema has no TimestampTime, so the examples fail on most schemas.

- clickstack_list_sources returns each source's database-qualified table,
  and the database for metric sources.
- The sql examples use <table> and <timestampColumn> placeholders.
- Unknown-table errors hint at clickstack_list_sources.
@squaredg
squaredg force-pushed the claude/hdx-5683-sql-table-guessing branch from f84f6d5 to cf7b0d8 Compare October 13, 2026 17:31
@squaredg
squaredg marked this pull request as ready for review October 13, 2026 17:31
}
// Checked before unknown identifiers: "Unknown table expression identifier"
// names a table, so describe_source's column list would not help.
if (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — New unknown-table hint is SQL-specific but shown for every MCP tool's ClickHouse errors

errorHint runs for every tool that calls clickHouseErrorResult, not just clickstack_sql. That includes runQuery/runConfigQuery in helpers.ts, runEventPatterns.ts, eventDeltas.ts, trace/waterfall.ts, trace/breakdown.ts and describeMetric.ts. When a builder tool's source points at a dropped or renamed table, the agent is told to "use the table of a source from clickstack_list_sources", which is the same broken table and doesn't help, since builder tools take a sourceId. Fix: attach this hint only from sql.ts (for example via the existing suffix context, or a flag passed to errorHint). For source-backed tools, give a hint that says the source's configured table is missing.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc07b20: errorHint only adds the unknown-table hint when called from the raw SQL path (clickHouseErrorResult(e, { rawSql: true })). Source-based tools get no hint for a missing source table, as on main. New unit tests and a clickstack_table integration test cover it.

// connection's default database. Metric sources keep one table per
// kind in metricTables, so they get the database to qualify them with.
if (s.from.tableName) {
meta.table = `${s.from.databaseName}.${s.from.tableName}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — table is joined from raw names without quoting, so it can produce SQL that won't run

The code builds ${databaseName}.${tableName} from the raw names. Callers are told to paste this value into FROM, so a database or table named like otel-prod or my.logs gives invalid or misparsed SQL. Quote each part with quoteIdentifierIfNeeded (packages/common-utils/src/core/metadata.ts:158). Bare names stay unchanged, so the existing tests still pass.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5fb2477: each part is quoted with quoteIdentifierIfNeeded, so observability-prod / otel.logs comes back as `observability-prod`.`otel.logs` and bare names are unchanged. Covered by a new list_sources test.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

2 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 2 minor

2 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

Comment thread packages/api/src/mcp/tools/query/sql.ts Resolved
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. Eight reviewers ran; the three prior greptile P1s (unquoted table names, missing metric database, timestamp examples) are all resolved in the current diff with regression tests, and correctness verified the raw-SQL caller chain, regex ordering, and schema guarantees end to end. Remaining items are advisory quality/guidance improvements.

🟡 P2 -- recommended

  • packages/api/src/mcp/tools/query/helpers.ts:968 -- The new unknown-table hint tells callers to use a source's table from clickstack_list_sources, but metric sources never emit a table field (only database + metricTables), so for the raw-SQL-over-summary-metrics path this tool explicitly routes to clickstack_sql the hint points at a field that does not exist.
    • Fix: Extend the hint to also cover metric sources, instructing callers to qualify a name from metricTables with the source's database, mirroring the sql tool description.
    • agent-native
🔵 P3 nitpicks (5)
  • packages/api/src/mcp/tools/query/helpers.ts:967 -- The unknown-table regex omits the i flag that every sibling errorHint pattern carries; real ClickHouse messages still match via the uppercase UNKNOWN_TABLE/UNKNOWN_DATABASE code anchor, so no current miss, but the natural-language alternations would silently fail on casing drift and only canonical casing is tested.
    • Fix: Add the i flag for consistency with sibling branches, or add a lowercased/apostrophe (doesn't exist) test case to pin the intended contract.
    • testing, kieran-typescript, correctness
  • packages/api/src/mcp/tools/sources/listSources.ts:252 -- A non-metric source with a falsy from.tableName falls into the database-only branch while the sql guidance only mentions table; the log/trace schema enforces .min(1) so this is effectively unreachable today, but the branch is untested and the guidance has no fallback.
    • Fix: Add a test asserting the branch shape, or confirm the state is unreachable and drop the else path for non-metric kinds.
    • testing, api-contract, agent-native
  • packages/api/src/mcp/tools/sources/listSources.ts:297 -- metricTables values are returned as bare table names while database is backtick-quoted, so an agent following the "qualify a metricTables entry with its database" guidance can build a half-quoted identifier that fails if a metric table name needs quoting.
    • Fix: Quote metricTables values with quoteIdentifierIfNeeded, or emit a pre-qualified database.table per metric kind so nothing is concatenated by hand.
    • agent-native
  • packages/api/src/mcp/tools/query/sql.ts:42 -- The description says to use "the first expression in its timestampColumn" but never states the field can be a comma-separated list or how to split it, and naive comma-splitting breaks on expressions containing commas inside function calls.
    • Fix: Note that timestampColumn may list multiple expressions and the first top-level (bracket-aware) one should be used, or have list_sources emit a single already-reduced timestamp expression.
    • agent-native
  • packages/api/src/mcp/tools/sources/listSources.ts:210 -- The new table/database output fields and their mutual-exclusivity (table for log/trace, database for metric) are explained only in the clickstack_sql description, not in clickstack_list_sources' own description.
    • Fix: Add a sentence to the list_sources tool description documenting the table/database shape so a consumer understands it without cross-referencing another tool.
    • api-contract

Reviewers (8): correctness, testing, maintainability, project-standards, kieran-typescript, api-contract, previous-comments, agent-native.

Testing gaps:

  • No executable test substitutes a multi-column timestampColumn into the sql examples to prove the "first expression" guidance round-trips to valid SQL; the multi-column failure mode is asserted only as prose.
  • No test covers a raw-SQL dashboard tile (query_tile/query_tiles) receiving the new hint through the same catch; only clickstack_sql is exercised.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@brandon-pereira

Copy link
Copy Markdown
Member

@squaredg can you please check the greptile feedback and fix linting?

The cast tripped no-unsafe-type-assertion and took the api package one
warning over the CI lint ceiling.
}
// Checked before unknown identifiers: "Unknown table expression identifier"
// names a table, so describe_source's column list would not help.
if (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The unknown-table hint also fires for source-based builder tools, where the agent never wrote a table name

errorHint is reached through clickHouseErrorResult, which is shared by every tool: the builder path at helpers.ts:617 (clickstack_timeseries/table/search), trace/waterfall.ts, trace/breakdown.ts, eventDeltas.ts, runEventPatterns.ts and describeMetric.ts. All of these get the table from the source's from. If that table has been dropped or renamed, the agent is told to "use the table of a source from clickstack_list_sources". That table is the one that just failed, and these tools take no table argument anyway. Fix: attach this hint only on the clickstack_sql path (for example, pass it as a suffix from sql.ts), or reword it for the builder tools so it says the source's configured table is missing.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc07b20: errorHint only adds the unknown-table hint when called from the raw SQL path (clickHouseErrorResult(e, { rawSql: true })). Source-based tools get no hint for a missing source table, as on main. New unit tests and a clickstack_table integration test cover it.

// connection's default database. Metric sources keep one table per
// kind in metricTables, so they get the database to qualify them with.
if (s.from.tableName) {
meta.table = `${s.from.databaseName}.${s.from.tableName}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The qualified table is built by plain string concatenation, so names that need quoting produce SQL that won't run

${databaseName}.${tableName} is returned for agents to paste into FROM as-is. A source whose database or table name contains -, ., or a space (allowed in ClickHouse when backtick-quoted) gives an unparseable or wrongly split FROM. Fix: backtick-quote each part, escaping backticks, when it isn't a plain identifier. The quoting rule already exists in tableExpr in packages/common-utils/src/clickhouse/index.ts:826, which renders {Identifier} parts.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5fb2477: each part is quoted with quoteIdentifierIfNeeded, so observability-prod / otel.logs comes back as `observability-prod`.`otel.logs` and bare names are unchanged. Covered by a new list_sources test.

… its database

Agents paste table into FROM, so a database like observability-prod
broke the query. Metric sources that also set from.tableName got a table
instead of the database their metricTables live in.
…lder guidance

Source-based tools were sent back to clickstack_list_sources, which hands
back the same table that just failed. The sql examples now say to use the
first timestampColumn expression and how to qualify metric tables.
@squaredg

Copy link
Copy Markdown
Contributor Author

@brandon-pereira lint is fixed (1c68142: a type assertion in the new test pushed the api package one warning over the ceiling; this PR adds no net warnings now).

Review feedback, all in new commits:

  • Unquoted table (Greptile, Claude review, deep review): quoted with quoteIdentifierIfNeeded. 5fb2477
  • Metric sources with from.tableName losing database (Greptile): metric sources always get database. 5fb2477
  • Multi-column timestampColumn (Greptile): the sql instructions now say to use the first expression. cc07b20
  • Unknown-table hint shown for source-based tools (Claude review): now only on the raw SQL path. cc07b20

Deep review P3s:

  • Asymmetry between table and database: the sql instructions now say how to qualify a metric table with database.
  • Test parses the description text: kept on purpose. The description is what agents read and copy, so the test checks the text they get and runs the filled-in examples.

Unit tests and the queryTool and sources integration suites pass locally. That's two rounds of bot findings, so I'll leave any further scope calls to you.

return formatQueryResult(result);
} catch (e) {
return clickHouseErrorResult(e);
return clickHouseErrorResult(e, { rawSql: true });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The rawSql hint reaches dashboard raw-SQL tiles too, not just clickstack_sql

runConfigTile's raw-SQL branch is also called by clickstack_query_tile and clickstack_query_tiles (tools/dashboards/queryTile.ts:85, queryTiles.ts:303). For a tile using $__sourceTable, the failing table is the source's own table, so pointing the agent at clickstack_list_sources just returns that same table. That is the case this PR deliberately excludes for builder tools. Fix: pass rawSql from sql.ts via a runConfigTile option, or set it only when savedConfig.source is unset.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

// same source table that just failed.
if (
options?.rawSql &&
/Unknown table expression identifier|(Table|Database) \S+ does not exist|UNKNOWN_(TABLE|DATABASE)/.test(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Unknown-table regex misses the "doesn't exist" wording

Older ClickHouse servers report Table db.t doesn't exist.. @clickhouse/client puts the UNKNOWN_TABLE code in error.type and leaves it out of .message, so neither branch of the regex matches on those servers. The existing int test (queryTool.int.test.ts ~line 928) already allows /doesn.t exist/ for this reason. Fix: use does(n't| not) exist, and/or check e.type for UNKNOWN_TABLE/UNKNOWN_DATABASE instead of the message.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@brandon-pereira brandon-pereira left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kodiakhq
kodiakhq Bot merged commit a06b17f into ClickHouse:main Oct 13, 2026
26 of 28 checks passed
kodiakhq Bot pushed a commit that referenced this pull request Oct 13, 2026
…es (#3355)

## Summary

`clickstack_sql` takes only a `connectionId`, so an agent has to write the table name itself, and it often guesses wrong (#3354 covers the guessing). `$__sourceTable` already exists in common-utils, and raw SQL dashboard tiles resolve it through `runConfigTile`. The MCP SQL tool just never accepted a source.

This adds an optional `sourceId` to `clickstack_sql` and passes it to the tile, so `$__sourceTable` expands to that source's `database.table`. It reuses the existing `runConfigTile` path that dashboard tiles use.

Checks on the input:
- A `sourceId` that doesn't exist or belongs to another team is rejected as "Source not found".
- A source whose connection differs from `connectionId` is rejected. Otherwise `$__sourceTable` would expand against a table on another server.
- `$__sourceTable` without a `sourceId` is rejected with an error that says to pass one. Today the renderer's message comes through instead ("requires a source to be selected"), which is written for the dashboard UI.

The time-series example now reads `FROM $__sourceTable`. The table example keeps `FROM <table>` to show the path for tables that aren't registered as sources.

Tests (all failed before the change):
- `$__sourceTable` resolves to the source's table (counts seeded rows).
- `$__sourceTable` without `sourceId` is rejected, and the error says to pass one.
- An unknown `sourceId` and another team's `sourceId` are both rejected as not found.
- A source on a different connection than `connectionId` is rejected.
- The tool schema exposes an optional `sourceId`.
- The time-series example uses `$__sourceTable` and the table example uses `<table>`.

This matches the `sourceId` handling in our enterprise build, which has had it since source filters were added. That build also scopes it with source filters and RBAC, which aren't part of OSS and aren't included here.

### How to test on Vercel preview

N/A — non-UI change.

### References

- Linear Issue: HDX-5684
- Related PRs: #3353, #3354

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants