Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/proud-walls-argue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@hyperdx/api": patch
---

feat: Support series mode heatmap in API and MCP
75 changes: 73 additions & 2 deletions packages/api/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -2225,14 +2225,14 @@
}
}
},
"HeatmapChartConfig": {
"HeatmapDistributionChartConfig": {
"type": "object",
"required": [
"displayType",
"sourceId",
"select"
],
"description": "Builder configuration for a heatmap tile. Heatmap is builder-only (no Raw SQL variant) and currently supports trace sources. The row-level filter lives at the chart-config level (where / whereLanguage), matching the HeatmapSeriesEditor in the UI.\n",
"description": "Builder configuration for a distribution-mode heatmap tile, which buckets a numeric value on the y-axis. Requires a trace source. The row-level filter lives at the chart-config level (where / whereLanguage), matching the HeatmapSeriesEditor in the UI.\n",
"properties": {
"displayType": {
"type": "string",
Expand All @@ -2242,6 +2242,14 @@
"description": "Display type discriminator. Must be \"heatmap\" for heatmap tiles.",
"example": "heatmap"
},
"heatmapMode": {
"type": "string",
"enum": [
"distribution"
],
"description": "Heatmap mode. \"distribution\" is the default when omitted.",
"example": "distribution"
},
"sourceId": {
"type": "string",
"description": "ID of the data source to query.",
Expand Down Expand Up @@ -2274,6 +2282,69 @@
}
}
},
"HeatmapSeriesChartConfig": {
"type": "object",
"required": [
"displayType",
"heatmapMode",
"sourceId",
"select"
],
"description": "Builder configuration for a series-mode heatmap tile, which shows one aggregated series per groupBy value as a row on the y-axis. Requires a trace, log, or metric source. Any filter lives on the select item (where / whereLanguage).\n",
"properties": {
"displayType": {
"type": "string",
"enum": [
"heatmap"
],
"description": "Display type discriminator. Must be \"heatmap\" for heatmap tiles.",
"example": "heatmap"
},
"heatmapMode": {
"type": "string",
"enum": [
"series"
],
"description": "Heatmap mode. Must be \"series\" for series heatmaps.",
"example": "series"
},
"sourceId": {
"type": "string",
"description": "ID of the data source to query.",
"example": "65f5e4a3b9e77c001a111111"
},
"select": {
"type": "array",
"minItems": 1,
"maxItems": 1,
"description": "Exactly one aggregated value used to color each cell.",
"items": {
"$ref": "#/components/schemas/SelectItem"
}
},
"groupBy": {
"type": "string",
"maxLength": 10000,
"description": "Field expression to group results by (one row per group value).",
"example": "ServiceName"
},
"numberFormat": {
"$ref": "#/components/schemas/NumberFormat",
"description": "Number formatting options for displayed values."
}
}
},
"HeatmapChartConfig": {
"description": "Heatmap tile. Heatmap is builder-only (no Raw SQL variant). Omit heatmapMode (or set it to \"distribution\") for the distribution variant; set heatmapMode to \"series\" for the series variant.\n",
"oneOf": [
{
"$ref": "#/components/schemas/HeatmapDistributionChartConfig"
},
{
"$ref": "#/components/schemas/HeatmapSeriesChartConfig"
}
]
},
"SearchChartConfig": {
"type": "object",
"required": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -823,6 +823,77 @@ describe('MCP Dashboard Tools - clickstack_patch_dashboard', () => {
expect(tile.config.seriesLimit).toBe(3);
});

it('should patch a series heatmap tile without losing its mode', async () => {
const logSource = await Source.create({
kind: SourceKind.Log,
team: ctx.team._id,
from: { databaseName: DEFAULT_DATABASE, tableName: 'otel_logs' },
timestampValueExpression: 'Timestamp',
connection: ctx.connection._id,
name: 'Logs',
});
const sourceId = logSource._id.toString();
const createResult = await callTool(
ctx.client!,
'clickstack_save_dashboard',
{
name: 'Series Heatmap Patch Test',
tiles: [
{
name: 'Original',
config: {
displayType: 'heatmap',
heatmapMode: 'series',
sourceId,
select: [{ aggFn: 'count' }],
groupBy: 'ServiceName',
},
},
],
},
);
expect(createResult.isError).toBeFalsy();
const created = JSON.parse(getFirstText(createResult));
const tileId = created.tiles[0].id;

// A valueExpression also satisfies the distribution schema, so this
Comment thread
pulpdrew marked this conversation as resolved.
Resolved
// guards the patch union order.
const patchedConfig = {
displayType: 'heatmap',
heatmapMode: 'series',
sourceId,
select: [
{
aggFn: 'avg',
valueExpression: 'Duration',
where: 'SeverityText:error',
whereLanguage: 'lucene',
},
],
groupBy: 'SeverityText',
};
const patchResult = await callTool(
ctx.client!,
'clickstack_patch_dashboard',
{
dashboardId: created.id,
tileId,
tile: { name: 'Patched', config: patchedConfig },
},
);
expect(patchResult.isError).toBeFalsy();

const getResult = await callTool(
ctx.client!,
'clickstack_get_dashboard_tile',
{ dashboardId: created.id, tileId },
);
expect(getResult.isError).toBeFalsy();
const tile = JSON.parse(getFirstText(getResult));
expect(tile.name).toBe('Patched');
expect(tile.config).toMatchObject(patchedConfig);
});

describe('raw SQL macro warnings', () => {
// Patching a tile to a macro-less raw SQL config succeeds (non-blocking)
// but surfaces an advisory `warnings` array on the response.
Expand Down
65 changes: 65 additions & 0 deletions packages/api/src/mcp/__tests__/dashboards/queryTile.int.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,71 @@ describe('MCP Dashboard Tools - clickstack_query_tile', () => {
expect(result.isError).toBeFalsy();
});

it('returns time-bucketed rows for a series-mode heatmap tile', async () => {
const logSource = await Source.create({
kind: SourceKind.Log,
team: ctx.team._id,
from: { databaseName: DEFAULT_DATABASE, tableName: DEFAULT_LOGS_TABLE },
timestampValueExpression: 'Timestamp',
connection: ctx.connection._id,
name: 'Series Heatmap Logs',
bodyExpression: 'Body',
severityTextExpression: 'SeverityText',
});
const now = new Date();
const logs: Parameters<typeof bulkInsertLogs>[0] = [];
for (const minutesAgo of [10, 5]) {
for (const serviceName of ['svc-a', 'svc-b']) {
logs.push({
Body: `series heatmap log ${serviceName} ${minutesAgo}`,
ServiceName: serviceName,
SeverityText: 'INFO',
Timestamp: new Date(now.getTime() - minutesAgo * 60 * 1000),
});
}
}
await bulkInsertLogs(logs);

const createResult = await callTool(
ctx.client!,
'clickstack_save_dashboard',
{
name: 'Series Heatmap Dashboard',
tiles: [
{
name: 'Count by service',
config: {
displayType: 'heatmap',
heatmapMode: 'series',
sourceId: logSource._id.toString(),
select: [{ aggFn: 'count' }],
groupBy: 'ServiceName',
},
},
],
},
);
if (createResult.isError) {
throw new Error(getFirstText(createResult));
}
const dashboard = JSON.parse(getFirstText(createResult));

const result = await callTool(ctx.client!, 'clickstack_query_tile', {
dashboardId: dashboard.id,
tileId: dashboard.tiles[0].id,
startTime: new Date(now.getTime() - 15 * 60 * 1000).toISOString(),
endTime: new Date(now.getTime() + 60 * 1000).toISOString(),
});

expect(result.isError).toBeFalsy();
const parsed: { result: { data: Array<Record<string, string>> } } =
JSON.parse(getFirstText(result));
expect(parsed.result.data).toHaveLength(4);
for (const row of parsed.result.data) {
expect(row).toHaveProperty('__hdx_time_bucket');
}
});

// Regression: a metric source rendered as a categorical (bar/pie) tile with a
// groupBy runs the chart config through convertToCategoricalChartConfig, which
// structuredClones it. `source.metricTables` is a live Mongoose subdocument
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,48 @@ describe('MCP Dashboard Tools - clickstack_save_dashboard', () => {
});
});

it('should save and re-read a series heatmap tile on a Log source', async () => {
const logSource = await Source.create({
kind: SourceKind.Log,
team: ctx.team._id,
from: { databaseName: DEFAULT_DATABASE, tableName: 'otel_logs' },
timestampValueExpression: 'Timestamp',
connection: ctx.connection._id,
name: 'Logs',
});
const config = {
displayType: 'heatmap',
heatmapMode: 'series',
sourceId: logSource._id.toString(),
select: [
{
aggFn: 'avg',
valueExpression: 'Duration',
where: 'SeverityText:error',
whereLanguage: 'lucene',
alias: 'Avg duration',
},
],
groupBy: 'ServiceName',
};

const result = await callTool(ctx.client!, 'clickstack_save_dashboard', {
name: 'Series Heatmap Dashboard',
tiles: [{ name: 'Duration by service', config }],
});
expect(result.isError).toBeFalsy();
const saved = JSON.parse(getFirstText(result));
expect(saved.tiles[0].config).toMatchObject(config);

const get = await callTool(ctx.client!, 'clickstack_get_dashboard', {
id: saved.id,
});
expect(get.isError).toBeFalsy();
expect(JSON.parse(getFirstText(get)).tiles[0].config).toMatchObject(
config,
);
});

it('should round-trip every MCP-specific heatmap field through save, get, update, and re-get', async () => {
const sourceId = ctx.traceSource._id.toString();

Expand Down
7 changes: 4 additions & 3 deletions packages/api/src/mcp/prompts/dashboards/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Use BUILDER tiles (with sourceId) for most cases:
pie Proportional breakdowns (traffic share by service, errors by type). Keep slice count under 8.
bar Categorical comparisons (request counts by service, errors by endpoint). One bar per group value; not a time series (use stacked_bar for that).
heatmap Distribution of a numeric value over time (latency buckets, payload size). Trace sources only. Requires non-empty valueExpression.
With heatmapMode: "series": one row per groupBy value, colored by a normal select item. Trace, Log, or Metric sources.
search Browse raw log/event rows (error logs, recent traces).
markdown Use sparingly. The dashboard already shows its name in the title bar at the top; do NOT add a "About this dashboard" tile that repeats it. Markdown bodies render h1/h2/h3 headings at title-bar scale, so a single \`## Service Catalog\` line eats most of the tile and pushes real KPIs below the fold. Skip markdown tiles for starter dashboards. If you must add one, size it to fit the text (h: 2-3 for a line or two; h: 1 clips it), use plain prose, no \`#\`/\`##\`/\`###\` headings. Use containers/tabs for section grouping instead.
Expand All @@ -88,7 +89,7 @@ Apply these before calling clickstack_save_dashboard. Each rule is enforced by t
1. ONE QUESTION PER DASHBOARD. A dashboard answers a single observability question well. Split if the request mixes traces, logs, and metrics into unrelated views.
2. ALIAS EVERY SELECT ITEM. Every entry in select MUST carry an alias. Tables, lines, stacked_bars, pies, AND number tiles. The number-tile case is the one most often missed because the rendered UI uses the tile's name (not the column alias), so the absence looks invisible; the underlying query still emits a raw-expression column name (count(), quantile(0.95)(Duration)), which breaks orderBy references, CSV export, and any downstream onClick template that references the column by name. Treat "no alias" as a save-time bug, not a style nit. Heatmap is the only exception: heatmap select items take a valueExpression and no alias.
2. ALIAS EVERY SELECT ITEM. Every entry in select MUST carry an alias. Tables, lines, stacked_bars, pies, AND number tiles. The number-tile case is the one most often missed because the rendered UI uses the tile's name (not the column alias), so the absence looks invisible; the underlying query still emits a raw-expression column name (count(), quantile(0.95)(Duration)), which breaks orderBy references, CSV export, and any downstream onClick template that references the column by name. Treat "no alias" as a save-time bug, not a style nit. Distribution heatmaps are the only exception: their select items take a valueExpression and no alias.
Number tile, correct: select: [{ aggFn: "count", alias: "Server Requests" }]
Number tile, wrong: select: [{ aggFn: "count" }]
Expand Down Expand Up @@ -139,7 +140,7 @@ Apply these before calling clickstack_save_dashboard. Each rule is enforced by t
14. SIZE TILES TO FIT THEIR CONTENT. The layout w/h are not one-size-fits-all; a tile that is too short clips its content (a table loses rows below the fold, a number tile crops its label) and one that is too wide wastes the row. Match the size to the displayType: number tiles stay small (w 6-8, h 3-4) so three or four KPIs share a row; line / stacked_bar / pie / bar want w 8-12 and h 4-6; tables and search lists want the full row (w 24) and h 6-10 so rows are not cut off; heatmaps want w 12 and h 5-6; a markdown note wants h 2-3 (never h 1, which clips the text). The per-field w/h descriptions on the tile schema carry the same per-displayType ranges; reach for them instead of leaving every tile at the 12x4 default.
15. FILTER A BUILDER TILE ON THE SELECT ITEM, NOT THE TILE. To scope a table / line / stacked_bar / number / pie / bar tile to a subset of rows (a service, an error status), put the filter on EACH select item's where: select: [{ aggFn: "count", where: "ServiceName:payment", whereLanguage: "lucene", alias: "..." }]. The chart editor renders that per-series where as the tile's visible "Where" box, so the user can see and edit it. Do NOT put a filter at the tile config's top level for these types: the editor does not show it, so it is ignored. For a whole-dashboard scope use a dashboard-level filter (gotcha 9). The only display types with a tile-level where are search, heatmap, and event_patterns, where the editor does render it.
15. FILTER A BUILDER TILE ON THE SELECT ITEM, NOT THE TILE. To scope a table / line / stacked_bar / number / pie / bar tile to a subset of rows (a service, an error status), put the filter on EACH select item's where: select: [{ aggFn: "count", where: "ServiceName:payment", whereLanguage: "lucene", alias: "..." }]. The chart editor renders that per-series where as the tile's visible "Where" box, so the user can see and edit it. Do NOT put a filter at the tile config's top level for these types: the editor does not show it, so it is ignored. For a whole-dashboard scope use a dashboard-level filter (gotcha 9). The only display types with a tile-level where are search, distribution heatmap, and event_patterns, where the editor does render it. Series heatmaps filter on the select item like the other builder types.
== ADAPT, DO NOT COPY ==
Expand All @@ -165,7 +166,7 @@ Dashboards open with a 15-minute default window. There is no dashboard-level fie
- Multiple select items on number / pie / bar / heatmap tiles (each takes exactly one).
- Missing level on aggFn "quantile" (must specify 0.5, 0.9, 0.95, or 0.99).
- Assuming StatusCode or SeverityText values (always inspect lowCardinalityValues from clickstack_describe_source).
- Heatmap on a non-Trace source (heatmap is Trace-only today).
- Distribution heatmap on a non-Trace source, or series heatmap on a source that is not Trace, Log, or Metric.
- Hardcoding a focus dimension into every tile's where clause (use a dashboard-level filter instead).
- Putting a filter at the tile-config top level on a table / line / stacked_bar / number / pie / bar tile (ignored for these types; put the where on each select item instead, see gotcha 15).
- Enabling both isBroadcastEnabled and isVariableEnabled on one filter (the picked value is then applied twice; set isBroadcastEnabled: false when you add a variable). The exception is rule 9c: a variable read only by another filter's where never reaches a tile, so that filter can keep broadcasting.
Expand Down
Loading
Loading