Repository navigation
fix: render percentile aggFns as quantile with a level - #3357
Conversation
A select item with a percentile shorthand aggFn (`p50`, `p90`, `p95`,
`p99`) renders as a ClickHouse function of the same name, which does not
exist. The renderer passes any aggFn it does not recognise through as
`${fn}If(...)`, and the chart config schema accepts any string as aggFn.
Reproduction: in Chart Explorer, pick a trace source and ask the AI
assistant "show me slow spans for the checkout service". It returns a
table with two series:
{ "aggFn": "max", "valueExpression": "Duration", "aggCondition": "ServiceName = 'checkout'" }
{ "aggFn": "p95", "valueExpression": "Duration", "aggCondition": "ServiceName = 'checkout'" }
grouped by SpanName (`POST /cart/items`, `SELECT inventory`, ...). The
chart fails with:
Function with name `p95If` does not exist. In scope SELECT
maxIf(toFloat64OrDefault(toString(Duration)), ...),
p95If(toFloat64OrDefault(toString(Duration)), ...), SpanName ...
The series editor shows the second series as "95th Percentile", because
the dropdown option values are `p50`..`p99`; only picking an option
converts it to `quantile` + level, so the config looks valid in the UI.
The same mapping gap produces a silently wrong result in two places:
- The AI assistant schema offers `quantile` with no level. A config with
`aggFn: 'quantile'` and no level renders as `quantileIf(expr, cond)`,
which ClickHouse evaluates as the median.
- v1 metric chart conversion maps `p95` to `quantile` and drops the
level, so converted percentile charts also show the median.
Changes:
- common-utils: add `resolvePercentileAggFn`, which rewrites
p50/p90/p95/p99 as `quantile` at the matching level, and apply it to
the select list in `renderChartConfig` before any SQL is built. `p95`
now renders as `quantileIf(0.95)(...)`, or
`quantileTDigestWeighted(0.95)(...)` on sample-weighted sources.
- api: the AI assistant schema and prompt offer p50/p90/p95/p99 instead
of a bare `quantile`, and the response mapping resolves them to
`quantile` + level.
- app: v1 metric chart conversion keeps the percentile level.
🦋 Changeset detectedLatest commit: 86410be The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Hi @octo47, thanks for the pull request! Before we review code from a first-time contributor we ask that a maintainer vouches for you, and you're not on our list yet. This PR stays open — it just isn't in the review queue until someone vouches. To get vouched, open an issue saying hello and what you're working on: https://github.com/hyperdxio/hyperdx/issues/new?template=introduce-yourself.md A maintainer will usually reply within a day or two, and then this PR gets picked up as normal. More detail in our contributing guide. |
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — v1 conversion repeats the _rate stripping and percentile check that mapV1AggFnToV2 already does
Have mapV1AggFnToV2 return { aggFn, level }, or wrap its result with resolvePercentileAggFn on the _rate-stripped aggFn. Right now the call site strips _rate with its own regex and computes the level separately from the function that decides the aggFn is quantile, so the two can disagree.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
PR Review
2 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 2 minor 1 posted as inline comment(s) on the changed lines. 1 listed below. Findings outside the changed lines1 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
[Medium impact] The PR appears safe to merge. SummaryThe PR maps percentile shortcuts to
No new actionable issues were found. Reviews (7) · Last reviewed commit: "fix mock source lint" · Reviewed by Greptile |
Deep Review✅ No critical issues found. The core fix is sound and verified: 🟡 P2 -- recommended
Reviewers (10): correctness, testing, maintainability, project-standards, kieran-typescript, api-contract, adversarial, previous-comments, agent-native, learnings-researcher. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
|
@octo47 is attempting to deploy a commit to the HyperDX Team on Vercel. A member of the Team first needs to authorize it. |
- Type-check PERCENTILE_AGG_FN_LEVELS against PercentileAggregateFunctionSchema with `satisfies`, so a percentile added to only one of them fails to compile. - Type resolvePercentileAggFn's return as `T | QuantileResolvedSelect<T>` instead of casting the rewritten item to T. - Use getPercentileAggFnLevel in AggFnSelect and mapV1AggFnToV2 in place of their own p50/p90/p95/p99 lists.
| const substitutedChartConfig = substituteChartConfigVariables(rawChartConfig); | ||
| const variableSubstitutedChartConfig = | ||
| substituteChartConfigVariables(rawChartConfig); | ||
| const substitutedChartConfig = Array.isArray( |
There was a problem hiding this comment.
🔵 minor — Percentiles are only resolved inside renderChartConfig, so the materialized-view optimizer still rejects saved p95 configs
tryOptimizeConfigWithMaterializedView runs on the raw config before renderChartConfig. There, isValidAggFn('p95') fails and convertSelectToMaterializedViewSelect throws "Aggregate function p95 is not valid" (packages/common-utils/src/core/materializedViews.ts:402). A saved config with aggFn: 'p95' therefore skips an MV that has a quantile column and shows a misleading reason. Fix: also apply resolvePercentileAggFn to the select list at the start of the MV optimizer (or in a shared normalization step both paths call), not only in the renderer.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — v1 conversion works out the percentile level separately from the aggFn mapping
mapV1AggFnToV2 already strips _rate and maps percentiles to quantile. The new code strips _rate again with its own regex and calls getPercentileAggFnLevel a second time. Replace both with one resolvePercentileAggFn({ aggFn: s.aggFn?.replace(/_rate$/, ''), ... }) before the remaining v1→v2 mapping, so the aggFn and its level come from one call.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| 'p95', | ||
| 'p99', | ||
| ]); | ||
| export const PERCENTILE_AGG_FN_LEVELS = { |
There was a problem hiding this comment.
🔵 minor — PERCENTILE_AGG_FN_LEVELS is not the only percentile mapping: hand-written reverse mappings remain
formatAggregateFunction in packages/common-utils/src/core/materializedViews.ts:743 hard-codes 0.9→'p90', 0.95→'p95', 0.99→'p99'. AggFnSelectControlled (packages/app/src/components/AggFnSelect.tsx:154) builds p${Math.round(level*100)}. Neither reads the new table, so a new level (e.g. p75) has to be added in all three places by hand. Derive the reverse lookup from PERCENTILE_AGG_FN_LEVELS and use it in both places.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const substitutedChartConfig = substituteChartConfigVariables(rawChartConfig); | ||
| const variableSubstitutedChartConfig = | ||
| substituteChartConfigVariables(rawChartConfig); | ||
| const substitutedChartConfig = Array.isArray( |
There was a problem hiding this comment.
🔵 minor — Percentile shorthand is resolved only inside renderChartConfig, so the materialized-view optimizer still rejects p95
tryOptimizeConfigWithMaterializedView runs on the config before renderChartConfig. Its convertSelectToMaterializedViewSelect throws Aggregate function p95 is not valid because of the isValidAggFn check in packages/common-utils/src/core/materializedViews.ts:402. As a result, a saved aggFn: 'p95' tile never uses a quantile MV column and falls back to the raw table. Apply resolvePercentileAggFn to the select list in the MV optimizer as well, or in a shared normalization step that both paths call, so the two paths agree.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| @@ -1218,8 +1214,12 @@ export const convertV1ChartConfigToV2 = ( | |||
| const metricDataType = z | |||
| .nativeEnum(MetricsDataTypeV2) | |||
| .parse(rawMetricDataType?.toLowerCase()); | |||
| const percentileLevel = getPercentileAggFnLevel( | |||
There was a problem hiding this comment.
🔵 minor — _rate suffix stripping is duplicated next to mapV1AggFnToV2
mapV1AggFnToV2 already strips _rate and recognises percentiles (lines 1114-1121), and the new code strips it again with a slightly different expression (/_rate$/ vs replace('_rate', '')). Have mapV1AggFnToV2 return { aggFn, level } so the conversion logic lives in one place.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
ea76f68 to
2ea6b19
Compare
| const substitutedChartConfig = substituteChartConfigVariables(rawChartConfig); | ||
| const variableSubstitutedChartConfig = | ||
| substituteChartConfigVariables(rawChartConfig); | ||
| const substitutedChartConfig = Array.isArray( |
There was a problem hiding this comment.
🔵 minor — Percentile rewrite runs only inside renderChartConfig, so the MV optimizer still gets the raw p95 and rejects it
tryOptimizeConfigWithMaterializedViewWithExplanations runs on the config before renderChartConfig is called. Its callers are packages/api/src/tasks/checkAlerts/index.ts:1194 and packages/app/src/hooks/useMVOptimizationExplanation.tsx:55. When it sees aggFn: 'p95', isValidAggFn fails and it throws Aggregate function p95 is not valid (packages/common-utils/src/core/materializedViews.ts:402). As a result a p95 tile never uses an MV quantile(0.95) column, and the MV explanation shows a misleading error. Fix: apply resolvePercentileAggFn at a shared normalization point that both the MV optimizer and the renderer go through, or call it at the start of tryOptimizeConfigWithMaterializedViewWithExplanations as well.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — The _rate suffix is stripped and the percentile looked up in two places that must be kept in sync by hand
mapV1AggFnToV2 already strips /_rate$/ and calls getPercentileAggFnLevel (lines 1107-1113). The metric branch now repeats both steps to get the level. Have mapV1AggFnToV2, or a small helper next to it, return { aggFn, level } in one pass, so the aggFn mapping and the level cannot drift apart.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| databaseName: 'default', | ||
| tableName: 'otel_traces', | ||
| }, | ||
| select: [], |
There was a problem hiding this comment.
🔵 minor — New percentile render tests never use a metric source, though that path depends on the rewrite running first
Both new tests use a trace table. The rewrite is placed before the multi-series metric branch and translateMetricChartConfig because histogram translation branches on select.aggFn === 'quantile' (packages/common-utils/src/core/histogram.ts:75). Moving the rewrite after metric translation would still pass every new test. Add a case for a histogram metric with aggFn: 'p95', and one for a multi-series metric with aggFn: 'p95', that assert the quantile SQL at level 0.95.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const substitutedChartConfig = substituteChartConfigVariables(rawChartConfig); | ||
| const variableSubstitutedChartConfig = | ||
| substituteChartConfigVariables(rawChartConfig); | ||
| const substitutedChartConfig = Array.isArray( |
There was a problem hiding this comment.
🔵 minor — Percentile aggFns are only resolved inside renderChartConfig, so materialized-view optimization still rejects p95 configs
The app calls tryOptimizeConfigWithMaterializedView on the config before it reaches renderChartConfig. There, convertSelectToMaterializedViewSelect (packages/common-utils/src/core/materializedViews.ts:402) throws Aggregate function p95 is not valid because p95 isn't in InternalAggregateFunctionSchema. A saved aggFn: 'p95' config now renders correctly, but it always skips the MV and reports that error as the explanation, even when the MV has a matching quantile column. Fix: also apply resolvePercentileAggFn to config.select at the start of tryOptimizeConfigWithMaterializedViewWithExplanations, or normalize where configs are loaded, so every consumer sees quantile + level.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — Percentile level is derived separately from mapV1AggFnToV2, duplicating the _rate stripping
mapV1AggFnToV2 already strips _rate and detects percentiles. The call site repeats both (s.aggFn?.replace(/_rate$/, '') plus getPercentileAggFnLevel) to get the level, so the two must be kept in sync by hand. Fix: strip _rate once and build the item with resolvePercentileAggFn({ aggFn: base, ... }), or have the mapper return { aggFn, level }, so the aggFn and its level come from one place.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
karl-power
left a comment
There was a problem hiding this comment.
Looks good. Thanks for contribution! Just one change needed to pass the checks then we can merge.
| const substitutedChartConfig = substituteChartConfigVariables(rawChartConfig); | ||
| const variableSubstitutedChartConfig = | ||
| substituteChartConfigVariables(rawChartConfig); | ||
| const substitutedChartConfig = Array.isArray( |
There was a problem hiding this comment.
🔵 minor — Percentile shorthand is resolved only inside the SQL renderer, so consumers that run before it still see p95
tryOptimizeConfigWithMaterializedView runs before renderChartConfig. It only accepts aggFn === 'quantile' with a numeric level (isQuantileSelectItem), and isValidAggFn rejects p95 (both in packages/common-utils/src/core/materializedViews.ts:275-291). So a p95 select item never uses a quantile MV, and the editor keeps storing p95. Normalize where the config enters the system instead, for example a Zod preprocess on the select item schema, or when the config is loaded or saved. That way every consumer sees quantile and a level, not just the renderer.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| 'p95', | ||
| 'p99', | ||
| ]); | ||
| export const PERCENTILE_AGG_FN_LEVELS = { |
There was a problem hiding this comment.
🔵 minor — PERCENTILE_AGG_FN_LEVELS restates percentile/level mappings that already exist elsewhere
The same set of levels is already defined in externalQuantileLevelSchema (packages/api/src/utils/zod.ts:229). The reverse mapping is hand-written in formatAggregateFunction (packages/common-utils/src/core/materializedViews.ts:743) and as p${Math.round(level*100)} in AggFnSelectControlled (packages/app/src/components/AggFnSelect.tsx:154). Build these from PERCENTILE_AGG_FN_LEVELS (for example z.union(Object.values(...).map(z.literal)) and a reverse lookup), or add doc comments naming the twins. Otherwise adding p75 means updating four places by hand.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — The v1 conversion strips _rate and checks for a percentile twice to get aggFn and level
mapV1AggFnToV2 already strips _rate and calls getPercentileAggFnLevel. The new code at lines 1217-1219 does both again separately to recover the level. Have mapV1AggFnToV2 return { aggFn, level? }, or run resolvePercentileAggFn on the mapped item, so the rate-suffix and percentile logic lives in one place.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| const metricDataType = z | ||
| .nativeEnum(MetricsDataTypeV2) | ||
| .parse(rawMetricDataType?.toLowerCase()); | ||
| const percentileLevel = getPercentileAggFnLevel( |
There was a problem hiding this comment.
🔵 minor — The _rate suffix is stripped in two places to work out one percentile
mapV1AggFnToV2 already strips _rate and checks for a percentile. The caller now strips _rate again and runs a second getPercentileAggFnLevel lookup, so the aggFn and the level come from two parallel code paths that have to be kept in sync. Instead, have mapV1AggFnToV2 return { aggFn, level? }. Or let it return the base aggFn (for example p95) and pass the select item through resolvePercentileAggFn. Either way, the percentile is resolved in one place.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
A select item with a percentile shorthand aggFn (
p50,p90,p95,p99) renders as a ClickHouse function of the same name, which does not exist. The renderer passes any aggFn it does not recognise through as${fn}If(...), and the chart config schema accepts any string as aggFn.Reproduction: in Chart Explorer, pick a trace source and ask the AI assistant "show me slow spans for the checkout service". It returns a table with two series:
{ "aggFn": "max", "valueExpression": "Duration", "aggCondition": "ServiceName = 'checkout'" }
{ "aggFn": "p95", "valueExpression": "Duration", "aggCondition": "ServiceName = 'checkout'" }
grouped by SpanName (
POST /cart/items,SELECT inventory, ...). The chart fails with:Function with name
p95Ifdoes not exist. In scope SELECTmaxIf(toFloat64OrDefault(toString(Duration)), ...),
p95If(toFloat64OrDefault(toString(Duration)), ...), SpanName ...
The series editor shows the second series as "95th Percentile", because the dropdown option values are
p50..p99; only picking an option converts it toquantile+ level, so the config looks valid in the UI.The same mapping gap produces a silently wrong result in two places:
quantilewith no level. A config withaggFn: 'quantile'and no level renders asquantileIf(expr, cond), which ClickHouse evaluates as the median.p95toquantileand drops the level, so converted percentile charts also show the median.Changes:
resolvePercentileAggFn, which rewrites p50/p90/p95/p99 asquantileat the matching level, and apply it to the select list inrenderChartConfigbefore any SQL is built.p95now renders asquantileIf(0.95)(...), orquantileTDigestWeighted(0.95)(...)on sample-weighted sources.quantile, and the response mapping resolves them toquantile+ level.Summary
Screenshots or video
How to test on Vercel preview
Preview routes:
Steps:
References