Repository navigation
feat: expose group-by values to webhook bodies as {{var.<column>}} #3384
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
84289b5
e721f29
a5a015b
a4c8993
4b9f050
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| '@hyperdx/api': minor | ||
| '@hyperdx/app': minor | ||
| '@hyperdx/common-utils': minor | ||
| --- | ||
|
|
||
| feat: expose group-by values to webhook body templates as `{{var.<column>}}`, and keep them on resolve notifications |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -491,6 +491,7 @@ router.post( | |
| thresholdMax: 10, | ||
| value: 7, | ||
| groupKey: 'test-group', | ||
| groupValues: { group: 'test-group' }, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Send test fills only The PR says the test lets you check a body before an alert fires. But the test payload is hardcoded to Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| sourceQuery: 'SeverityText: "error"', | ||
| teamId: teamId.toString(), | ||
| note: 'Test webhook — no runbook', | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -562,6 +562,7 @@ const fireChannelEvent = async ({ | |
| teamId, | ||
| teamWebhooksById, | ||
| sampleLines, | ||
| groupValues: attributes, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — No test checks that real alert group values reach a webhook body as {{var.*}} If you deleted Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — No test covers the new The new unit test in generic.test.ts calls Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| }); | ||
| return { failures, timings, dispatchDurationMs }; | ||
| }; | ||
|
|
@@ -1286,6 +1287,8 @@ export const processAlert = async ( | |
| string, | ||
| { value: number; attributes: Record<string, string>; startTime: Date } | ||
| >(); | ||
| // ponytail: a group that vanished from the data resolves without its values. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Stray
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| const groupAttributes = new Map<string, Record<string, string>>(); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor —
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
|
|
||
| // Helper to get or create history for a group | ||
| const getOrCreateHistory = (groupKey: string): IAlertHistory => { | ||
|
|
@@ -1471,6 +1474,7 @@ export const processAlert = async ( | |
| group: groupKey, | ||
| totalCount: lastValue?.count || 0, | ||
| startTime: lastValue?.startTime || nowInMinsRoundDown, | ||
| attributes: groupAttributes.get(groupKey), | ||
| }); | ||
| } | ||
| }; | ||
|
|
@@ -1635,6 +1639,7 @@ export const processAlert = async ( | |
| ? extraFields.map(([k, v]) => `${k}:${v}`).join(', ') | ||
| : ''; | ||
| const attributes = hasGroupBy ? Object.fromEntries(extraFields) : {}; | ||
| groupAttributes.set(groupKey, attributes); | ||
|
|
||
| const exceeds = doesExceedThreshold(alert, value); | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -617,6 +617,7 @@ export const renderAlertTemplate = async ({ | |
| teamWebhooksById, | ||
| dispatcher = inlineNotificationDispatcher, | ||
| sampleLines, | ||
| groupValues, | ||
| }: { | ||
| alertProvider: AlertProvider; | ||
| clickhouseClient: ClickhouseClient; | ||
|
|
@@ -634,6 +635,8 @@ export const renderAlertTemplate = async ({ | |
| * per group; without one they are fetched here. | ||
| */ | ||
| sampleLines?: () => Promise<string>; | ||
| // Flat, unlike view.attributes, so a dotted column name stays one key. | ||
| groupValues?: Record<string, string>; | ||
| }): Promise<RenderedAlert> => { | ||
| // Internal mutable view with __hdx_query_results__ populated on the | ||
| // saved-search path. Untrusted values must flow through the view so | ||
|
|
@@ -752,6 +755,7 @@ export const renderAlertTemplate = async ({ | |
| : undefined, | ||
| value, | ||
| groupKey: group ?? '', | ||
| groupValues: groupValues ?? {}, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — No test checks that real alert notifications pass The new int test only checks the Slack title, which comes from the separate Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| sourceQuery: truncateString( | ||
| describeSourceQuery(alert, savedSearch, dashboard), | ||
| MAX_SOURCE_QUERY_LENGTH, | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,7 @@ import { | |
| WebhookTemplateVariable, | ||
| } from '@hyperdx/common-utils/dist/types'; | ||
| import Handlebars from 'handlebars'; | ||
| import _ from 'lodash'; | ||
| import { performance } from 'perf_hooks'; | ||
| import { serializeError } from 'serialize-error'; | ||
|
|
||
|
|
@@ -96,14 +97,18 @@ export const buildWebhookTemplateVariables = (message: Message) => | |
| alertType: escapeJsonString(message.alertType ?? ''), | ||
| comparator: escapeJsonString(message.comparator ?? ''), | ||
| groupKey: escapeJsonString(message.groupKey ?? ''), | ||
| var: _.mapValues(message.groupValues ?? {}, v => escapeJsonString(v)), | ||
| note: escapeJsonString(message.note ?? ''), | ||
| sourceQuery: escapeJsonString(message.sourceQuery ?? ''), | ||
| status: escapeJsonString(message.status ?? ''), | ||
| teamId: escapeJsonString(message.teamId ?? ''), | ||
| threshold: message.threshold, | ||
| thresholdMax: message.thresholdMax, | ||
| value: message.value, | ||
| }) satisfies Record<WebhookTemplateVariable, string | number | undefined>; | ||
| }) satisfies Record< | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Widened Adding Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Widening the Adding Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Widened Adding Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| WebhookTemplateVariable, | ||
| string | number | undefined | Record<string, string> | ||
| >; | ||
|
|
||
| /** | ||
| * Creates a Handlebars instance with common helpers registered. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,6 +58,7 @@ const describeTemplateVariable = ( | |
| thresholdMax: 'Upper bound of a between/outside range (number)', | ||
| value: 'Value that triggered or resolved the alert (number)', | ||
| groupKey: 'The breaching group, if grouped', | ||
| var: 'Group-by values by column, e.g. {{var.customer_id}}', | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Form lists
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — Variable list shows
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — The variable list shows
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 minor — The variable list shows
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond |
||
| sourceQuery: 'Search expression or SQL behind the alert', | ||
| startTime: 'Window start, Unix ms (number)', | ||
| endTime: 'Window end, Unix ms (number)', | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔵 minor — Bracket syntax in the docs doesn't cover map-attribute group-bys
parseAlertData(index.ts:940) uses ClickHouse's result column name as the key. For the common OTel group-byResourceAttributes['service.name'], the key is that whole expression, notservice.name. It also contains a], so the documented{{var.[service.name]}}renders empty and a plain[...]segment can't reach the key. Fix: say that the key is the literal group-by expression unless it is aliased, and document{{lookup var "ResourceAttributes['service.name']"}}for keys like this.Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond