Skip to content

feat: expose group-by values to webhook bodies as {{var.<column>}} - #3384

Merged
jordan-simonovski merged 5 commits into
mainfrom
jordansimonovski/webhook-group-by-vars
Oct 21, 2026
Merged

jordan-simonovski merged 5 commits into
mainfrom
jordansimonovski/webhook-group-by-vars

Conversation

@jordan-simonovski

Copy link
Copy Markdown
Contributor

Summary

Webhook bodies for grouped alerts can now name the group they're about: {"summary": "{{var.customer_id}} has an elevated error rate"} renders the breaching group's value. Before, the only group variable was {{groupKey}}, a joined string like customer_id:acme, region:eu that a receiver had to parse.

  • {{var.<column>}} works in Generic and incident.io bodies, keyed by group-by column or SQL alias. Values are JSON-escaped like the other string variables.
  • Resolve notifications now carry the group's values. {{attributes.x}} in alert names and messages used to render empty on resolve, so a firing title of "Errors for service-b" resolved as "Errors for ".
  • "Send test" fills {{var.group}} so a body can be checked before an alert fires.
  • A group that drops out of the data entirely (errors stop, so it has no rows) still resolves with empty values. The only record left of it is groupKey, and parsing values back out of that string isn't safe.
  • Column names with dots need bracket syntax: {{var.[service.name]}}. The docs say so.

Screenshots or video

The webhook form's variable list gains a {{var}} row. Screenshots not captured; I'll add them before review.

How to reproduce

Starting point: yarn dev (alerts and webhooks need the API).

On this branch:

  1. Team settings: add a Generic webhook with body {"text": "{{var.ServiceName}} has errors"} pointed at a request bin.
  2. Create a saved-search alert on errors with group by ServiceName, using that webhook.
  3. When it fires, the payload reads e.g. "checkout has errors". On main the same body renders " has errors".

How to test on Vercel preview

N/A: needs the API.

Generic and incident.io webhook bodies can reference the breaching group's
values by column, e.g. "{{var.customer_id}} has an elevated error rate".
Resolve notifications now carry the group's values too, so {{attributes.x}}
in alert names and messages no longer renders empty on resolve.
@changeset-bot

changeset-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 4b9f050

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/api Minor
@hyperdx/app Minor
@hyperdx/common-utils Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Oct 21, 2026 2:56am UTC
hyperdx-storybook Ready Ready Preview Oct 21, 2026 2:56am UTC

Request Review

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Oct 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Touches API routes or data models — hidden complexity risk

Additional context: touches background tasks or the delivery pipeline lightly (18 lines, under the 30-line bar for Tier 4), spans packages but only 21 prod lines — under the 100-line cross-layer bar

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 7
  • Production lines changed: 21 (+ 78 in test files, excluded from tier calculation)
  • Branch: jordansimonovski/webhook-group-by-vars
  • Author: jordan-simonovski

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 123 in total (1.62%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@@ -21,6 +21,7 @@ a HyperDX alert without parsing the human-readable message body.
| `{{thresholdMax}}` | number | Upper bound of a `between`/`outside` condition. Always unset for every other comparator, even if the alert was once a range, so guard it (see below). |
| `{{value}}` | number | The value that triggered or resolved the alert. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Bracket syntax in the docs doesn't cover map-attribute group-bys

parseAlertData (index.ts:940) uses ClickHouse's result column name as the key. For the common OTel group-by ResourceAttributes['service.name'], the key is that whole expression, not service.name. It also contains a ], so the documented {{var.[service.name]}} renders empty and a plain [...] segment can't reach the key. Fix: say that the key is the literal group-by expression unless it is aliased, and document {{lookup var "ResourceAttributes['service.name']"}} for keys like this.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: 'Upper bound of a between/outside range (number)',
value: 'Value that triggered or resolved the alert (number)',
groupKey: 'The breaching group, if grouped',
var: 'Group-by values by column, e.g. {{var.customer_id}}',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Form lists {{var}} as a variable, but using it as shown renders [object Object]

getWebhookTemplateVariables displays each entry as {{name}}, so the list shows {{var}} next to scalar variables. That object is useless on its own: a body containing {{var}} renders [object Object]. Fix: display this entry as {{var.<column>}}, for example by special-casing object-valued names in getWebhookTemplateVariables, so the label shows the form users actually need.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

4 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 4 minor

4 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Grouped alerts now expose individual group values through {{var.<column>}} in Generic and incident.io webhook bodies.

  • Webhook bodies can name each alert group value by column.
  • Resolve notifications keep values for groups that remain in the data.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Grouped query row] --> B[Flat groupValues]
  B --> C[Firing or resolve message]
  C --> D[JSON-escape each value]
  D --> E["Render {{var.column}}"]
  E --> F[Generic or incident.io webhook]
Loading

Reviews (5) · Last reviewed commit: "Merge branch 'main' into jordansimonovsk..." · Reviewed by Greptile

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical (P0/P1) issues found. The change reuses the established escapeJsonString escaping path, correctly prevents JSON-slot breakout and Handlebars template re-evaluation of group values, and introduces no injection, SSRF, or auth regression. The resolve-path fix (carrying groupAttributes onto the OK notification) is sound, and the "vanished group resolves empty" behavior is intentional and matches the code.

🟡 P2 — recommended

  • packages/api/src/tasks/checkAlerts/transports/generic.ts:99 — The new var entry maps group values straight into escapeJsonString (v => escapeJsonString(v)) with no ?? '' or string coercion, unlike every other string variable on the object which guards with ?? ''; escapeJsonString is JSON.stringify(str).slice(1,-1), so a non-string group-by value mangles (a numeric column value 200 renders as "0", 42 as "2") and an undefined value throws inside the builder, failing that notification's dispatch. The attributes type is Record<string, string> only by a documented TODO, and the values originate from ClickHouse group-by columns which can be numeric or Nullable.
    • Fix: Coerce each value before escaping, e.g. escapeJsonString(String(v ?? '')).
    • correctness-analysis, adversarial-analysis
  • packages/api/src/tasks/checkAlerts/transports/__tests__/generic.test.ts:131 — The new behavior documented in docs/alert-webhook-template-variables.md is only partially tested: there is no coverage for the dotted-column bracket syntax {{var.[service.name]}}, the vanished-group empty-values resolve, a group value containing JSON metacharacters, or the firing-path (not just resolve-title) {{var.<column>}} rendering through the real alert pipeline; the integration test asserts only the resolve title via {{attributes.ServiceName}}, not a var-bearing body.
    • Fix: Add unit/integration cases for the bracket syntax, a vanished-group resolve, and a JSON-metacharacter group value.
    • testing-analysis, security
🔵 P3 nitpicks (2)
  • packages/api/src/tasks/checkAlerts/transports/generic.ts:99 — A bare {{var}} in a user body (rather than {{var.<column>}}) renders as [object Object], which breaks the surrounding JSON payload; only the dotted form is safe and the docs rely on authors knowing this.
    • Fix: Document the constraint inline near the variable list, or render the bare object as an empty string.
  • packages/api/src/tasks/checkAlerts/transports/generic.ts:116 — Widening the satisfies Record<WebhookTemplateVariable, string | number | undefined | Record<string,string>> constraint loosens the type for every key, not just var, so another variable could silently be assigned an object without a type error.
    • Fix: Keep the object type scoped to var only (e.g. a per-key mapped type) if practical.

Reviewers (returned at synthesis): security, previous-comments, agent-native, plus orchestrator diff analysis (correctness/adversarial/testing lenses). Additional persona reviewers (api-contract, reliability, maintainability, project-standards, kieran-typescript, learnings) were dispatched but had not returned when output was required; findings above reflect completed results and direct code reading.

Note: ponytail: comments and the import _ from 'lodash' default import are established conventions in this directory (confirmed against sibling files) and were deliberately not flagged. Security review confirmed no JSON-injection, template-injection, prototype-pollution, or SSRF vector is introduced.

Testing gaps:

  • No assertion that a group value containing JSON metacharacters stays escaped and cannot alter the parsed webhook JSON structure.
  • No coverage of a __proto__/constructor group-by column name.
  • Incident.io body path not exercised for {{var.<column>}} despite being in scope.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 445 passed • 1 skipped • 1430s

Status Count
✅ Passed 445
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

pulpdrew
pulpdrew previously approved these changes Oct 16, 2026

@pulpdrew pulpdrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

thresholdMax: 10,
value: 7,
groupKey: 'test-group',
groupValues: { group: 'test-group' },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Send test fills only var.group, so a body using a real group-by column can't be checked

The PR says the test lets you check a body before an alert fires. But the test payload is hardcoded to { group: 'test-group' }, so {{var.customer_id}} or {{var.ServiceName}} (the PR's own repro) still renders as an empty string. Fix: walk the body's Handlebars AST for var.<name> paths, the way findUndeliverableConstruct in packages/api/src/utils/webhookBodyTemplate.ts already walks it, and fill each one with a sample value.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: 'Upper bound of a between/outside range (number)',
value: 'Value that triggered or resolved the alert (number)',
groupKey: 'The breaching group, if grouped',
var: 'Group-by values by column, e.g. {{var.customer_id}}',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Variable list shows {{var}} as a token to use, but on its own it renders [object Object]

getWebhookTemplateVariables wraps every name as {{name}}, so the form lists {{var}}. Pasting that into a body puts [object Object] in the payload. Every other entry in the list works as a complete token. Fix: render this entry as {{var.<column>}}, either with a display-name override for var or by building the name from the descriptions map.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: message.thresholdMax,
value: message.value,
}) satisfies Record<WebhookTemplateVariable, string | number | undefined>;
}) satisfies Record<

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Widened satisfies lets any template variable be an object, which weakens the type check

Adding Record<string, string> to the shared value union means title, body and the rest could become objects without a type error. Fix: give var its own type, e.g. satisfies Record<Exclude<WebhookTemplateVariable, 'var'>, string | number | undefined> & { var: Record<string, string> }.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

: undefined,
value,
groupKey: group ?? '',
groupValues: groupValues ?? {},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — No test checks that real alert notifications pass groupValues through to the webhook message

The new int test only checks the Slack title, which comes from the separate attributes path. generic.test.ts calls buildWebhookTemplateVariables directly. So if groupValues: attributes in fireChannelEvent or groupValues ?? {} here were dropped, every test would still pass. Fix: add a checkAlerts int test with a Generic webhook whose body uses {{var.ServiceName}}, and assert the rendered request body on both fire and resolve.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

string,
{ value: number; attributes: Record<string, string>; startTime: Date }
>();
// ponytail: a group that vanished from the data resolves without its values.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Stray ponytail: prefix in a code comment

ponytail: doesn't match any comment-tag convention in this file and looks like a leftover personal marker. Drop the prefix and keep the explanation of why a vanished group resolves without its values.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

The group-by test cast the mock and its argument, adding two
no-unsafe-type-assertion warnings that put @hyperdx/api over its lint
warning limit. Use jest.mocked like the rest of the file.
teamId,
teamWebhooksById,
sampleLines,
groupValues: attributes,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — No test checks that real alert group values reach a webhook body as {{var.*}}

If you deleted groupValues: attributes from fireChannelEvent, or groupValues: groupValues ?? {} from template.ts:758, every test would still pass. generic.test.ts calls buildWebhookTemplateVariables directly, webhooks.int.test.ts only covers the hardcoded test-send payload, and the new checkAlerts test only checks the Slack title, which comes from view.attributes. Add a checkAlerts int test with a generic webhook whose body is {{var.ServiceName}}, and assert the posted body on both the firing and the resolve notification.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: 'Upper bound of a between/outside range (number)',
value: 'Value that triggered or resolved the alert (number)',
groupKey: 'The breaching group, if grouped',
var: 'Group-by values by column, e.g. {{var.customer_id}}',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The variable list shows {{var}}, and using it as shown sends [object Object]

getWebhookTemplateVariables displays every entry as {{name}}. Every other entry in the list works if copied as-is, but {{var}} is an object, so Handlebars renders it as the string [object Object] in the request body. Show this entry as {{var.<column>}}, or skip the {{name}} wrapping for var, so the copyable token actually works.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: message.thresholdMax,
value: message.value,
}) satisfies Record<WebhookTemplateVariable, string | number | undefined>;
}) satisfies Record<

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Widening the satisfies value type lets any variable be an object

Adding Record<string, string> to the shared value union means a mistake like title: someObject now type-checks and renders [object Object]. Keep the old union for the scalar variables and type only var as the object, e.g. satisfies Record<Exclude<WebhookTemplateVariable, 'var'>, string | number | undefined> & { var: Record<string, string> }.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

teamId,
teamWebhooksById,
sampleLines,
groupValues: attributes,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — No test covers the new groupValues path from a real alert to the webhook body

The new unit test in generic.test.ts calls buildWebhookTemplateVariables directly. The new checkAlerts int test only checks the Slack title, which comes from view.attributes, not groupValues. So deleting groupValues: attributes here, or groupValues: groupValues ?? {} in template.ts:758, would still pass every test. Add an int test where a grouped alert uses a Generic webhook with body {{var.ServiceName}}, and assert the rendered body on both fire and resolve.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: 'Upper bound of a between/outside range (number)',
value: 'Value that triggered or resolved the alert (number)',
groupKey: 'The breaching group, if grouped',
var: 'Group-by values by column, e.g. {{var.customer_id}}',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The variable list shows {{var}}, which renders [object Object] if pasted as written

getWebhookTemplateVariables displays every entry as {{${name}}}. Every other row can be pasted as-is, but {{var}} in a body renders the object as [object Object]. Display this entry as {{var.<column>}}, either with a per-variable display-name override or by special-casing object variables in getWebhookTemplateVariables.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

{ value: number; attributes: Record<string, string>; startTime: Date }
>();
// ponytail: a group that vanished from the data resolves without its values.
const groupAttributes = new Map<string, Record<string, string>>();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — groupAttributes repeats attributes already stored in bucketEvaluations and latestAlertContext

groupKey is built deterministically from extraFields, so the new groupAttributes map always holds the same attributes as bucketEvaluations[groupKey].attributes and latestAlertContext[groupKey].attributes. That leaves three copies. Use groupAttributes.get(groupKey) in the fire path too and remove attributes from the other two maps. Also replace the ponytail: prefix on the comment with a plain explanation.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

thresholdMax: message.thresholdMax,
value: message.value,
}) satisfies Record<WebhookTemplateVariable, string | number | undefined>;
}) satisfies Record<

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Widened satisfies type lets every template variable be an object

Adding Record<string, string> to the shared value union means a scalar variable such as title or groupKey could become an object and still compile. That silently loosens the guarantee the comment above claims. Keep the scalar union for the other keys and type var alone as Record<string, string>, e.g. Record<Exclude<WebhookTemplateVariable, 'var'>, string | number | undefined> & { var: Record<string, string> }.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 4b9f050f Deployed Oct 21, 2026 by vercel[bot]
Preview – hyperdx-storybook — 4b9f050f Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants