Skip to content

feat: Add promql var format for flexible escaping - #3297

Merged
kodiakhq[bot] merged 4 commits into
mainfrom
drew/promql-regex-formatting
Oct 8, 2026
Merged

kodiakhq[bot] merged 4 commits into
mainfrom
drew/promql-regex-formatting

Conversation

@pulpdrew

@pulpdrew pulpdrew commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR adds a new format for variable references - promql. This is intended to be used in promql expressions and is the new default when another format is not specified. This replaces the previous default (regex).

The new format is identical in behavior to the previous format, except that single-select variables are not regex-escaped.

Motivation: Single select variables are typically referenced in PromQL with field="$var". If $var is regex escaped, that doesn't work. Multi-select should always be regex escaped and referenced via the regex match operator field=~"$var".

This is backwards compatible except for references to single-select variables (#3295), which have not yet been released.

Screenshots or video

Screenshot 2026-10-05 at 1 04 19 PM

How to test locally

  1. Create a timeseries / promql source off of metrics_ts timeseries table
  2. Create a dashboard with a promql labels variable
  3. Reference the variable in a promql tile

References

  • Linear Issue: Closes HDX-5563
  • Related PRs:

@changeset-bot

changeset-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 50d0434

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Oct 8, 2026 9:28am UTC
hyperdx-storybook Ready Ready Preview Oct 8, 2026 9:28am UTC

Request Review

Comment thread packages/common-utils/src/variables.ts Resolved
Comment thread packages/common-utils/src/variables.ts Resolved
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

2 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 2 minor

None could be anchored to changed lines; they are listed below.

Findings outside the changed lines

2 minor
  • 🔵 packages/app/tests/e2e/features/dashboard-filter-variables.spec.ts:233 — E2E single-select test also passes under the old regex default → Use a value that contains a regex metacharacter, or assert on the generated PromQL text. As written, the test passes before and after this PR: 'api-server' (seed-clickhouse.ts:76) has no characters that escapeRegexValue would escape, and the old default's empty value .* inside service="..." also matches nothing.
  • 🔵 packages/common-utils/src/filters.ts:1115 — Single-select promql reference with an empty selection is not flagged as pending → Exclude promql here only when the variable is multi-select. For a single-select variable, formatVariableValues (variables.ts:101) renders an empty selection as '', so ServiceName = '${svc:promql}' matches no rows and the dependent dropdown comes up empty instead of showing the pending state. Look up the variable and use isFilterSingleSelect, add a test with maxSelections: 1, and update the doc comment at filters.ts:1088-1092, which still lists only regex as safe.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@greptile-apps

greptile-apps Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Adds a new variable format for PromQL dashboard filters.

The PR appears safe to merge; no new actionable issue was identified.

Summary

The PR adds a promql variable format that renders single-select values for exact matchers and multi-select values as escaped regexes. It makes that format the PromQL default and updates dashboard guidance, editor completions, and tests.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[PromQL variable reference] --> B{Single-select?}
  B -->|Yes| C[Value escaped for string literal]
  B -->|No| D[Regex-escaped selection]
  C --> E[Exact matcher]
  D --> F[Regex matcher]
Loading

Reviews (5) · Last reviewed commit: "Merge branch 'main' into drew/promql-reg..." · Reviewed by Greptile

Comment thread packages/common-utils/src/variables.ts Resolved
Comment thread packages/common-utils/src/variables.ts Resolved
Comment thread packages/common-utils/src/filters.ts Resolved
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

This PR adds a promql variable-reference format and makes it the PromQL default (replacing regex). Single-select variables (maxSelections: 1) render their raw first value unescaped for exact matchers (label="$var"), while multi-select variables keep regex-escaping for label=~"$var". The signature of formatVariableValues changed from (values: string[], format) to (variable, format), and isFilterSingleSelect moved from filters.ts to variables.ts.

Mechanically the change is clean: all internal callers of formatVariableValues (both in variables.ts) and every isFilterSingleSelect import were updated, no stale imports remain, values[0] ?? '' correctly mirrors the single-value capping already done in resolveFilterSelection, and the new behavior is well unit-tested. The remaining concerns are query-semantics edge cases around the empty and regex-matcher cases.

✅ No critical issues found. The single-select reference behavior these findings touch was introduced in the as-yet-unreleased #3295, so none is a released regression — but each is a concrete code path worth closing before this ships.

🟡 P2 -- recommended

  • packages/common-utils/src/variables.ts:102 -- An empty optional single-select variable renders service="", which in PromQL matches every series that lacks the service label (or has it empty), so an unselected tile can show unintended series instead of nothing, unlike the multi-select empty case which renders match-all .*.
    • Fix: Decide and encode the intended empty single-select semantics (e.g., render a never-match sentinel or surface a pending-selection state) and add a test covering series that lack the label.
  • packages/common-utils/src/variables.ts:101 -- A single-select value placed in a regex matcher {service=~"$svc"} now expands unescaped (a.b instead of a\.b), silently changing which series match, and validateVariableReferencesInTemplate only warns on unquoted references, not on this quoted =~-with-single-select combination.
    • Fix: Emit a validation warning when a single-select reference is used inside a =~ matcher, steering the author to = or to ${svc:regex}.
  • packages/common-utils/src/filters.ts:1114 -- getPendingFilterValuesVariables now exempts every promql reference from the pending-dependency check, but an empty single-select ${svc:promql} renders '' rather than a match-all, so a SQL-input condition like ServiceName = '${svc:promql}' silently queries for an empty value with no "waiting on dependency" warning.
    • Fix: Narrow the exemption so only multi-select promql references (which render .*) are exempted, leaving empty single-select references reported as pending.
🔵 P3 nitpicks (1)
  • packages/common-utils/src/variables.ts:101 -- formatVariableValues now couples value rendering to selection-cardinality configuration (maxSelections) by requiring the whole variable, which widens the function's input contract beyond pure value formatting.
    • Fix: Consider passing an explicit singleSelect: boolean to keep the renderer decoupled from filter-config shape, or document the coupling in the function's doc comment.

Reviewers (6): correctness, adversarial, typescript, testing, api-contract, previous-comments

Testing gaps:

  • No coverage for an empty single-select exact matcher against series that lack the matched label (the e2e test only exercises series with populated labels).
  • No test for a single-select value used inside a =~ regex matcher being left unescaped.
  • getPendingFilterValuesVariables has no case for an empty single-select promql reference rendering ''.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

kodiakhq Bot pushed a commit that referenced this pull request Oct 7, 2026
## Summary

Support configuring dashboard variables / filters as single-select, so that no more than one value can be selected.

If the URL contains multiple values for the filter, then the first is used.

### Screenshots or video

<img width="653" height="146" alt="Screenshot 2026-10-05 at 11 59 45 AM" src="https://github.com/user-attachments/assets/e0c214bb-b40e-47ef-8047-cdf7d70b4122" />
<img width="282" height="189" alt="Screenshot 2026-10-05 at 11 59 49 AM" src="https://github.com/user-attachments/assets/78b08baf-5602-4c03-b485-d294fe4dee36" />

### How to test on Vercel preview

- Create a dashboard
- Add some single select variables
- Observe the selection behavior

### References



- Linear Issue: Related to HDX-5563
- Related PRs: #3297
@pulpdrew
pulpdrew force-pushed the drew/promql-regex-formatting branch from 431e3eb to 2f35bb4 Compare October 7, 2026 16:50
@pulpdrew
pulpdrew changed the base branch from drew/single-select-vars to main October 7, 2026 16:50
@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 9
  • Production lines changed: 116 (+ 274 in test files, excluded from tier calculation)
  • Branch: drew/promql-regex-formatting
  • Author: pulpdrew

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 127 in total (1.77%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@greptile-apps

greptile-apps Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings could not be posted inline.

  • P2 Review finding ▶

    Confidence Score: 3/5

    The PR does not appear safe to merge while two previously reported PromQL matcher defects remain.

    Findings

    1.  Empty selection can match series ▶
    2.  Existing regex matchers change meaning ▶
    3.  Empty dropdown dependency goes unreported ▶

    Summary

    The PR adds a promql variable format that renders single-select values without regex escaping and multi-select values as escaped regexes. It also updates dashboard guidance, editor completions, and tests, and extends the MCP dashboard schemas to accept stacked-line and series-heatmap tiles.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 406 passed • 1 skipped • 1599s

Status Count
✅ Passed 406
❌ Failed 0
⚠️ Flaky 1
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@pulpdrew

pulpdrew commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Responded to the Greptile comments inline.

@karl-power karl-power left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One behaviour I found a bit unexpected (the P2 from Deep Review, not sure if a new feature or not): an empty single-select variable renders service="" which matches series where the label is missing, so the chart is usually empty but can possibly render an un-labelled series. An empty multi-select renders .*, which matches every series.

So with nothing selected, single select would show nothing (or empty label series) and multi select shows everything. Do you think this should be addressed in a follow up?

@kodiakhq
kodiakhq Bot merged commit 1be9de8 into main Oct 8, 2026
29 checks passed
@kodiakhq
kodiakhq Bot deleted the drew/promql-regex-formatting branch October 8, 2026 09:36
@pulpdrew

pulpdrew commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

LGTM. One behaviour I found a bit unexpected (the P2 from Deep Review, not sure if a new feature or not): an empty single-select variable renders service="" which matches series where the label is missing, so the chart is usually empty but can possibly render an un-labelled series. An empty multi-select renders .*, which matches every series.

So with nothing selected, single select would show nothing (or empty label series) and multi select shows everything. Do you think this should be addressed in a follow up?

It's a good question, the challenge is that there is no way to "match everything" using the = operator which the single-select promql format is intended for.

There are a few workarounds to restore match all behavior:

  1. Use a =~${var:regex} format to get .* for the variable empty state
  2. Make the filter required so that tiles that reference it don't load until the variable has been selected.

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 50d0434f Deployed Oct 8, 2026 by vercel[bot]
Preview – hyperdx-storybook — 50d0434f Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants