Skip to content

feat: Grafana global variables in raw SQL and link templates - #3685

Draft
knudtty wants to merge 1 commit into
mainfrom
aaron/support-more-global-variables
Draft

knudtty wants to merge 1 commit into
mainfrom
aaron/support-more-global-variables

Conversation

@knudtty

@knudtty knudtty commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Raw SQL charts and row-click link templates now expand:

Variable Expands to
${__from} / ${__to} epoch ms; :date / :date:iso give ISO 8601, :date:seconds gives epoch seconds
$__range / $__range_s / $__range_ms 3600s / 3600 / 3600000
${__timezone} the viewer's IANA zone (user setting, page tz override, or browser zone); UTC for alerts and MCP
${__url_time_range} from=<ms>&to=<ms>
${__all_variables} filters=<encoded variable selections>, the param ClickStack dashboards read

Each works as $__name or ${__name}.

How:

  • Definitions live in packages/common-utils/src/globalVariables.ts, used by replaceMacros and by the link builder. Link templates are expanded before Handlebars, so Grafana URLs paste in unchanged, and a #fragment is not treated as a SQL comment.
  • The template scanner now accepts ${__name:format} for any macro, so ${__fromTime} also works.
  • Time values are inlined like Grafana does, so '${__from:date:iso}' works inside quotes. Without a date range (saved-config validation) they fall back to the query params, so the time-filter checks still pass.
  • The app adds the display time zone to raw SQL configs that use ${__timezone} in the three query hooks. Other charts don't refetch when the zone changes.
  • A tile that uses ${__all_variables} (in SQL or an onClick template) receives every dashboard variable, not just the ones it references.
  • ${__timezone} must look like an IANA name, and ${__all_variables} percent-encodes ', so neither can break out of a SQL string.
  • Autocomplete, TIME_RANGE_MACROS, and the MCP prompts/SQL tool description are updated.

Behaviour changes:

  • SQL that literally contained $__from, $__to, $__range or $__timezone now gets expanded.
  • ${__filter} is now read as the $__filter macro and errors for missing arguments, instead of passing through.

Not supported: custom moment formats (${__from:date:YYYY-MM}) raise a clear error. PromQL and markdown tiles don't get the new variables.

How to reproduce

Starting point: yarn app:dev:local connected to the demo server.

On main:

  1. Open Chart Explorer, switch to SQL, display type Table.
  2. Run SELECT ${__from} AS f, $__range AS r, '${__timezone}' AS tz.
  3. The query fails: the variables reach ClickHouse unexpanded.

On this branch:

  1. Run the same query.
  2. It returns the start of the range in epoch ms, the range length (e.g. 900s), and your time zone.
  3. On a dashboard Table tile, add a row click "External link" with URL https://example.com/?${__url_time_range}&${__all_variables}. Hover a row: the link shows from=…&to=…&filters=….

How to test on Vercel preview

Preview routes: /chart

Steps:

  1. Switch the query mode to SQL and choose the Table display type.
  2. Enter SELECT ${__from} AS f, $__range AS r, '${__timezone}' AS tz and run the query.
  3. Verify the table shows one row with a numeric f, an r ending in s, and an IANA zone in tz.

References

  • Linear Issue:
  • Related PRs:

Raw SQL charts and row-click link templates now expand ${__from}/${__to}
(with :date, 📅iso, 📅seconds), $__range/_s/_ms, ${__timezone},
${__url_time_range} and ${__all_variables}, so SQL and links migrated from
Grafana work unchanged.
@knudtty
knudtty requested a review from a team as a code owner November 16, 2026 18:58
@vercel

vercel Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 16, 2026 7:03pm UTC
hyperdx-storybook Ready Ready Preview Nov 16, 2026 7:03pm UTC

Request Review

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 401 production lines changed (Tier 2 max: < 250)
  • Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 12
  • Production lines changed: 401 (+ 127 in test files, excluded from tier calculation)
  • Branch: aaron/support-more-global-variables
  • Author: knudtty

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@changeset-bot

changeset-bot Bot commented Nov 16, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1ecb71d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Minor
@hyperdx/app Minor
@hyperdx/api Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

1 new clone introduced by this PR.

  • packages/common-utils/src/core/linkUrlBuilder.ts:162-181 ↔ packages/common-utils/src/core/linkUrlBuilder.ts:258-277 — 20 lines

If one of these duplicates a helper that already exists, import it instead
of redefining it. If the duplication is deliberate, ignore this comment —
the check is advisory and never fails the build.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.18%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

variables: ChartVariable[],
): ChartVariable[] {
// `${__all_variables}` reads every variable, from the SQL or a link template.
if (/\$\{?__all_variables\b/.test(JSON.stringify(config))) return variables;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — A link that uses ${__all_variables} makes the tile's query re-run whenever any variable changes

filterReferencedVariables decides which variables go into the tile config, and that config is the query key in useQueriedChartConfig and useOffsetPaginatedQuery (DBDashboardPage.tsx:619-630). So when only an onClick template reads ${__all_variables}, the tile still gets every variable and re-queries ClickHouse on each unrelated selection change, even though its SQL is unchanged. The JSON.stringify(config) regex also matches the token inside SQL comments, builder where strings, aliases and markdown. Fix: only short-circuit here when the SQL itself uses the token, found with the scanner rather than a regex over the whole config. Pass the full dashboard variable list to useOnClickLinkBuilder separately (from DBDashboardPage through DBTableChart), not via config.variables.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

if (input.charAt(i + 1) === '{') {
const closeIndex = input.indexOf('}', i + 2); // no escaping because variable names can't contain `}` or `\`

// Grafana's braced macro form, `${__from}` / `${__from:date:iso}`, whose format is the macro's argument

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The new braced form for every macro has no tests outside the global variables

This branch now turns ${__<any macro>} / ${__<macro>:fmt} into a macro token for every caller: SQL, PromQL and the variable-macro scan. It also changes ${__filter} from a pass-through into an error, which the PR lists as a behaviour change. The added tests only cover the new global names. Add scanTemplateTokens/replaceMacros cases for ${__fromTime}, ${__interval_s}, ${__filter} (now an argument error), and a PromQL ${__range}. Also add a case where the text up to } merely starts with a macro name, e.g. ${__from_x}, and should stay verbatim.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@github-actions

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor

2 posted as inline comment(s) on the changed lines. 1 listed below.

Findings outside the changed lines

1 minor
  • 🔵 packages/app/src/useDisplayTimeZone.ts:249 — useWithDisplayTimeZone is the only path that puts the zone into a query, and it has no test → ${__timezone} reaches a query only through this hook in three query hooks. Nothing checks that a raw SQL config using the macro gets timeZone, that other configs come back as the same object (so they don't refetch), or that a malformed macro falls back safely. Add a renderHook test covering: a raw SQL config with ${__timezone}, one without it (expect the same reference), a builder config, and a template like $__timezone( that hits the catch.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

Copy link
Copy Markdown
Contributor

Deep Review

Scope: 16 files vs base 155f493 (743c3c1 merge-base) — new globalVariables.ts, scanner change in variables.ts, macros.ts, linkUrlBuilder.ts, three app query hooks, MCP docs, tests.
Intent: Expand Grafana global variables (${__from}, ${__to}, $__range, ${__timezone}, ${__url_time_range}, ${__all_variables}) in raw SQL and row-click link templates, plus a braced ${__name} form for any macro.

✅ No critical issues found. The injection surfaces are handled well: ${__timezone} is validated against an IANA pattern that excludes quotes/backslashes and fails closed via MacroExpansionError; ${__all_variables} runs encodeURIComponent plus an explicit '→%27 replacement, so neither can terminate a SQL string literal. Time values are inlined as numbers or toISOString() output. URL expansion runs before Handlebars with URL-safe values, and isAllowedExternalUrl still gates the result. No P0/P1 defects were confirmed.

🟡 P2 — recommended

  • packages/app/src/useDisplayTimeZone.ts:33 — the new useWithDisplayTimeZone hook decides whether a config refetches on zone change (isRawSqlChartConfig + hasMacro(..., 'timezone') + swallowed try/catch), but no test exercises this branching or the error path.
    • Fix: Add unit coverage asserting raw-SQL-with-${__timezone} gets timeZone injected, other configs pass through unchanged, and a malformed template returns the config without throwing.
    • testing, maintainability
  • packages/common-utils/src/core/linkUrlBuilder.ts:97 — renderOnClickSearch and renderOnClickDashboard now thread timeZone/variables into global-variable expansion, but only renderOnClickExternal has a test covering the new globals.
    • Fix: Add link-builder tests asserting ${__timezone}/${__all_variables}/${__from} expand inside search and dashboard whereTemplate/target.template/filters templates.
    • testing, api-contract
🔵 P3 nitpicks (5)
  • packages/common-utils/src/macros.ts:388 — widening the findMacros pattern to \$\{?__name\b means raw SQL that previously contained literal $__from/$__to/$__range/$__timezone text now gets expanded, silently changing any pre-existing saved query that used those as non-macro strings.
    • Fix: Note the migration risk in the changeset and add a regression test pinning expansion of a literal-looking occurrence so the change is intentional and visible.
  • packages/common-utils/src/variables.ts:356 — a template literally containing ${__filter} (no args) is now parsed as the $__filter macro and throws instead of passing through, turning a previously-rendering tile/link into a render error when a variable context is present.
    • Fix: Add a test documenting the new error behavior for the argument-less braced ${__filter} form.
  • packages/common-utils/src/globalVariables.ts:149 — GLOBAL_VARIABLES_BY_NAME.get(name)! relies on a non-null assertion; a future caller passing an unregistered name would hit an undefined-method crash rather than a typed error.
    • Fix: Guard the lookup and throw a MacroExpansionError when the name is absent.
  • packages/common-utils/src/macros.ts:254 — GLOBAL_VARIABLES is mapped into both MACRO_SUGGESTIONS and the replaceMacros macro list with the same {name,minArgs,maxArgs,description} projection, duplicating the registration shape.
    • Fix: Extract a shared helper that produces the suggestion/macro projection from GLOBAL_VARIABLES once.
  • packages/app/src/hooks/useChartConfig.tsx:361 — the three query hooks now unconditionally call useWithDisplayTimeZone, coupling every raw-SQL query path to the time-zone-preference providers; a consumer rendering these hooks outside that provider tree would break.
    • Fix: Confirm all call sites sit under the user-preferences/time-zone providers, or document the dependency on the hooks.

Reviewers (9): correctness, security, maintainability, testing, kieran-typescript, performance, api-contract, adversarial, project-standards.

Testing gaps: useWithDisplayTimeZone branching/error path untested; renderOnClickSearch/renderOnClickDashboard global-variable expansion untested; braced form for non-global macros (${__fromTime}, ${__sourceTable}) and prefix-collision cases (${__range} vs ${__range_ms}) not explicitly asserted.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 466 passed • 1 skipped • 1199s

Status Count
✅ Passed 466
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@knudtty
knudtty marked this pull request as draft November 16, 2026 19:51

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 1ecb71d6 Deployed Nov 16, 2026 by vercel[bot]
Preview – hyperdx-storybook — 1ecb71d6 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant