Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/grafana-global-variables.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'@hyperdx/common-utils': minor
'@hyperdx/app': minor
'@hyperdx/api': patch
---

feat: Grafana global variables in raw SQL and link templates

Raw SQL charts and row-click link templates now expand `${__from}` / `${__to}`
(epoch ms, or `:date`, `:date:iso`, `:date:seconds`), `$__range`, `$__range_s`,
`$__range_ms`, `${__timezone}`, `${__url_time_range}` and `${__all_variables}`.
Each also works in the bare `$__name` form, and any macro can now be written in
braces, e.g. `${__fromTime}`.
10 changes: 9 additions & 1 deletion packages/api/src/mcp/prompts/dashboards/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -775,6 +775,10 @@ SQL TEMPLATE REFERENCE:
$__fromTime / $__toTime start/end as DateTime values
$__fromTime_ms / $__toTime_ms start/end as DateTime64 values
$__interval_s raw interval in seconds
\${__from} / \${__to} Grafana globals: start/end epoch ms; :date:iso or :date:seconds formats
$__range / $__range_s / $__range_ms range length as "3600s" / seconds / ms
\${__timezone} viewer's IANA time zone (UTC for alerts); quote it
\${__url_time_range} from=<ms>&to=<ms>; \${__all_variables} filters=<variable selections>
$__filters dashboard filter conditions (resolves to 1=1 when none).
REQUIRES sourceId on the tile. Without it, filters never apply.
$__sourceTable the source's \`database\`.\`table\`.
Expand Down Expand Up @@ -967,6 +971,10 @@ For configType: "sql" tiles, write ClickHouse SQL with template macros:
$__fromTime / $__toTime start/end as DateTime values
$__fromTime_ms / $__toTime_ms start/end as DateTime64 values
$__interval_s raw interval in seconds (for arithmetic)
\${__from} / \${__to} Grafana globals: start/end epoch ms; :date:iso or :date:seconds formats
$__range / $__range_s / $__range_ms range length as "3600s" / seconds / ms
\${__timezone} viewer's IANA time zone (UTC for alerts); quote it
\${__url_time_range} from=<ms>&to=<ms>; \${__all_variables} filters=<variable selections>
$__filters dashboard filter conditions (1=1 when none).
REQUIRES sourceId on the tile. Set sourceId (unless the
query reads multiple tables) so dashboard filters apply.
Expand Down Expand Up @@ -1325,7 +1333,7 @@ Destination types:
{ type: "dashboard", target, whereLanguage, whereTemplate?, filters? }
Opens another ClickStack dashboard owned by the same team.
{ type: "external", urlTemplate }
Opens an arbitrary external URL in a new tab (e.g. a Grafana or Langfuse dashboard, a runbook). The urlTemplate is rendered against the clicked row and the rendered value must be an absolute http(s) URL. It takes no target, whereLanguage, whereTemplate, or filters, and references no ClickStack source or dashboard.
Opens an arbitrary external URL in a new tab (e.g. a Grafana or Langfuse dashboard, a runbook). The urlTemplate is rendered against the clicked row and the rendered value must be an absolute http(s) URL. Every link template also expands the Grafana globals \${__from}, \${__to}, \${__from:date:iso}, $__range, \${__timezone}, \${__url_time_range} (from=<ms>&to=<ms>) and \${__all_variables} (filters=<variable selections>). It takes no target, whereLanguage, whereTemplate, or filters, and references no ClickStack source or dashboard.

Target shape, how the destination is identified:
target: { mode: "id", id: "<object-id>" }
Expand Down
3 changes: 3 additions & 0 deletions packages/api/src/mcp/tools/query/sql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ const sqlSchema = z.object({
' $__timeInterval(column) — time bucket expression: toStartOfInterval(toDateTime(column), INTERVAL ...)\n' +
' $__timeInterval_ms(column) — same with millisecond precision\n' +
' $__interval_s — raw interval in seconds\n' +
' ${__from} / ${__to} — start/end epoch ms; ${__from:date:iso} / ${__from:date:seconds} for other formats\n' +
' $__range / $__range_s / $__range_ms — range length as 3600s / seconds / milliseconds\n' +
" ${__timezone} — the viewer's IANA time zone (UTC here); quote it\n" +
' $__filters — dashboard filter conditions (resolves to 1=1 when no filters)\n' +
" $__sourceTable — the source's database.table. Requires sourceId.\n" +
' {intervalSeconds:Int64} / {intervalMilliseconds:Int64} — bucket size parameters\n\n' +
Expand Down
1 change: 1 addition & 0 deletions packages/app/src/components/DBTableChart.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,7 @@ export default function DBTableChart({
const getRowAction = useOnClickLinkBuilder({
onClick: config.onClick,
dateRange: queriedConfig.dateRange,
variables: config.variables,
});

return (
Expand Down
7 changes: 5 additions & 2 deletions packages/app/src/hooks/useChartConfig.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import { useMetadataWithSettings } from '@/hooks/useMetadata';
import { useQueryAttribution } from '@/queryAttribution';
import { useSource } from '@/source';
import { ChartQueryResult } from '@/types';
import { useWithDisplayTimeZone } from '@/useDisplayTimeZone';
import { stripClientSideConfigFields } from '@/utils/chartConfig';
import {
queryPromqlChartConfig,
Expand Down Expand Up @@ -358,10 +359,11 @@ export function appendChunk(
* - `data.isComplete` indicates whether all chunks have been fetched.
*/
export function useQueriedChartConfig(
config: ChartConfigWithOptDateRange,
configProp: ChartConfigWithOptDateRange,
options?: Partial<UseQueryOptions<TQueryFnData>> &
AdditionalUseQueriedChartConfigOptions,
) {
const config = useWithDisplayTimeZone(configProp);
const { enabled = true } = options ?? {};
const clickhouseClient = useClickhouseClient();
const attribution = useQueryAttribution();
Expand Down Expand Up @@ -501,9 +503,10 @@ export function useQueriedChartConfig(
}

export function useRenderedSqlChartConfig(
config: ChartConfigWithOptDateRange,
configProp: ChartConfigWithOptDateRange,
options?: Partial<UseQueryOptions<string>>,
) {
const config = useWithDisplayTimeZone(configProp);
const { enabled = true } = options ?? {};

const metadata = useMetadataWithSettings();
Expand Down
4 changes: 3 additions & 1 deletion packages/app/src/hooks/useOffsetPaginatedQuery.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import { useMetadataWithSettings } from '@/hooks/useMetadata';
import { useMVOptimizationExplanation } from '@/hooks/useMVOptimizationExplanation';
import { useQueryAttribution } from '@/queryAttribution';
import { useSource } from '@/source';
import { useWithDisplayTimeZone } from '@/useDisplayTimeZone';
import { queryPromqlChartConfig } from '@/utils/promqlChartQuery';
import {
DEFAULT_TIME_WINDOWS_SECONDS,
Expand Down Expand Up @@ -478,7 +479,7 @@ function flattenData(data: TData | undefined): TQueryFnData | null {
}

export default function useOffsetPaginatedQuery(
config: ChartConfigWithOptTimestamp,
configProp: ChartConfigWithOptTimestamp,
{
isLive,
keepPreviousData,
Expand All @@ -494,6 +495,7 @@ export default function useOffsetPaginatedQuery(
enableSmallFirstWindow?: boolean;
} = {},
) {
const config = useWithDisplayTimeZone(configProp);
const { data: meData, isLoading: isLoadingMe } = api.useMe();
const key = queryKeyFn(queryKeyPrefix, config, meData?.team?.queryTimeout);
const queryClient = useQueryClient();
Expand Down
26 changes: 24 additions & 2 deletions packages/app/src/hooks/useOnClickLinkBuilder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,18 @@ import {
renderOnClickExternal,
renderOnClickSearch,
} from '@hyperdx/common-utils/dist/core/linkUrlBuilder';
import { isSearchableSource, OnClick } from '@hyperdx/common-utils/dist/types';
import {
ChartVariable,
isSearchableSource,
OnClick,
} from '@hyperdx/common-utils/dist/types';
import { notifications } from '@mantine/notifications';

import { useDashboards } from '@/dashboard';
import { useSources } from '@/source';
import { getLocalTimeZone } from '@/timeZone';
import { useTimeZoneLinkParam } from '@/timeZoneOverride';
import { useDisplayTimeZone } from '@/useDisplayTimeZone';

/**
* The action a row click should take. `url` is set when the row's
Expand Down Expand Up @@ -49,13 +55,17 @@ export type RowAction = {
export function useOnClickLinkBuilder({
onClick,
dateRange,
variables,
}: {
onClick: OnClick | undefined;
dateRange: [Date, Date];
/** The dashboard's variables, read by `${__all_variables}`. */
variables?: ChartVariable[];
}): ((row: Record<string, unknown>) => RowAction) | null {
const { data: sources } = useSources();
const { data: dashboards } = useDashboards();
const addTimeZone = useTimeZoneLinkParam();
const timeZone = useDisplayTimeZone() ?? getLocalTimeZone();

const [sourceIdsByName, sourceIds, sourceNamesById] = useMemo(() => {
const idsByName = new Map<string, string[]>();
Expand Down Expand Up @@ -113,6 +123,8 @@ export function useOnClickLinkBuilder({
sourceIds,
sourceIdsByName,
dateRange,
timeZone,
variables,
})
: onClick.type === 'dashboard'
? renderOnClickDashboard({
Expand All @@ -121,8 +133,16 @@ export function useOnClickLinkBuilder({
dashboardIds,
dashboardIdsByName,
dateRange,
timeZone,
variables,
})
: renderOnClickExternal({ onClick, row });
: renderOnClickExternal({
onClick,
row,
dateRange,
timeZone,
variables,
});

if (renderResult.ok) {
return {
Expand Down Expand Up @@ -163,6 +183,8 @@ export function useOnClickLinkBuilder({
sourceIdsByName,
sourceNamesById,
dateRange,
timeZone,
variables,
dashboardIds,
dashboardIdsByName,
dashboardNamesById,
Expand Down
26 changes: 25 additions & 1 deletion packages/app/src/useDisplayTimeZone.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import { useMemo } from 'react';
import { isRawSqlChartConfig } from '@hyperdx/common-utils/dist/guards';
import { hasMacro } from '@hyperdx/common-utils/dist/macros';
import { ChartConfigWithOptDateRange } from '@hyperdx/common-utils/dist/types';

import { resolveTimeZonePreference } from './timeZone';
import { getLocalTimeZone, resolveTimeZonePreference } from './timeZone';
import { useTimeZoneOverride } from './timeZoneOverride';
import { useUserPreferences } from './useUserPreferences';

Expand All @@ -20,3 +23,24 @@ export function useDisplayTimeZone(): string | undefined {
[override, timeZone],
);
}

/**
* Adds the display time zone to a raw SQL config that reads `${__timezone}`.
* Other configs are returned as is, so a zone change doesn't refetch them.
*/
export function useWithDisplayTimeZone<T extends ChartConfigWithOptDateRange>(
config: T,
): T {
const timeZone = useDisplayTimeZone() ?? getLocalTimeZone();
return useMemo(() => {
try {
return isRawSqlChartConfig(config) &&
hasMacro(config.sqlTemplate, 'timezone')
? { ...config, timeZone }
: config;
} catch {
// A half-typed macro; rendering reports it.
return config;
}
}, [config, timeZone]);
}
69 changes: 69 additions & 0 deletions packages/common-utils/src/__tests__/macros.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,75 @@ describe('replaceMacros', () => {
});
});

describe('replaceMacros global variables', () => {
const dateRange: [Date, Date] = [new Date(1500), new Date(3601500)];

it.each([
['$__from', '1500'],
['${__from}', '1500'],
['${__to}', '3601500'],
["'${__from:date}'", "'1970-01-01T00:00:01.500Z'"],
["'${__to:date:iso}'", "'1970-01-01T01:00:01.500Z'"],
['${__from:date:seconds}', '1'],
['$__range', '3600s'],
['${__range_s}', '3600'],
['$__range_ms', '3600000'],
["'${__timezone}'", "'UTC'"],
["'${__url_time_range}'", "'from=1500&to=3601500'"],
])('expands %s', (sqlTemplate, expected) => {
expect(replaceMacros({ sqlTemplate, dateRange })).toBe(expected);
});

it('leaves the longer macros sharing a prefix alone', () => {
expect(replaceMacros({ sqlTemplate: '$__fromTime_ms', dateRange })).toBe(
'fromUnixTimestamp64Milli({startDateMilliseconds:Int64})',
);
});

it('uses the query params without a date range, so validation sees a time filter', () => {
expect(replaceMacros({ sqlTemplate: '${__from:date:iso} ${__to}' })).toBe(
'{startDateMilliseconds:Int64} {endDateMilliseconds:Int64}',
);
});

it('expands the configured time zone and rejects one that is not a zone name', () => {
expect(
replaceMacros({
sqlTemplate: "'${__timezone}'",
timeZone: 'America/New_York',
}),
).toBe("'America/New_York'");
expect(() =>
replaceMacros({ sqlTemplate: '$__timezone', timeZone: "x' OR 1=1" }),
).toThrow("Invalid time zone 'x' OR 1=1'");
});

it('encodes selected variables as a URL param that cannot end the SQL string', () => {
expect(
replaceMacros({
sqlTemplate: "'${__all_variables}'",
variables: [variable('svc', ["a'b"]), EMPTY_SERVICE],
}),
).toBe(
"'filters=%5B%7B%22type%22%3A%22variable%22%2C%22name%22%3A%22svc%22%2C%22values%22%3A%5B%22a%27b%22%5D%7D%5D'",
);
});

it('rejects unsupported formats', () => {
expect(() =>
replaceMacros({ sqlTemplate: '${__from:date:YYYY}', dateRange }),
).toThrow("Unsupported format 'date:YYYY'");
expect(() =>
replaceMacros({ sqlTemplate: '${__range:raw}', dateRange }),
).toThrow('takes no format');
});

it('is detected by hasMacro in either form', () => {
expect(hasMacro('WHERE ts >= ${__from:date:seconds}', 'from')).toBe(true);
expect(hasMacro('WHERE ts >= $__fromTime', 'from')).toBe(false);
});
});

describe('replaceMacros with variables', () => {
const variables = [
{ name: 'service', expression: 'ServiceName', values: ['api', 'web'] },
Expand Down
18 changes: 18 additions & 0 deletions packages/common-utils/src/__tests__/variables.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1470,6 +1470,24 @@ describe('filterReferencedVariables', () => {
);
});

it('keeps every variable when the SQL or a link reads ${__all_variables}', () => {
expect(
filterReferencedVariables(rawSqlConfig("'$__all_variables'"), variables),
).toEqual(variables);
expect(
filterReferencedVariables(
{
...rawSqlConfig('SELECT 1'),
onClick: {
type: 'external',
urlTemplate: 'https://example.com/?${__all_variables}',
},
},
variables,
),
).toEqual(variables);
});

it('keeps the variables a PromQL expression references', () => {
expect(
filterReferencedVariables(
Expand Down
40 changes: 40 additions & 0 deletions packages/common-utils/src/core/__tests__/linkUrlBuilder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -903,6 +903,46 @@ describe('renderOnClickExternal', () => {
}
});

it('expands Grafana global variables alongside row columns', () => {
const onClick: OnClickExternal = {
type: 'external',
urlTemplate:
'https://grafana.example.com/d/abc?svc={{ServiceName}}&${__url_time_range}&tz=${__timezone}&start=${__from:date:iso}&${__all_variables}#$__to',
};
const result = renderOnClickExternal({
onClick,
row: { ServiceName: 'checkout' },
dateRange: [new Date(1000), new Date(61000)],
timeZone: 'Etc/GMT+5',
variables: [
{ name: 'env', values: ["it's"] },
{ name: 'empty', values: [] },
],
});
expect(result).toEqual({
ok: true,
url:
'https://grafana.example.com/d/abc?svc=checkout&from=1000&to=61000&tz=Etc%2FGMT%2B5' +
'&start=1970-01-01T00:00:01.000Z' +
'&filters=%5B%7B%22type%22%3A%22variable%22%2C%22name%22%3A%22env%22%2C%22values%22%3A%5B%22it%27s%22%5D%7D%5D' +
'#61000',
});
});

it('reports an unsupported global variable format', () => {
const result = renderOnClickExternal({
onClick: {
type: 'external',
urlTemplate: 'https://example.com/?t=${__from:date:YYYY}',
},
row: {},
dateRange: [new Date(0), new Date(1)],
});
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toContain("Unsupported format 'date:YYYY'");
});

it('allows http URLs', () => {
const onClick: OnClickExternal = {
type: 'external',
Expand Down
Loading
Loading