Skip to content

feat: Add schemas and types for adhoc dashboard filters - #3335

Merged
pulpdrew merged 1 commit into
mainfrom
drew/adhoc-filters-schemas
Oct 13, 2026
Merged

pulpdrew merged 1 commit into
mainfrom
drew/adhoc-filters-schemas

Conversation

@pulpdrew

@pulpdrew pulpdrew commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds the schemas and types for adhoc filters.

Known limitations (will be addressed in subsequent PRs):

  • No MCP or External API Support
  • No import support
  • No UI for creating or using adhoc filters
  • etc... this is just the schemas, types, and internal dashboard API updates

Screenshots or video

No behavior change expected other than the internal API now accepts adhoc filters, as demonstrated in the integration tests.

How to test on Vercel preview

References

  • Linear Issue: Related to HDX-5658
  • Related PRs:

@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Oct 13, 2026 11:27am UTC
hyperdx-storybook Ready Ready Preview Oct 13, 2026 11:27am UTC

Request Review

@changeset-bot

changeset-bot Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 901ead5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

  • Critical-path files (1) — tenancy, public API, or shipped database config:
    • packages/api/src/routers/external-api/v2/utils/dashboards.ts
  • Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 11
  • Production lines changed: 355 (+ 473 in test files, excluded from tier calculation)
  • Critical-path lines changed: 12
  • Branch: drew/adhoc-filters-schemas
  • Author: pulpdrew

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 125 in total (1.75%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

Comment thread packages/common-utils/src/types.ts Resolved
Comment thread packages/app/src/DBDashboardPage.tsx Resolved
Comment thread packages/common-utils/src/types.ts Resolved
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

3 finding(s): 🔴 1 critical · 🟠 0 major · 🔵 2 minor

3 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge within its stated schemas-and-internal-API scope.

Summary

Adds ad hoc filter schemas, saved conditions, and internal dashboard API source checks.

  • Dashboards can describe ad hoc filters and save their conditions.
  • Internal dashboard saves check sources used by ad hoc filters.
  • The external dashboard API leaves ad hoc filters out.
  • The dashboard page and importer skip ad hoc filters.

Reviews (5) · Last reviewed commit: "feat: Add schemas and types for adhoc da..." · Reviewed by Greptile

Comment thread packages/common-utils/src/types.ts Resolved
Comment thread packages/common-utils/src/core/utils.ts Resolved
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. This is a schema/types-only change with strong integration-test coverage; the internal API gains ADHOC filters while they are deliberately stripped from the external API, MCP, dashboard UI, and import. Source-kind/tenancy validation (validateFilterSources) was verified to have no bypass, and the free-form key/value strings are only persisted — nothing renders them into SQL/PromQL in this diff. The prior-review P1 (variableName must be required) is a false positive: no test asserts a missing variableName returns 400, and deriving the name from the display label is the intended fallback. The items below are recommended follow-ups, none blocking.

🟡 P2 -- recommended

  • packages/common-utils/src/core/utils.ts:831 -- convertToDashboardTemplate maps an ad hoc filter's sources through getSourceName without dropping unresolved names, so a since-deleted source becomes '', which violates AdhocDashboardFilterSchema (sources: z.string().min(1)) and makes the whole exported template fail re-import.
    • Fix: Filter out empty names as the adjacent appliesToSourceIds branch already does, and omit the ad hoc filter entirely when no sources remain.
    • previous-comments, testing
  • packages/common-utils/src/core/utils.ts:831 -- the new export path for a deleted source inside the ad hoc sources array is untested; the existing test only exercises a deleted id in appliesToSourceIds.
    • Fix: Add a convertToDashboardTemplate case with a deleted id in sources asserting the exported template still passes DashboardTemplateSchema.parse.
    • testing
  • packages/app/src/DBDashboardImportPage.tsx:970 -- the new import-time stripping of ad hoc filters and ad hoc saved values is entirely untested; the existing import test file has no ad hoc coverage.
    • Fix: Add a test importing a template with an ADHOC filter plus an adhoc saved value and assert both are dropped while source-backed filters still produce a mapping row.
    • testing
  • packages/app/src/DBDashboardPage.tsx:1984 -- the "internal-only filter type" exclusion is reimplemented as a bare !isAdhocFilter / !isAdhocFilterValue runtime filter at five sites (render, import filters, import values, and both external-API strip helpers), and only three sites carry a satisfies never compile-time backstop, so the next internal-only filter type will silently leak through render, import, and the external GET.
    • Fix: Centralize the policy in common-utils as a single predicate driven by an explicit union and share it across the API and app paths so adding a type forces one edit.
    • maintainability, api-contract
  • packages/api/src/utils/externalApi.ts:247 -- the external/MCP GET silently strips ad hoc filters and values while the PUT replaces filters/savedFilterValues wholesale, so once an ad hoc filter exists on a dashboard (creatable through the internal API today), an external GET→PUT round-trip silently deletes it.
    • Fix: When the follow-up external support lands, either expose ADHOC end-to-end or merge stored ad hoc filters/values back by id on update so round-trips are non-destructive.
    • agent-native
🔵 P3 nitpicks (4)
  • packages/common-utils/src/types.ts:1258 -- AdhocFilterCondition.key/value are stored as free-form SQL/PromQL strings; nothing interpolates them yet, but a future renderer must parameterize/escape them through the same path as QUERY_EXPRESSION expressions rather than string concatenation.
    • Fix: Route ad hoc key/value through the existing escaping/substitution path when the query renderer is added in a later PR.
  • packages/common-utils/src/types.ts:1246 -- DashboardFilterValueSchema is a non-discriminated z.union; it is safe today only because FilterSchema/VariableFilterValueSchema/AdhocFilterValueSchema have disjoint type literals.
    • Fix: Migrate to z.discriminatedUnion('type', ...) so the disjoint-literal invariant is enforced rather than incidental.
  • packages/app/src/DBDashboardImportPage.tsx:399 -- isSourceBackedTemplateFilter (boolean guard) and sourceBackedTemplateFilter (returns filter-or-undefined) differ only by the is prefix yet have different return contracts, inviting a wrong pick at call sites.
    • Fix: Rename the extractor to a value-accessor name such as getTemplateFilterSourceName that returns the source name directly.
  • packages/common-utils/src/types.ts:2256 -- AdhocDashboardFilterSchema is .strict() while its discriminated-union siblings are not, so an unexpected key returns 400 for ADHOC but is silently stripped for other filter types; intentional per the schema comment but an undocumented divergence.
    • Fix: Document the ADHOC strictness exception in the API contract, or align strictness across the union members.

Reviewers (8): correctness/adversarial (orchestrator-verified), security, api-contract, testing, maintainability, project-standards, agent-native, previous-comments, learnings.

Testing gaps:

  • Export of an ad hoc filter whose sources entry references a deleted source is untested and currently produces an unimportable template.
  • Import-page stripping of ad hoc filters and ad hoc saved values has no unit test.
  • No structural/regression test asserts that every internal-only filter type stays excluded from render, import, and the external GET.
  • Minor: external-API GET strip test lacks a mixed (ad hoc + non-ad hoc) case, and no test locks the PATCH source-validation skip against reusing a filter.id to smuggle a cross-team source (code path verified safe).

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 406 passed • 1 skipped • 1141s

Status Count
✅ Passed 406
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Comment thread packages/app/src/DBDashboardPage.tsx Resolved
Comment thread packages/app/src/DBDashboardPage.tsx Resolved

@brandon-pereira brandon-pereira left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pulpdrew
pulpdrew force-pushed the drew/adhoc-filters-schemas branch from 7d7c818 to 85dda5a Compare October 13, 2026 11:04
@@ -682,10 +684,14 @@ export function convertToExternalDashboard(
.map(tile => convertTileToExternalChart(tile, containerById, dashboardId))
.filter(t => t !== undefined),
tags: dashboard.tags || [],
filters: dashboard.filters?.map(translateFilterToExternalFilter) || [],
filters: getExternalCompatibleFilters(dashboard.filters).map(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 critical — External API PUT and MCP saveDashboard silently delete the ADHOC filters and saved values that GET hides

convertToExternalDashboard now leaves ADHOC filters and adhoc saved values out of GET. The update paths still overwrite the whole array, though: routers/external-api/v2/dashboards.ts:3201-3219 sets setPayload.filters = convertExternalFiltersToInternal(filters, …) and setPayload.savedFilterValues = savedFilterValues, and mcp/tools/dashboards/saveDashboard.ts:339,354 does the same. So if a client GETs a dashboard and PUTs it back unchanged (the documented round trip), every ADHOC filter and its saved conditions are wiped. Fix: on update, append the stored ADHOC filters (existingDashboard.filters.filter(isAdhocFilter)) and the stored adhoc values to what the client sent before writing. Add savedFilterValues to the projection at line 3161. Also add an integration test that a GET→PUT round trip keeps them.

Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond

@@ -1978,7 +1979,11 @@ function DBDashboardPage({

const [showFiltersModal, setShowFiltersModal] = useState(false);

const filters = dashboard?.filters ?? [];
// Ad hoc filters aren't rendered or applied yet
const filters = useMemo(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Hiding ADHOC filters from the filters modal skips the variable-name uniqueness check, so the server rejects the save

The filtered filters list goes to DashboardFiltersModal (line 3527). VariableNameInput then checks validateVariableName only against those filters. A user can pick a variable name that a hidden ADHOC filter already uses. The client accepts it, handleSaveFilter writes into the full dashboard.filters, and the PATCH fails with 400 from validateDashboardFilterVariableNames. The user gets no field-level error. Fix: pass the unfiltered dashboard.filters to the modal for the uniqueness check, and still hide ADHOC filters from the list.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

sources.find(source => source.id === id)?.name ?? '';

if (isAdhocFilter(filter)) {
filter.sources = filter.sources.map(getSourceName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Exporting an ADHOC filter with a deleted source produces a template that fails DashboardTemplateSchema

filter.sources.map(getSourceName) turns an unresolved ID into '', but AdhocDashboardFilterSchema.sources is z.array(z.string().min(1)). The exported template fails validation, so the whole dashboard can't be imported, even though import drops ADHOC filters anyway (DBDashboardImportPage.tsx:972). appliesToSourceIds already filters out empty names on the next lines. Fix: filter out empty names from sources the same way and drop the filter if none remain, or leave ADHOC filters out of the export until import supports them. Add a test case for a deleted entry in sources.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@pulpdrew
pulpdrew force-pushed the drew/adhoc-filters-schemas branch from 85dda5a to 901ead5 Compare October 13, 2026 11:23
@pulpdrew
pulpdrew merged commit 9891a79 into main Oct 13, 2026
29 checks passed

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 901ead54 Deployed Oct 13, 2026 by vercel[bot]
Preview – hyperdx-storybook — 901ead54 Deployed Oct 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants