Skip to content

test(common-utils): cover mixed query operator generation - #3131

Open
Tyagiquamar wants to merge 2 commits into
ClickHouse:mainfrom
Tyagiquamar:fix/query-parser-mixed-operators-3032
Open

Tyagiquamar wants to merge 2 commits into
ClickHouse:mainfrom
Tyagiquamar:fix/query-parser-mixed-operators-3032

Conversation

@Tyagiquamar

@Tyagiquamar Tyagiquamar commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Adds baseline coverage for query generation before changing mixed-operator behavior, as requested in #3032. The PR now changes only tests, so existing saved searches and dashboards keep their current query generation.

Adds 16 cases to the existing SQL and English explanation table: mixed AND/OR chains, explicit grouping on either side, symbolic operators, implicit AND, infix NOT/AND NOT/OR NOT, leading NOT, and negated fields. Removes the previous grouping implementation and duplicate test harness. The behavior fix remains for a separate decision after this baseline is established.

Validated in Docker with Node 22.23.1 and Yarn 4.13.0:

  • Prettier check and changed-file ESLint passed.
  • yarn workspace @hyperdx/common-utils tsc --noEmit passed.
  • yarn workspace @hyperdx/common-utils build passed, including declaration generation.
  • yarn workspace @hyperdx/common-utils jest src/__tests__/queryParser.test.ts --ci --coverage=false --runInBand --silent --verbose=false passed all 421 tests.
  • yarn lint-staged passed for both staged files.

Related to #3032. This tests-only PR does not fix the underlying issue or require a release changeset.

@vercel

vercel Bot commented Sep 15, 2026

Copy link
Copy Markdown

@Tyagiquamar is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

@changeset-bot

changeset-bot Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b19515c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@greptile-apps

greptile-apps Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Low risk] Adds test cases for query parser logic.

The PR is not safe to merge because mixed AND/OR searches can return rows outside the Lucene expression’s grouping.

Summary

The latest revision removes the mixed-operator grouping logic and replaces the earlier grouping assertions with tests expecting flat SQL.

  • This restores the search-result error the PR set out to fix.
  • The previous missing-changeset finding no longer applies because the current PR has no production-code change relative to the base.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["Lucene: a AND (b OR c)"] --> B["Serialized: a AND b OR c"]
  B --> C["ClickHouse: (a AND b) OR c"]
  C --> D["A row matching only c is included"]
Loading

Reviews (2) · Last reviewed commit: "test(common-utils): cover mixed query op..."

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

1 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 1 minor

1 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

This PR is titled a fix for #3032 (mixed AND/OR operator grouping), but after re-grading the merged reviewer findings the net change is test-only: the production fix was added in 3ee7657a and then fully reverted in the latest commit b19515c4. git diff <base>..HEAD -- packages/common-utils/src/queryParser.ts is empty, so serialize() still emits flat SQL, and the new test cases assert that flat output as expected — enshrining the exact bug #3032 describes.

Ground truth for correctness was established from the reverted fix's own assertions in 3ee7657a, which prove the Lucene parser is right-associative (a AND b OR c → AST a AND (b OR c), expected SQL ((a) AND ((b) OR (c)))).

🔴 P0/P1 — must fix

  • packages/common-utils/src/queryParser.ts — the serializeBinaryChild/getOperatorFamily grouping logic added in 3ee7657a was reverted in b19515c4, so the production serializer is byte-identical to base and Search mixing AND and OR without parentheses returns wrong rows #3032 remains unfixed despite the PR title; mixed queries like a AND b OR c still emit flat SQL that ClickHouse evaluates as (a AND b) OR c instead of the parsed a AND (b OR c).
    • Fix: Re-apply the operator-family-aware child serialization so a child binary node whose operator family differs from its parent and is not already parenthesized is wrapped in parentheses.
    • correctness, testing, previous-comments, kieran-typescript
  • packages/common-utils/src/__tests__/queryParser.test.ts:199 — new mixed-operator cases assert semantically incorrect flat SQL; a AND b OR c expects ((ServiceName = 'a') AND (ServiceName = 'b') OR (ServiceName = 'c')), which under ClickHouse AND-over-OR precedence means (a AND b) OR c, contradicting the parsed AST a AND (b OR c) and locking the suite against the real fix.
    • Fix: Correct the expected SQL for AND-parent/OR-child cases at lines 181, 199, 211, 230, and 242 to group the OR subtree (e.g. ((ServiceName = 'a') AND ((ServiceName = 'b') OR (ServiceName = 'c')))), matching the assertions in commit 3ee7657a.
    • correctness, testing

🟡 P2 — recommended

  • packages/common-utils/src/__tests__/queryParser.test.ts:187 — the NOT/AND NOT mixed cases (a NOT b OR c at line 187, a AND NOT b OR c at line 248) also assert flat SQL whose exclusion scope and OR grouping do not match the right-associative AST, so they encode incorrect precedence for negated operands.
    • Fix: Verify the exact NOT scoping against the parser and parenthesize the OR subtree so the emitted SQL matches the AST intent.
    • correctness
🔵 P3 nitpicks (1)
  • packages/common-utils/src/__tests__/queryParser.test.ts:186 — several added cases (e.g. lines 186/247, 198/229/241, 204/235) assert byte-identical sql/english from different operator spellings with no comment, so the intentional alias-equivalence coverage reads as copy-paste redundancy.
    • Fix: Group the alias-equivalence cases under a labeled describe block or inline comment (bare NOT == AND NOT, && == AND, || == OR, implicit == AND).

Reviewers (4): correctness, testing, previous-comments, kieran-typescript.

Testing gaps:

  • No test asserts the corrected grouped output for an implicit mixed query — every inner-grouping assertion (lines 175, 193, 218, 224, 260) comes from input that is already explicitly parenthesized, so none exercises the reverted serializeBinaryChild logic.
  • The english explanation assertions mirror the same flat/un-grouped expectations and share the false-confidence problem.
  • Tests were not executed in this environment (lucene package not installed); confirm the suite in CI once expected values are corrected.
  • The prior greptile "missing changeset" comment is now moot rather than violated — the user-facing behavior change it referenced was reverted, so no changeset is required for a test-only diff.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@brandon-pereira

Copy link
Copy Markdown
Member

@Tyagiquamar the review bot raised some valid comments to review. I also commented on the issue: #3032 (comment)

@Tyagiquamar Tyagiquamar changed the title fix(common-utils): preserve operator grouping for mixed AND/OR search queries test(common-utils): cover mixed query operator generation Oct 3, 2026
@Tyagiquamar

Copy link
Copy Markdown
Contributor Author

Updated in b19515c to the tests-only baseline you requested. The existing SQL/English table now covers 16 mixed-operator, explicit-parenthesis, symbolic, implicit, and negation cases. Query generation is unchanged from before this PR. Docker formatting, lint, type checking, package build, and all 421 query-parser tests pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants