Skip to content

fix(common-utils): fall back to the scan when the Map key text index query fails - #3266

Closed
squaredg wants to merge 1 commit into
ClickHouse:mainfrom
squaredg:claude/map-keys-text-index-fallthrough
Closed

squaredg wants to merge 1 commit into
ClickHouse:mainfrom
squaredg:claude/map-keys-text-index-fallthrough

Conversation

@squaredg

@squaredg squaredg commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Map key autocomplete shows no keys on tables with row policies. ClickHouse refuses mergeTreeTextIndex with ACCESS_DENIED for any user under a row policy on the table (ClickHouse/ClickHouse#119032). getMapKeys caught that error and returned [], so the rollup and bounded-scan paths never ran.

Both text-index catch blocks in getMapKeys now log and fall through to the next strategy, as getMapTextIndexKeyValues and getTextIndexKeyValues already do. The scan reads the table itself, so ClickHouse applies the row policy and users only see keys from rows they can read.

Nothing changes when the index query succeeds. When it fails, the cost is the failed query plus the scan. The scan result is cached under the same key, so later loads for that range skip both.

Tests: two cases where the key or kv index query is denied and keys come back from the scan, plus a control showing a successful index query still skips the scan. The two failure cases fail without the fix.

Verified on a local dev stack with ClickHouse 26.8.15.10, using otel_logs from the seed schema, a user with SELECT ON default.* and system.data_skipping_indices, and a row policy on ServiceName = 'api':

  • Before: the mergeTreeTextIndex query returns 403 and LogAttributes. autocomplete is empty.
  • After: the same 403, then the scan runs and autocomplete lists http.method, http.route and api.only_key, which are only the keys from the policy's rows. An admin connection still gets keys from the text index.

How to test on Vercel preview

N/A. Reproducing this needs a ClickHouse user under a row policy, which the preview's demo ClickHouse doesn't have.

References

🤖 Generated with Claude Code

…query fails

getMapKeys returned [] from both text-index catch blocks, so a failed
mergeTreeTextIndex query ended key discovery before the rollup and bounded
scan paths ran. ClickHouse refuses mergeTreeTextIndex with ACCESS_DENIED for
any user under a row policy on the table, so those users saw no Map keys.

Log and fall through instead, matching getMapTextIndexKeyValues and
getTextIndexKeyValues. The scan respects the row policy.

Fixes ClickHouse#3265
@changeset-bot

changeset-bot Bot commented Oct 2, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 310047e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/api Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 2, 2026

Copy link
Copy Markdown

@squaredg is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added external needs-vouch Author needs a maintainer to vouch for them labels Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Hi @squaredg, thanks for the pull request!

Before we review code from a first-time contributor we ask that a maintainer vouches for you, and you're not on our list yet. This PR stays open — it just isn't in the review queue until someone vouches.

To get vouched, open an issue saying hello and what you're working on:

https://github.com/hyperdxio/hyperdx/issues/new?template=introduce-yourself.md

A maintainer will usually reply within a day or two, and then this PR gets picked up as normal. More detail in our contributing guide.

} catch (e) {
// Fall through rather than return []: ClickHouse refuses
// mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED),
// and the scan below still works and respects the policy. #3265

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — After a denied text-index query, the next step is the rollup table, not a scan that respects the row policy

When metadataMVs is set, the code after this catch tries the key/kv rollup at line 1082 before the scan. The rollup reads its own table, and a row policy on otel_logs does not apply to that table, so a user under the policy can see keys from rows they cannot read (where the code used to return []). The shipped rollups only index NativeColumn, so this mainly affects custom rollups. Fix: on ACCESS_DENIED, go straight to the scan and skip the rollup. Otherwise, correct the comment and the PR description so they don't say the fallback respects the policy.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor

1 posted as inline comment(s) on the changed lines. 2 listed below.

Findings outside the changed lines

2 minor
  • 🔵 packages/common-utils/src/__tests__/metadata.test.ts:114 — The as KvIndexInfo cast hides missing required fields (useHasAny, mapColumn) → KvIndexInfo (packages/common-utils/src/queryParser.ts:971) requires useHasAny and mapColumn. The cast lets the fixture leave them out, and the repo conventions say to avoid as. Fix: add useHasAny: true, mapColumn: 'LogAttributes' and use satisfies KvIndexInfo, or no annotation at all.
  • 🔵 packages/common-utils/src/__tests__/metadata.test.ts:98 — No test covers a denied text-index query when metadataMVs is configured → Every new case leaves out metadataMVs, so the new route from a failed index query into the rollup branch is never run. Fix: add a case that passes metadataMVs and asserts which query supplies the keys after an ACCESS_DENIED. Also assert that a second call is served from the cache instead of repeating the failing mergeTreeTextIndex query, which the PR description promises.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@greptile-apps

greptile-apps Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 3/5

[Medium risk] Changes error handling in key autocomplete queries.

This PR should not merge until denied index queries cannot return keys from a rollup that is less restricted than the source table.

Fix All in Claude CodeFindings

  1. P1 Security Rollup can expose restricted keys ▶

Summary

The PR lets Map-key discovery continue to a rollup or bounded table scan when a ClickHouse text-index query fails, and adds regression tests for denied index queries.

  • The rollup path needs attention for connections whose source-table row policy does not also protect the rollup.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Map-key text-index query] -->|success| B[Return index keys]
  A -->|failure| C{Rollup configured?}
  C -->|yes, keys found| D[Return rollup keys]
  C -->|no or empty| E[Scan source table]
Loading

Reviews (1) · Last reviewed commit: "fix(common-utils): fall back to the scan..."

Comment on lines 1030 to 1038
} catch (e) {
// Fall through rather than return []: ClickHouse refuses
// mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED),
// and the scan below still works and respects the policy. #3265
console.warn(
'getMapKeys rollup query failed for key text index query',
'getMapKeys key text index query failed; falling back to the next strategy',
e,
);
return [];
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Rollup can expose restricted keys

If a connection has a row policy on the source table but can read its separate key or KV rollup table, a denied text-index query now falls through to the rollup before the source-table scan. The rollup is populated from the raw table, so autocomplete can return keys from rows the policy excludes. The KV-index catch has the same path. Skip the rollup after this denial unless it is known to enforce the same policy.

How this was verified: The failed index query reaches a separate rollup-table SELECT that can return keys before the source-table scan.

Fix in Claude Code Fix in Conductor Fix in Cursor Fix in Codex

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. The core fix is sound: removing return [] from both text-index catch blocks correctly lets a denied mergeTreeTextIndex query fall through, the base-table scan respects the row policy, the cache is not poisoned, and the three new tests genuinely fail without the fix. The items below are recommended follow-ups, not blockers.

🟡 P2 -- recommended

  • packages/common-utils/src/core/metadata.ts:1082 -- When a row policy denies the text index, the fallthrough now reaches the rollup path, which queries a separate keyRollupTable/kvRollupTable grouped only by ColumnIdentifier; ClickHouse row policies do not propagate to that distinct table, so if a deployment's rollups index the Map column the query can return keys from rows the user cannot read, whereas the scan path that follows is the only one that re-applies the policy.
    • Fix: On a denied text-index query, bypass the rollup block and fall through directly to the base-table scan so results are always evaluated under the row policy; alternatively only trust the rollup when a matching policy is known to exist on the rollup table.
  • packages/common-utils/src/__tests__/metadata.test.ts:2088 -- The failure tests pass no metadataMVs, so they skip the rollup block entirely and jump straight to the scan, leaving the newly-reachable index-failure-to-rollup transition uncovered.
    • Fix: Add a failure-path test that supplies metadataMVs and alignedDateRange and asserts the rollup query behavior after a denied text index, covering the hop the current scan-only tests bypass.
🔵 P3 nitpicks (5)
  • packages/common-utils/src/core/metadata.ts:1030 -- The broadened catch now falls through for every error class, so a transient timeout, network error, or abort on the text-index query triggers the rollup + full scan instead of failing fast.
    • Fix: Narrow the fall-through to the ACCESS_DENIED case and rethrow or return [] for other error classes so a slow index does not incur an extra scan on each autocomplete call.
    • correctness, reliability
  • packages/common-utils/src/core/metadata.ts:1034 -- Both catch blocks log identical warn text for the expected row-policy denial and for genuine failures, so an operator cannot distinguish a benign denial from a real outage without parsing the error object.
    • Fix: Branch on the ClickHouse error code so expected ACCESS_DENIED denials log at debug/info while other failures stay at warn.
    • reliability, maintainability
  • packages/common-utils/src/__tests__/metadata.test.ts:2104 -- The as KvIndexInfo assertion silences two missing required fields (useHasAny, mapColumn) rather than exposing the shape gap, while the sibling key fixture type-checks without a cast.
    • Fix: Add useHasAny and mapColumn to the kv fixture and drop the as KvIndexInfo assertion.
  • packages/common-utils/src/__tests__/metadata.test.ts:2088 -- No test asserts the diagnostic warn branch fires on a denied text-index query; the swallow-not-rethrow behavior is only implicitly proven by resolves.
    • Fix: Assert console.warn is called with the fall-back message in the key and kv denial tests.
  • packages/common-utils/src/core/metadata.ts:1082 -- Under a permanently-denying row policy, moving-window dashboards without MVs produce a distinct cacheKey per refresh and re-run the bounded scan each time instead of the prior free [] return.
    • Fix: Consider short-TTL negative-caching the index-query denial per (connectionId, db, table, column) so repeated distinct-range misses skip re-attempting the doomed index query.

Pre-existing (does not count toward verdict)

  • packages/common-utils/src/core/metadata.ts:1016 -- The mergeTreeTextIndex queries omit the max_execution_time: 15, timeout_overflow_mode: 'break', and abort_signal that the rollup and scan queries set, so a hung index query is unbounded and ignores caller cancellation; this is untouched by the diff but now sits on the common path.

Reviewers (9): correctness, security, reliability, testing, maintainability, performance, kieran-typescript, project-standards, previous-comments.

Testing gaps:

  • Index-failure-to-rollup transition is untested (all failure tests omit metadataMVs and reach only the scan).
  • No test exercises a non-ACCESS_DENIED failure (timeout/generic error) falling through to the scan.
  • No assertion that the scan result is cached after a denial so repeated same-range loads skip re-scanning.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@wrn14897

wrn14897 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

duplicate #3268

@wrn14897 wrn14897 closed this Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external needs-vouch Author needs a maintainer to vouch for them

Projects

None yet

Development

Successfully merging this pull request may close these issues.

getMapKeys returns no keys when the mergeTreeTextIndex query fails (e.g. ACCESS_DENIED under row policies) instead of falling through

2 participants