Repository navigation
Conversation
…query fails getMapKeys returned [] from both text-index catch blocks, so a failed mergeTreeTextIndex query ended key discovery before the rollup and bounded scan paths ran. ClickHouse refuses mergeTreeTextIndex with ACCESS_DENIED for any user under a row policy on the table, so those users saw no Map keys. Log and fall through instead, matching getMapTextIndexKeyValues and getTextIndexKeyValues. The scan respects the row policy. Fixes ClickHouse#3265
🦋 Changeset detectedLatest commit: 310047e The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@squaredg is attempting to deploy a commit to the HyperDX Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @squaredg, thanks for the pull request! Before we review code from a first-time contributor we ask that a maintainer vouches for you, and you're not on our list yet. This PR stays open — it just isn't in the review queue until someone vouches. To get vouched, open an issue saying hello and what you're working on: https://github.com/hyperdxio/hyperdx/issues/new?template=introduce-yourself.md A maintainer will usually reply within a day or two, and then this PR gets picked up as normal. More detail in our contributing guide. |
| } catch (e) { | ||
| // Fall through rather than return []: ClickHouse refuses | ||
| // mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED), | ||
| // and the scan below still works and respects the policy. #3265 |
There was a problem hiding this comment.
🔵 minor — After a denied text-index query, the next step is the rollup table, not a scan that respects the row policy
When metadataMVs is set, the code after this catch tries the key/kv rollup at line 1082 before the scan. The rollup reads its own table, and a row policy on otel_logs does not apply to that table, so a user under the policy can see keys from rows they cannot read (where the code used to return []). The shipped rollups only index NativeColumn, so this mainly affects custom rollups. Fix: on ACCESS_DENIED, go straight to the scan and skip the rollup. Otherwise, correct the comment and the PR description so they don't say the fallback respects the policy.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
PR Review
3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor 1 posted as inline comment(s) on the changed lines. 2 listed below. Findings outside the changed lines2 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
|
| } catch (e) { | ||
| // Fall through rather than return []: ClickHouse refuses | ||
| // mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED), | ||
| // and the scan below still works and respects the policy. #3265 | ||
| console.warn( | ||
| 'getMapKeys rollup query failed for key text index query', | ||
| 'getMapKeys key text index query failed; falling back to the next strategy', | ||
| e, | ||
| ); | ||
| return []; | ||
| } |
There was a problem hiding this comment.
Rollup can expose restricted keys
If a connection has a row policy on the source table but can read its separate key or KV rollup table, a denied text-index query now falls through to the rollup before the source-table scan. The rollup is populated from the raw table, so autocomplete can return keys from rows the policy excludes. The KV-index catch has the same path. Skip the rollup after this denial unless it is known to enforce the same policy.
How this was verified: The failed index query reaches a separate rollup-table SELECT that can return keys before the source-table scan.
Deep Review✅ No critical issues found. The core fix is sound: removing 🟡 P2 -- recommended
🔵 P3 nitpicks (5)
Pre-existing (does not count toward verdict)
Reviewers (9): correctness, security, reliability, testing, maintainability, performance, kieran-typescript, project-standards, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
|
duplicate #3268 |
Summary
Map key autocomplete shows no keys on tables with row policies. ClickHouse refuses
mergeTreeTextIndexwithACCESS_DENIEDfor any user under a row policy on the table (ClickHouse/ClickHouse#119032).getMapKeyscaught that error and returned[], so the rollup and bounded-scan paths never ran.Both text-index catch blocks in
getMapKeysnow log and fall through to the next strategy, asgetMapTextIndexKeyValuesandgetTextIndexKeyValuesalready do. The scan reads the table itself, so ClickHouse applies the row policy and users only see keys from rows they can read.Nothing changes when the index query succeeds. When it fails, the cost is the failed query plus the scan. The scan result is cached under the same key, so later loads for that range skip both.
Tests: two cases where the key or kv index query is denied and keys come back from the scan, plus a control showing a successful index query still skips the scan. The two failure cases fail without the fix.
Verified on a local dev stack with ClickHouse 26.8.15.10, using
otel_logsfrom the seed schema, a user withSELECT ON default.*andsystem.data_skipping_indices, and a row policy onServiceName = 'api':mergeTreeTextIndexquery returns 403 andLogAttributes.autocomplete is empty.http.method,http.routeandapi.only_key, which are only the keys from the policy's rows. An admin connection still gets keys from the text index.How to test on Vercel preview
N/A. Reproducing this needs a ClickHouse user under a row policy, which the preview's demo ClickHouse doesn't have.
References
BAD_ARGUMENTSon Distributed tables)🤖 Generated with Claude Code