Repository navigation
fix(common-utils): fall back to the scan when the Map key text index query fails #3266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
| '@hyperdx/common-utils': patch | ||
| '@hyperdx/app': patch | ||
| --- | ||
|
|
||
| fix: fall back to the table scan when the Map key text index query fails, so key | ||
| autocomplete works on tables with row policies (where ClickHouse refuses | ||
| `mergeTreeTextIndex` with `ACCESS_DENIED`) instead of showing no keys. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1028,11 +1028,13 @@ export class Metadata { | |
| return keys; | ||
| } | ||
| } catch (e) { | ||
| // Fall through rather than return []: ClickHouse refuses | ||
| // mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED), | ||
| // and the scan below still works and respects the policy. #3265 | ||
| console.warn( | ||
| 'getMapKeys rollup query failed for key text index query', | ||
| 'getMapKeys key text index query failed; falling back to the next strategy', | ||
| e, | ||
| ); | ||
| return []; | ||
| } | ||
|
Comment on lines
1030
to
1038
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If a connection has a row policy on the source table but can read its separate key or KV rollup table, a denied text-index query now falls through to the rollup before the source-table scan. The rollup is populated from the raw table, so autocomplete can return keys from rows the policy excludes. The KV-index catch has the same path. Skip the rollup after this denial unless it is known to enforce the same policy. How this was verified: The failed index query reaches a separate rollup-table SELECT that can return keys before the source-table scan. |
||
| } else if ( | ||
| textIndexInfo?.kv?.indexName && | ||
|
|
@@ -1068,11 +1070,11 @@ export class Metadata { | |
| return keys; | ||
| } | ||
| } catch (e) { | ||
| // See the key text index catch above. | ||
| console.warn( | ||
| 'getMapKeys rollup query failed for kv text index query', | ||
| 'getMapKeys kv text index query failed; falling back to the next strategy', | ||
| e, | ||
| ); | ||
| return []; | ||
| } | ||
| } | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔵 minor — After a denied text-index query, the next step is the rollup table, not a scan that respects the row policy
When
metadataMVsis set, the code after this catch tries the key/kv rollup at line 1082 before the scan. The rollup reads its own table, and a row policy onotel_logsdoes not apply to that table, so a user under the policy can see keys from rows they cannot read (where the code used to return []). The shipped rollups only index NativeColumn, so this mainly affects custom rollups. Fix: on ACCESS_DENIED, go straight to the scan and skip the rollup. Otherwise, correct the comment and the PR description so they don't say the fallback respects the policy.Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond