Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/map-keys-text-index-fallthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@hyperdx/common-utils': patch
'@hyperdx/app': patch
---

fix: fall back to the table scan when the Map key text index query fails, so key
autocomplete works on tables with row policies (where ClickHouse refuses
`mergeTreeTextIndex` with `ACCESS_DENIED`) instead of showing no keys.
103 changes: 101 additions & 2 deletions packages/common-utils/src/__tests__/metadata.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { ColumnMeta } from '@/clickhouse';
import { ClickHouseQueryError, ColumnMeta } from '@/clickhouse';
import { ClickhouseClient } from '@/clickhouse/node';
import {
GET_ALL_KEY_VALUES_CHUNK_SIZE,
Expand All @@ -10,7 +10,7 @@ import {
import * as renderChartConfigModule from '@/core/renderChartConfig';
import { timeFilterExpr } from '@/core/renderChartConfig';
import { isBuilderChartConfig } from '@/guards';
import { TextIndexInfoLookup } from '@/queryParser';
import { KvIndexInfo, TextIndexInfo, TextIndexInfoLookup } from '@/queryParser';
import { BuilderChartConfigWithDateRange, SourceKind, TSource } from '@/types';

// Mock ClickhouseClient
Expand Down Expand Up @@ -2025,6 +2025,105 @@ describe('Metadata', () => {
warn.mockRestore();
});

describe('text index query failures', () => {
const dateRange: [Date, Date] = [
new Date('2026-05-11T16:00:00Z'),
new Date('2026-05-11T17:00:00Z'),
];
const getKeys = (md: Metadata) =>
md.getMapKeys({
databaseName: 'otel',
tableName: 'otel_logs',
column: 'LogAttributes',
connectionId: 'conn-1',
dateRange,
timestampValueExpression: 'Timestamp',
});

// ClickHouse refuses mergeTreeTextIndex for any user under a row policy
// on the table (ClickHouse#119032); the scan respects the policy.
const rowPolicyDenied = () =>
Promise.reject(
new ClickHouseQueryError(
'Cannot read from `mergeTreeTextIndex` because a row policy is applied on table otel.otel_logs. (ACCESS_DENIED)',
'',
),
);

const buildWithTextIndex = (textIndexInfo: TextIndexInfo) => {
const md = new Metadata(mockClickhouseClient, new MetadataCache());
jest.spyOn(md, 'getServerVersion').mockResolvedValue([26, 8, 0, 0]);
jest
.spyOn(md, 'getMapColumnTextIndexes')
.mockResolvedValue(new Map([['LogAttributes', textIndexInfo]]));
return md;
};

const mockQueries = (onTextIndex: () => Promise<any>) =>
(mockClickhouseClient.query as jest.Mock).mockImplementation(
({ query }: any) => {
if (query.includes('mergeTreeTextIndex(')) return onTextIndex();
if (query.includes('sampledKeys')) {
return Promise.resolve({
json: () => Promise.resolve({ data: [{ key: 'http.method' }] }),
});
}
return Promise.resolve({
json: () => Promise.resolve({ data: [lowCardinalityMapColumn] }),
});
},
);

const queriesSent = () =>
(mockClickhouseClient.query as jest.Mock).mock.calls.map(
([opts]) => opts.query as string,
);

let warn: jest.SpyInstance;
beforeEach(() => {
warn = jest.spyOn(console, 'warn').mockImplementation(() => {});
});
afterEach(() => warn.mockRestore());

it('falls back to the scan when the key text index query is denied', async () => {
const md = buildWithTextIndex({
key: { indexName: 'idx_log_attr_key', mapColumn: 'LogAttributes' },
});
mockQueries(rowPolicyDenied);

await expect(getKeys(md)).resolves.toEqual(['http.method']);
expect(queriesSent().some(q => q.includes('sampledKeys'))).toBe(true);
});

it('falls back to the scan when the kv text index query is denied', async () => {
const md = buildWithTextIndex({
kv: {
columnName: 'LogAttributeItems',
indexName: 'idx_log_attr_items',
separator: '=',
} as KvIndexInfo,
});
mockQueries(rowPolicyDenied);

await expect(getKeys(md)).resolves.toEqual(['http.method']);
expect(queriesSent().some(q => q.includes('sampledKeys'))).toBe(true);
});

it('returns text index keys without scanning when the index query succeeds', async () => {
const md = buildWithTextIndex({
key: { indexName: 'idx_log_attr_key', mapColumn: 'LogAttributes' },
});
mockQueries(() =>
Promise.resolve({
json: () => Promise.resolve({ data: [{ key: 'from.index' }] }),
}),
);

await expect(getKeys(md)).resolves.toEqual(['from.index']);
expect(queriesSent().some(q => q.includes('sampledKeys'))).toBe(false);
});
});

it('scans the last 24h when the caller supplies no dateRange', async () => {
const md = buildMetadata();
(mockClickhouseClient.query as jest.Mock)
Expand Down
10 changes: 6 additions & 4 deletions packages/common-utils/src/core/metadata.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1028,11 +1028,13 @@ export class Metadata {
return keys;
}
} catch (e) {
// Fall through rather than return []: ClickHouse refuses
// mergeTreeTextIndex under any row policy on the table (ACCESS_DENIED),
// and the scan below still works and respects the policy. #3265

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — After a denied text-index query, the next step is the rollup table, not a scan that respects the row policy

When metadataMVs is set, the code after this catch tries the key/kv rollup at line 1082 before the scan. The rollup reads its own table, and a row policy on otel_logs does not apply to that table, so a user under the policy can see keys from rows they cannot read (where the code used to return []). The shipped rollups only index NativeColumn, so this mainly affects custom rollups. Fix: on ACCESS_DENIED, go straight to the scan and skip the rollup. Otherwise, correct the comment and the PR description so they don't say the fallback respects the policy.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

console.warn(
'getMapKeys rollup query failed for key text index query',
'getMapKeys key text index query failed; falling back to the next strategy',
e,
);
return [];
}
Comment on lines 1030 to 1038

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Rollup can expose restricted keys

If a connection has a row policy on the source table but can read its separate key or KV rollup table, a denied text-index query now falls through to the rollup before the source-table scan. The rollup is populated from the raw table, so autocomplete can return keys from rows the policy excludes. The KV-index catch has the same path. Skip the rollup after this denial unless it is known to enforce the same policy.

How this was verified: The failed index query reaches a separate rollup-table SELECT that can return keys before the source-table scan.

Fix in Claude Code Fix in Conductor Fix in Cursor Fix in Codex

} else if (
textIndexInfo?.kv?.indexName &&
Expand Down Expand Up @@ -1068,11 +1070,11 @@ export class Metadata {
return keys;
}
} catch (e) {
// See the key text index catch above.
console.warn(
'getMapKeys rollup query failed for kv text index query',
'getMapKeys kv text index query failed; falling back to the next strategy',
e,
);
return [];
}
}

Expand Down
Loading