Skip to content

Commit 022fe89

Browse files
fix: scope ClickHouse settings cache key by connectionId in NodeClickhouseClient (#2271)
## Summary - Fix MCP server queries failing with `Setting query_plan_optimize_lazy_materialization is neither a builtin setting` when querying ClickHouse instances that don't support that setting - Forward `connectionId` in `NodeClickhouseClient.__query()` to `processClickhouseSettings()`, matching the browser client's behavior - Guard the `metadata.getSettings()` lookup on `connectionId` presence inside `processClickhouseSettings()`, so default settings and externally-provided settings always apply even when `connectionId` is absent ## Problem `NodeClickhouseClient.__query()` did not destructure or forward `connectionId` to `processClickhouseSettings()`. This caused `metadata.getSettings()` to always use the cache key `"undefined.availableSettings"`. All `NodeClickhouseClient` instances on the API server (MCP tools, alert checkers, external API charts, etc.) share a single global `MetadataCache` with no TTL. When the first Node-side query hit a ClickHouse that **has** `query_plan_optimize_lazy_materialization` in `system.settings`, that Map was cached under `"undefined.availableSettings"`. Subsequent queries to a **different** ClickHouse (e.g. an older version) reused the stale cache, `applySettingIfAvailable` saw the setting as available, and ClickHouse rejected it. The browser client was unaffected because `BrowserClickhouseClient.__query()` already passes `connectionId`, scoping the cache key correctly as `"<connectionId>.availableSettings"`. ## Approach The `connectionId` guard is placed **inside** `processClickhouseSettings()` around only the `metadata.getSettings()` call, rather than wrapping the entire function at the call site. This ensures: - **Default settings** (`allow_experimental_analyzer`, `max_execution_time`, `output_format_json_quote_64bit_integers`, etc.) always apply - **Externally-provided** `clickhouse_settings` always flow through - **Server-specific optimization settings** (lazy materialization, skip indexes, etc.) are only looked up when `connectionId` is present, preventing the `"undefined.availableSettings"` cache key collision ## Changes | File | Change | |------|--------| | `packages/common-utils/src/clickhouse/node.ts` | Destructure `connectionId` from `QueryInputs` and forward it to `processClickhouseSettings()` | | `packages/common-utils/src/clickhouse/index.ts` | Guard `metadata.getSettings()` on `connectionId` presence inside `processClickhouseSettings()` — returns `undefined` serverSettings when missing, so defaults still apply | | `packages/common-utils/src/core/metadata.ts` | Make `connectionId` required (`string`, was `string?`) in `getSettings()` to prevent future callers from passing `undefined` | | `packages/common-utils/src/__tests__/clickhouse.test.ts` | Add `connectionId: 'test-conn'` to tests exercising the server settings lookup path |
1 parent d3a5a57 commit 022fe89

5 files changed

Lines changed: 29 additions & 5 deletions

File tree

‎.changeset/forty-beers-applaud.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@hyperdx/common-utils": patch
3+
---
4+
5+
Fix issue with incorrect cache key being set in settings queries in nodejs

‎packages/common-utils/src/__tests__/clickhouse.test.ts‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -472,6 +472,7 @@ describe('processClickhouseSettings - optimization settings', () => {
472472
await client.query({
473473
query: 'SELECT 1',
474474
format: 'JSON',
475+
connectionId: 'test-conn',
475476
});
476477

477478
const actualQueryCall = mockQueryMethod.mock.calls.find(
@@ -504,6 +505,7 @@ describe('processClickhouseSettings - optimization settings', () => {
504505
await client.query({
505506
query: 'SELECT 1',
506507
format: 'JSON',
508+
connectionId: 'test-conn',
507509
});
508510

509511
const actualQueryCall = mockQueryMethod.mock.calls.find(
@@ -531,6 +533,7 @@ describe('processClickhouseSettings - optimization settings', () => {
531533
await client.query({
532534
query: 'SELECT 1',
533535
format: 'JSON',
536+
connectionId: 'test-conn',
534537
clickhouse_settings: {
535538
max_rows_to_read: '1000000',
536539
},
@@ -582,6 +585,7 @@ describe('processClickhouseSettings - optimization settings', () => {
582585
await client.query({
583586
query: 'SELECT 1',
584587
format: 'JSON',
588+
connectionId: 'test-conn',
585589
});
586590

587591
const actualQueryCall = mockQueryMethod.mock.calls.find(
@@ -602,8 +606,16 @@ describe('processClickhouseSettings - optimization settings', () => {
602606
setupMockQuery([{ name: 'use_skip_indexes_for_top_k', value: '1' }]);
603607

604608
// Run two queries
605-
await client.query({ query: 'SELECT 1', format: 'JSON' });
606-
await client.query({ query: 'SELECT 2', format: 'JSON' });
609+
await client.query({
610+
query: 'SELECT 1',
611+
format: 'JSON',
612+
connectionId: 'test-conn',
613+
});
614+
await client.query({
615+
query: 'SELECT 2',
616+
format: 'JSON',
617+
connectionId: 'test-conn',
618+
});
607619

608620
// Should only fetch settings once
609621
const settingsCalls = mockQueryMethod.mock.calls.filter(

‎packages/common-utils/src/clickhouse/index.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -532,8 +532,13 @@ export abstract class BaseClickhouseClient {
532532
output_format_json_quote_64bit_integers: 1, // In 25.8, the default value for this was changed from 1 to 0. Due to JavaScript's poor precision for big integers, we should enable this https://github.com/ClickHouse/ClickHouse/pull/74079
533533
};
534534

535-
const metadata = getMetadata(this);
536-
const serverSettings = await metadata.getSettings({ connectionId });
535+
// Only look up server-specific optimization settings when we have a
536+
// connectionId to scope the cache key. Without one the cache key would
537+
// be "undefined.availableSettings", which can collide across different
538+
// ClickHouse instances sharing the same MetadataCache singleton.
539+
const serverSettings = connectionId
540+
? await getMetadata(this).getSettings({ connectionId })
541+
: undefined;
537542

538543
const applySettingIfAvailable = (name: string, value: string) => {
539544
if (!serverSettings || !serverSettings.has(name)) return;

‎packages/common-utils/src/clickhouse/node.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ export class ClickhouseClient extends BaseClickhouseClient {
3434
query_params = {},
3535
abort_signal,
3636
clickhouse_settings: externalClickhouseSettings,
37+
connectionId,
3738
queryId,
3839
shouldSkipApplySettings,
3940
}: QueryInputs<Format>): Promise<BaseResultSet<ReadableStream, Format>> {
@@ -44,6 +45,7 @@ export class ClickhouseClient extends BaseClickhouseClient {
4445
if (!shouldSkipApplySettings) {
4546
clickhouseSettings = await this.processClickhouseSettings({
4647
externalClickhouseSettings,
48+
connectionId,
4749
});
4850
}
4951

‎packages/common-utils/src/core/metadata.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -845,7 +845,7 @@ export class Metadata {
845845
});
846846
}
847847

848-
async getSettings({ connectionId }: { connectionId?: string }) {
848+
async getSettings({ connectionId }: { connectionId: string }) {
849849
return this.cache.getOrFetch(
850850
`${connectionId}.availableSettings`,
851851
async () => {

0 commit comments

Comments
 (0)