Repository navigation
perf: compress the seed schemas with ZSTD(3) instead of ZSTD(1) - #3514
Conversation
The default codec in ClickHouse is ZSTD(3) since 26.9. Pin it explicitly, so that servers older than 26.9 (default LZ4) get it too. Drop the unneeded byte-size argument of Delta, which is inferred from the type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 4fb555a The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@alexey-milovidov is attempting to deploy a commit to the HyperDX Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @alexey-milovidov, thanks for the pull request! Before we review code from a first-time contributor, a maintainer vouches for them, and you're not on our list yet. There's nothing else to file and this PR stays open — a maintainer will look at it, usually within a day or two, and vouch for you right here if it's a genuine attempt at the problem. To make that quick, check the PR description has:
More detail in our contributing guide. |
PR Review✅ No issues found. |
Deep Review✅ No critical issues found. This is a mechanical, low-risk schema change: across the six ClickHouse seed files and the Key correctness checks pass:
🔵 P3 nitpicks (2)
Reviewers (6): correctness, testing, maintainability, project-standards, data-migrations, performance. Testing gaps: No automated check confirms the seed SQL ( No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
Summary
The seed schemas in
docker/otel-collector/schema/seed/(and their copy indocker/clickhouse/local/init-db-e2e.sh) pinZSTD(1)on almost every column.ZSTD(3)gives a noticeably better compression ratio for this kind of data at a small insert-time CPU cost, and decompression speed (so query performance) stays the same. It is also the default codec of ClickHouse since 26.9 (ClickHouse/ClickHouse@9c56196a1a4a).The codec is pinned explicitly rather than removed, because the bundled ClickHouse is 26.8 and the
_compatschemas target servers older than 26.2: on these versions the default isLZ4, so removing the codec would make compression worse.CODEC(ZSTD(1))→CODEC(ZSTD(3)),CODEC(T64, ZSTD(1))→CODEC(T64, ZSTD(3)).CODEC(Delta(8), ZSTD(1))→CODEC(Delta, ZSTD(3)): the byte-size argument ofDeltais inferred from the column type, so it is unneeded.Only newly created tables are affected (
CREATE TABLE IF NOT EXISTS); existing tables keep their codecs. Applied migrations inpackages/api/migrations/ch/and test fixtures are not changed.How to reproduce
Starting point: any ClickHouse server, or
clickhouse local.On this branch:
00002–00005, including_compat) were loaded into ClickHouse with${DATABASE}and${*_TTL}substituted (toIntervalDay(30)); every table is created successfully.References