Repository navigation
feat: replicated schema migration - #3309
SpencerTorres wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: f07caf7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
🔴 Tier 4 — CriticalTouches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI. Why this tier:
Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
Duplicate code analysisNo new duplicate code introduced by this PR. jscpd compared this branch against |
|
|
||
| // A fallback database keeps the plain engines it was seeded with. | ||
| replicated := false | ||
| if database == cfg.Database { |
There was a problem hiding this comment.
🔵 minor — Replicated mode is dropped without a warning when the fallback database is used
With HYPERDX_OTEL_EXPORTER_CLICKHOUSE_REPLICATED=true and a fallback that wins (the target is missing and the fallback has tables), planReplicated never runs. The fallback is seeded with plain engines and nothing says Replicated was not applied. Log a warning in that branch, for example: if cfg.Replicated && database != cfg.Database { log.Printf("WARNING: Replicated engines not applied; staying on fallback %s", database) }.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
PR Review
3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor 1 posted as inline comment(s) on the changed lines. 2 listed below. Findings outside the changed lines2 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
|
| // --------------------------------------------------------------------------- | ||
| // replicated engines | ||
| // --------------------------------------------------------------------------- |
There was a problem hiding this comment.
The new section dividers violate the AGENTS.md directive: “No banner comments, no ASCII section dividers.” This rule also applies to tests. Remove this banner and the matching fallback banner at lines 974–976 before merging to satisfy the repository requirement.
Context Used: AGENTS.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Deep ReviewScope: PR #3309 ✅ No critical issues found. No P0/P1: both features are opt-in, the fallback write path is fail-closed ( 🟡 P2 — recommended
🔵 P3 nitpicks (6)
Reviewers (11): correctness, testing, maintainability, project-standards, data-migrations, reliability, adversarial, deployment-verification, previous-comments, learnings-researcher, agent-native. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
E2E Test Results✅ All tests passed • 398 passed • 1 skipped • 1523s
Tests ran across 4 shards in parallel. |
wrn14897
left a comment
There was a problem hiding this comment.
Could you elaborate on why we need HYPERDX_OTEL_EXPORTER_CLICKHOUSE_FALLBACK_DATABASE?
| func hasClickStackTables(ctx context.Context, db *sql.DB, database string) (bool, error) { | ||
| var n uint64 | ||
| err := db.QueryRowContext(ctx, | ||
| "SELECT count() FROM system.tables WHERE database = ? AND name IN ('otel_logs', 'otel_traces')", database).Scan(&n) |
There was a problem hiding this comment.
do we want to check only logs and traces here?
| -- +goose Up | ||
| -- +goose NO TRANSACTION | ||
| CREATE DATABASE IF NOT EXISTS ${DATABASE}; | ||
| CREATE DATABASE IF NOT EXISTS ${DATABASE}${DATABASE_ENGINE}; |
There was a problem hiding this comment.
What's the possible value of DATABASE_ENGINE here?
Summary
Adds opt-in support for ClickHouse's Replicated engines to the schema migration, so the database and tables can replicate across nodes.
HYPERDX_OTEL_EXPORTER_CLICKHOUSE_REPLICATED=truecreates the database with theReplicatedengine and seedsReplicated*tables. An existing non-Replicated database keeps plain engines, with a warning.HYPERDX_OTEL_EXPORTER_CLICKHOUSE_FALLBACK_DATABASEkeeps writing to an existing database when the configured one doesn't exist yet. The collector is pointed at whichever database the migration used.Both are off by default, and nothing changes when unset. Tested with unit tests and against ClickHouse 25.7 and 26.2 with Keeper, including multi-node replication.
References