Area: dedupe — gap + latent correctness bug · found via WaveHouse-Stats dogfooding
Expected: each table dedupes on its own natural key, in its own keyspace.
Actual: two problems with the single global dedupe key:
- One global
id_field name applies deployment-wide (internal/config/config.go:105-108) — every table that wants dedupe must carry an identically-named column.
- Shared keyspace (latent bug):
CheckAndMark keys Pebble on the raw id value with no table namespace (internal/dedupe/embedded.go:29, key := []byte(eventID)). Equal id values across two different tables collide — the second is silently dropped as a "duplicate." Only bites when ids aren't globally unique across tables (e.g. per-table sequential ids), but it's silent data loss when it does.
Impact (Stats): we standardize on globally-unique event_id, so the collision doesn't bite us today — but it's a sharp edge for any multi-table deployment, and the single id_field forces a naming convention on every future table.
Scope: namespace the dedupe key by table (fixes #2 regardless), and allow a per-table id_field (fixes #1). The keyspace fix is the more urgent half.
Related: #219, #220, #221.
From WAVEHOUSE-FEEDBACK.md (WaveHouse-Stats dogfooding); validated by code-read against 0f8826c on 2026-06-04.
Area: dedupe — gap + latent correctness bug · found via WaveHouse-Stats dogfooding
Expected: each table dedupes on its own natural key, in its own keyspace.
Actual: two problems with the single global dedupe key:
id_fieldname applies deployment-wide (internal/config/config.go:105-108) — every table that wants dedupe must carry an identically-named column.CheckAndMarkkeys Pebble on the raw id value with no table namespace (internal/dedupe/embedded.go:29,key := []byte(eventID)). Equal id values across two different tables collide — the second is silently dropped as a "duplicate." Only bites when ids aren't globally unique across tables (e.g. per-table sequential ids), but it's silent data loss when it does.Impact (Stats): we standardize on globally-unique
event_id, so the collision doesn't bite us today — but it's a sharp edge for any multi-table deployment, and the single id_field forces a naming convention on every future table.Scope: namespace the dedupe key by table (fixes #2 regardless), and allow a per-table id_field (fixes #1). The keyspace fix is the more urgent half.
Related: #219, #220, #221.
From
WAVEHOUSE-FEEDBACK.md(WaveHouse-Stats dogfooding); validated by code-read against0f8826con 2026-06-04.