refactor(storage): establish canonical Bun schema and DuckDB dialect

[mariusvniekerk]

SQLite, PostgreSQL, and DuckDB previously evolved independent schema and query foundations, so parity depended on duplicated declarations and backend-specific plumbing. This first stack layer establishes one Bun model registry, guarded backend handles, and a dedicated DuckDB dialect while converging SQLite and PostgreSQL in place under their existing serialization boundaries.

Stamped schemas now validate structural, index, and trigger drift without repair; expensive row invariants remain limited to one-time convergence. Shared timestamps preserve the shipped SQLite empty sentinel and RFC3339Nano text contract, and Quack rejects direct driver arguments that would otherwise be discarded.

Direct database/sql access remains limited to lifecycle, connection-local, transport, and compatibility seams. DuckDB remains a disposable mirror, while narrowly scoped FTS and vector behavior stays adapter-specific. Stack 1 of 5.

^{generated by a clanker}

[roborev-ci]

roborev: Combined Review (ce3270c)

Medium-severity schema compatibility issues remain in the storage refactor.

Medium

• internal/db/bunmodel/identity.go:72 — The canonical worktree mapping model generates TIMESTAMP DEFAULT CURRENT_TIMESTAMP columns in DuckDB, while Quack attach rejects catalogs containing dynamic timestamp defaults. Schema v10 mirrors therefore break remote Quack access.

  • Fix: Remove these defaults for DuckDB and explicitly supply created_at and updated_at in mirror writers, or strip the defaults in the DuckDB schema adapter.

• internal/postgres/schema.go:1414 — PostgreSQL convergence rejects sessions with empty source provenance without first migrating legacy rows. Databases upgraded from versions predating these columns receive empty defaults and fail EnsureSchema, blocking the push that would populate them.

  • Fix: Backfill legacy provenance where ownership is known, or allow legacy rows through convergence and require a full push before stamping compatibility.

---

Reviewers: 2 done | Synthesis: codex, 13s | Total: 19m21s

[roborev-ci]

roborev: Combined Review (7899144)

The refactor has two medium-severity data-integrity issues; no security regressions were identified.

Medium

• internal/duckdb/push.go:829 — Pin ID reuse can abort curation refreshes. Curation refresh upserts pins before deleting stale rows. SQLite may reuse a deleted highest pin ID, causing a new pin to collide with a stale DuckDB primary key despite having a different (session_id, ordinal). Delete stale pins before inserting current pins, or avoid using source pin IDs as DuckDB primary keys. Add a regression test for SQLite ID reuse.

• internal/postgres/schema.go:1349 — Schema convergence omits canonical foreign keys. Legacy tables already exist when CreateCommonSchema runs, so CREATE TABLE IF NOT EXISTS cannot add constraints such as the composite message/tool-call relationships. CheckCommonSchema can nevertheless stamp the schema as compatible. Add idempotent ALTER TABLE ... ADD CONSTRAINT migrations for all canonical foreign keys and verify them through pg_constraint before writing the compatibility stamp.

---

Reviewers: 2 done | Synthesis: codex, 12s | Total: 20m2s

[roborev-ci]

roborev: Combined Review (0a688f7)

High-severity PostgreSQL schema convergence issue and two medium-severity migration gaps must be addressed before merge.

High

• internal/postgres/schema.go:2655 — pushSchemaCurrent accepts pre-change PostgreSQL schemas without validating the common-schema stamp, newly required columns, or native pricing timestamp types. A normal pg push can therefore skip convergence and leave updated_at as TEXT, causing later pricing updates that use timestamp/interval operators to fail.
  • Fix: Require bun_common_schema_v1, then validate the common schema and pricing column types before taking the fast path.

Medium

• internal/postgres/schema.go:1337, internal/postgres/push.go:2253 — PostgreSQL push, scan, compatibility, and fingerprint paths do not fully handle the newly canonical session fields: file_size, file_mtime, file_inode, file_device, file_hash, and local_modified_at. Values remain NULL, and changes to most fields can be silently skipped.

  • Fix: Add these fields to insert/update arguments, the conflict predicate, sessionPushFingerprint, pgSessionCols, scanPGSession, and the compatibility probe.

• internal/postgres/schema.go:1404 — CreateCommonSchema relies on CREATE TABLE IF NOT EXISTS, which cannot add canonical composite foreign keys to tables previously created by coreDDL. Required message-to-tool-call, tool-call-to-result, and message-to-pin relationships may remain absent even though CheckCommonSchema stamps the migration complete.

  • Fix: Add explicit idempotent PostgreSQL constraint migrations and validate the required foreign keys before writing the compatibility stamp.

---

Reviewers: 2 done | Synthesis: codex, 12s | Total: 13m20s

[roborev-ci]

roborev: Combined Review (95a5c2b)

Changes requested: four medium-severity backend persistence and schema-convergence issues remain.

Medium

• internal/postgres/schema.go:1337 — Six new session file-metadata columns are omitted from PostgreSQL pushes, reads, compatibility checks, and mostly from sessionPushFingerprint. Values remain NULL, and metadata-only updates may be skipped. Add all six columns throughout these paths and add round-trip tests.

• internal/duckdb/push.go:1038 — parser_parent_session_id is omitted from DuckDB insert, update, argument, and fingerprint paths, losing parser-derived parent relationships. Include ParserParentSessionID and add a round-trip test where it differs from ParentSessionID.

• internal/postgres/worktree_mappings_push.go:157 — Mapping publication omits created_at, replacing source creation timestamps with migration or insertion time. Publish CreatedAt, update compatibility checks, and republish existing mappings to correct defaulted values.

• internal/postgres/schema.go:1349 — Canonical foreign keys are not migrated into existing PostgreSQL tables. Add idempotent constraint migrations, validate them against the PostgreSQL catalog, and test that orphaned relationships are rejected.

---

Reviewers: 2 done | Synthesis: codex, 9s | Total: 24m27s

[roborev-ci]

roborev: Combined Review (6d0f950)

Storage convergence has two medium-severity correctness issues; no critical or high-severity findings were identified.

Medium

• DuckDB pin ID collision — internal/db/bunmodel/curation.go:15, internal/duckdb/push.go:829
  DuckDB treats copied SQLite pin IDs as primary keys. Because SQLite can reuse an INTEGER PRIMARY KEY after unpinning, inserting a replacement before deleting the stale mirror row can fail on id, despite the upsert targeting (session_id, ordinal). Let DuckDB generate mirror-local IDs, make id non-key data, or clear existing pins before reinsertion.

• Missing PostgreSQL foreign-key convergence — internal/postgres/schema.go:1349
  Convergence stamps the schema without adding most registered canonical foreign keys. CREATE TABLE IF NOT EXISTS cannot retrofit constraints onto existing coreDDL tables, leaving relationships such as sessions-to-archives, pins-to-messages, and results-to-tool-calls unenforced. Add idempotent constraint migrations, update fresh DDL, and validate constraints before writing the compatibility stamp.

---

Reviewers: 2 done | Synthesis: codex, 8s | Total: 14m47s

[roborev-ci]

roborev: Combined Review (6791f37)

The storage refactor has four medium-severity schema-parity and data-integrity issues across PostgreSQL and DuckDB.

Medium

• PostgreSQL provenance fields are not returned — internal/postgres/sessions.go:93
  Session queries neither select nor scan source_archive_id and source_database_generation, so PostgreSQL-backed reads silently return empty provenance even though push code stores it. Add both columns to pgSessionCols and destinations to scanPGSession, plus a PostgreSQL round-trip test.

• DuckDB drops parser lineage — internal/duckdb/push.go:1026, internal/duckdb/sync.go:1041, internal/duckdb/store.go:344
  Although parser_parent_session_id is created, it is omitted from writes, fingerprints, reads, and scans. Lineage is lost, and changes cannot trigger incremental updates. Wire the field through each path and test differing parser and effective parent IDs.

• DuckDB schema version was not incremented — internal/duckdb/schema.go:25
SchemaVersion remains 10 despite an incompatible layout change, allowing materially different mirror schemas to appear compatible. Increment the version and update metadata expectations, fixtures, and documentation.

• PostgreSQL convergence omits canonical foreign keys — internal/postgres/schema.go:1501
  Convergence creates existing tables with IF NOT EXISTS and stamps the canonical schema without installing or validating required foreign keys for source provenance, tool calls/results, and pinned messages. Add idempotent constraint migrations and validate the PostgreSQL catalog before writing the compatibility stamp.

---

Reviewers: 2 done | Synthesis: codex, 26s | Total: 23m48s

[roborev-ci]

roborev: Combined Review (f2df2ed)

Changes requested: two medium-severity compatibility issues could break CockroachDB initialization and leave stale DuckDB mirrors unreconstructed.

Medium

• internal/postgres/schema.go:1514 — CockroachDB does not support PostgreSQL advisory locks. Common-schema convergence now unconditionally calls pg_advisory_xact_lock, causing CockroachDB schema checks to fail before compatibility validation, even for current databases. Detect CockroachDB and use a compatible migration-serialization mechanism while retaining advisory locks for PostgreSQL. Add integration coverage for both initialization paths.

• internal/duckdb/schema.go:25 — DuckDB schema version was not incremented. The schema remains at version 10 despite material changes to columns, types, keys, indexes, and tables. Existing version-10 mirrors may pass compatibility checks despite having an incompatible physical schema. Increment SchemaVersion and update related expectations so existing mirrors are rebuilt.

---

Reviewers: 2 done | Synthesis: codex, 18s | Total: 18m31s