Skip to content

fix(sync): op_checkpoints completed_keys double-encode blocks all sync#2309

Open
immy2good wants to merge 1 commit into
garrytan:masterfrom
immy2good:fix/op-checkpoints-jsonb-double-encode
Open

fix(sync): op_checkpoints completed_keys double-encode blocks all sync#2309
immy2good wants to merge 1 commit into
garrytan:masterfrom
immy2good:fix/op-checkpoints-jsonb-double-encode

Conversation

@immy2good

Copy link
Copy Markdown

Problem

recordCompleted (src/core/op-checkpoint.ts) blocks all gbrain sync on Postgres. It binds JSON.stringify(keys) to a $3::jsonb parameter via executeRawDirectconn.unsafe(sql, params) with { prepare: false }. postgres.js v3 sends the JSON string as a text param, so $3::jsonb parses it into a jsonb string scalar ("[\"k\"]"), not an array. The op_checkpoints_completed_keys_array CHECK (jsonb_typeof = 'array') then rejects every sync-target checkpoint write:

[op-checkpoint] write failed (sync-target, <fp>): new row for relation "op_checkpoints"
  violates check constraint "op_checkpoints_completed_keys_array"
[sync] checkpoint target write failed — aborting before import; nothing drained.
Sync PARTIAL: imported 0 of N file(s), reason=checkpoint_unavailable.

Result: 0 files imported, last_commit never advances; "safe to retry" but every retry hits the same. Fixes #2305.

This is the parameterized twin of the ${JSON.stringify(x)}::jsonb double-encode that scripts/check-jsonb-pattern.sh guards against — the grep guard only catches the template form, so this slipped through. The in-code comment even claimed the parameterized form was safe ("NOT the double-encode trap"); it isn't, under .unsafe() + { prepare: false }.

Minimal repro:

const sql = postgres(DATABASE_URL, { prepare: false });
await sql.unsafe("select jsonb_typeof($1::jsonb) as t", [JSON.stringify(["x"])]);
// => [{ t: "string" }]   // jsonb STRING scalar, not an array

Fix

Bind the JS string[] as a Postgres text[] and build the jsonb in SQL via to_jsonb($3::text[]) — the same native binding appendCompleted already relies on (works on both Postgres and PGLite). Drops the JSON.stringify + $3::jsonb.

Test

Adds an e2e regression test in test/e2e/postgres-jsonb.test.ts asserting recordCompleted stores a jsonb array (jsonb_typeof = 'array'). The bug only manifests on real Postgres (PGLite uses a different driver path and hides it), so the test lives in the Postgres e2e suite.

Verified RED→GREEN against a real Postgres; existing PGLite op-checkpoint unit suite passes (30/0); tsc --noEmit clean.

garrytan#2305)

recordCompleted bound JSON.stringify(keys) to a $3::jsonb param via conn.unsafe({prepare:false}); postgres.js v3 sends the JSON string as a text param, so $3::jsonb parses it into a jsonb STRING scalar ("[\"k\"]"), not an array. The op_checkpoints_completed_keys_array CHECK (jsonb_typeof = array) then rejects every sync-target checkpoint write -- blocking all sync/ingestion.

Fix: bind the JS string[] as text[] and build the jsonb in SQL via to_jsonb($3::text[]) -- the same native binding appendCompleted already uses (works on Postgres + PGLite). Adds an e2e regression test (real Postgres; PGLite hides the bug).

Closes garrytan#2305

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sync always fails: recordCompleted writes completed_keys as a jsonb string (double-encode), violating op_checkpoints_completed_keys_array

1 participant