fix(sync): op_checkpoints completed_keys double-encode blocks all sync#2309
Open
immy2good wants to merge 1 commit into
Open
fix(sync): op_checkpoints completed_keys double-encode blocks all sync#2309immy2good wants to merge 1 commit into
immy2good wants to merge 1 commit into
Conversation
garrytan#2305) recordCompleted bound JSON.stringify(keys) to a $3::jsonb param via conn.unsafe({prepare:false}); postgres.js v3 sends the JSON string as a text param, so $3::jsonb parses it into a jsonb STRING scalar ("[\"k\"]"), not an array. The op_checkpoints_completed_keys_array CHECK (jsonb_typeof = array) then rejects every sync-target checkpoint write -- blocking all sync/ingestion. Fix: bind the JS string[] as text[] and build the jsonb in SQL via to_jsonb($3::text[]) -- the same native binding appendCompleted already uses (works on Postgres + PGLite). Adds an e2e regression test (real Postgres; PGLite hides the bug). Closes garrytan#2305 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
recordCompleted(src/core/op-checkpoint.ts) blocks allgbrain syncon Postgres. It bindsJSON.stringify(keys)to a$3::jsonbparameter viaexecuteRawDirect→conn.unsafe(sql, params)with{ prepare: false }. postgres.js v3 sends the JSON string as a text param, so$3::jsonbparses it into a jsonb string scalar ("[\"k\"]"), not an array. Theop_checkpoints_completed_keys_arrayCHECK (jsonb_typeof = 'array') then rejects everysync-targetcheckpoint write:Result: 0 files imported,
last_commitnever advances; "safe to retry" but every retry hits the same. Fixes #2305.This is the parameterized twin of the
${JSON.stringify(x)}::jsonbdouble-encode thatscripts/check-jsonb-pattern.shguards against — the grep guard only catches the template form, so this slipped through. The in-code comment even claimed the parameterized form was safe ("NOT the double-encode trap"); it isn't, under.unsafe()+{ prepare: false }.Minimal repro:
Fix
Bind the JS
string[]as a Postgrestext[]and build the jsonb in SQL viato_jsonb($3::text[])— the same native bindingappendCompletedalready relies on (works on both Postgres and PGLite). Drops theJSON.stringify+$3::jsonb.Test
Adds an e2e regression test in
test/e2e/postgres-jsonb.test.tsassertingrecordCompletedstores a jsonb array (jsonb_typeof = 'array'). The bug only manifests on real Postgres (PGLite uses a different driver path and hides it), so the test lives in the Postgres e2e suite.Verified RED→GREEN against a real Postgres; existing PGLite op-checkpoint unit suite passes (30/0);
tsc --noEmitclean.