Skip to content

Prepare statements again after a schema change - #10

Merged
necessarylion merged 1 commit into
mainfrom
fix/stale-prepared-type
Nov 2, 2026
Merged

necessarylion merged 1 commit into
mainfrom
fix/stale-prepared-type

Conversation

@mydoxie

@mydoxie mydoxie commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Problem

A text param can be stored with the wrong type after a table is changed on the same pool.

Example on one connection: v is an int column. INSERT ... VALUES ($1) runs with '7'. Then the table is dropped and made again with v text. The same insert with '007' stores '7'. The same SELECT ... WHERE v = $1 then fails with 42883 operator does not exist: text = integer.

Cause

Strings are sent with type oid 0, so Postgres picks the param type at Parse time (here int4). The driver caches the statement. When the table changes, Postgres plans the statement again, but it keeps the old param types. No error comes for the insert, so an error retry can not catch it.

Fix

  • The pool has a schema counter. It goes up when a connection gets a CommandComplete of CREATE, ALTER or DROP.
  • The counter is part of the statement cache key. After a schema change, each connection prepares its statements again, so the param types come from the new tables.
  • Old statements stay in the cache until the LRU closes them. A query that still waits for its answer keeps its columns.
  • Normal queries: one relaxed atomic load and one more value in the hash. No extra round trip.
  • Not seen: DDL from another process, or inside DO or a function.

No FFI change, no version or ABI bump.

Tests

  • New test: "a table rebuilt with another column type keeps text as text". It failed before the fix (stored 7) and passes now.
  • cargo fmt, cargo clippy -D warnings, cargo test (new unit test for the DDL tag check), dart analyze, full dart test on Postgres 16: all pass.

A cached statement kept its old param types after a table was rebuilt,
so '007' sent to a new text column was stored as '7'. A pool-wide
schema counter is now part of the statement cache key.
@mydoxie

mydoxie commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Bot review

Verdict: OK. Nothing blocks.

Checks I ran (Postgres 16 in a throwaway container):

  • cargo fmt --check, cargo clippy --release --all-targets -D warnings, cargo test (12 pass): green.
  • dart format, dart analyze --fatal-infos, dart test (41 pass): green.
  • I put back the old native/src from main and ran the new Dart test. It fails. So the test catches the bug.

What I checked:

  • The counter lives on Pool, so all connections of the pool share it. Relaxed is fine. It is only a hint for the cache key, and nothing else is published through it.
  • The bump comes when the server sends CommandComplete. So another connection that prepares after that sees the new table. A connection that prepares a little before the bump gets its statement thrown away one more time. That costs one extra prepare and is still correct.
  • Old statements: the key changes, so they stay in the map until evict picks them. evict sends a Close to the server. The cache stays at CACHE_MAX. No leak.
  • Hot path: one atomic load and one more u64 in the hash. No lock, no allocation, no round trip. is_ddl runs on each CommandComplete, but it only compares a few bytes.
  • Transactions use the same encode path, so the fix works there too. A DDL that is rolled back still bumps the counter. That only costs one extra prepare.
  • No FFI change, no version bump, changelog is under ## Unreleased.

Nits (do not block):

  • Stmt.schema and the s.schema == schema check only help on a 64-bit hash collision, because the key already has the schema in it. It is cheap, so keeping it is fine.
  • An app that runs CREATE TEMP TABLE or DROP on every request bumps the counter each time. Then every connection prepares its statements again on each request. It may be worth one line in the docs.
  • CREATE INDEX (and also CREATE ROLE, CREATE FUNCTION and others) bumps the counter too. That is harmless: it only causes one extra prepare.

@necessarylion
necessarylion merged commit 667288c into main Nov 2, 2026
8 checks passed
@necessarylion
necessarylion deleted the fix/stale-prepared-type branch November 2, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants