Repository navigation
feat: queues, partition sets, fix hazards, data-checked --prove, stale-migration guard, schema files as a source - #146
Merged
Merged
Conversation
…does not
dbdiff now diffs range types, default privileges, event triggers, rules,
aggregates, extended statistics, text search configurations and
dictionaries, casts, foreign servers, user mappings and foreign tables,
and renames columns and tables rather than dropping them. Each is titled
by what it is ("Cast missing: flag AS boolean", "Column renamed:
public.people.name → full_name"); a changed one is "<Kind> modified",
named from the statement that makes it rather than the DROP that
replaces it, which read as the object going and was marked critical.
dbdiff's warnings — kinds it does not compare, columns or tables it
would not take for renames — are now findings too, with nothing to
apply, so they are seen before an apply rather than missed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pgmq makes each queue's tables and registers them as its extension's own, so the schema diff — which leaves an extension's objects to the extension — never saw a queue: one the target lacked went unreported. A queues check now compares pgmq.meta by name and kind, creating a missing queue with pgmq.create (or create_unlogged), installing pgmq first where the target lacks it, and dropping an extra one with pgmq.drop_queue — held back as destructive, since it drops the queue's messages. A partitioned queue, whose pg_partman settings pgmq.meta does not record, and a change of kind, are left as manual steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pg_partman's partitions are its own to make and drop, so the schema diff leaves them alone; what makes a partition set is its registration in part_config, written by create_parent(). A partitions check now compares those: a set the target lacks is registered through create_parent, with its settings (premake, retention and the rest) put as the source has them, and a setting that differs is updated where it stands. How a set is partitioned — control column, interval, type — cannot be changed in place, so a difference there, and a set only the target has, are left as manual steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fix can be correct and still take production down — an ALTER TABLE holds a lock that stops every read and write, a column type change may rewrite the table under it, a new CHECK scans every row — and a report showed a one-line fix and an hour-long outage the same way. Every fix now carries hazards, read from PostgreSQL's own parse of its SQL with the lock levels PostgreSQL documents: blocks reads and writes, blocks writes, may rewrite, scans every row (with how to avoid holding the lock: NOT VALID, then VALIDATE), cannot run in a transaction, and data loss. A table the same fix creates is not flagged. They show under each finding in the report and are in --json and over MCP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itten A schema can converge and the data not — a table recreated, a cast that loses values — and the proof's copy held no rows, so none of that showed. The copy now gets a sample of each table's rows first (the first 1000, SUPAFORGE_PROVE_ROWS to change it, 0 for structure only), piped from the target with psql's \copy, foreign keys set aside where the role may. After the fixes, a table they keep must hold as many rows, with the same values in each column whose type did not change — unless a fix deletes from or truncates it — or the proof fails, naming the table. A table whose sample would not load is listed as unchecked. A table whose storage the fixes replaced on the copy is named as rewritten: on the real target, a long lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as since changed A migration file runs later than it is written — after review, after a merge — against a target that may have moved on: a column added by hand, another migration run. Applied then, it could fail half way, or succeed and leave something wrong. migrate create now records the target environment and a digest of its structure in the file. migrate run checks the first pending migration written against the same environment before running anything: if the target has changed since, it says so and applies nothing, unless --force. The migrations after it follow from it, so they are not checked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In one transaction, every table, index and sequence a restore creates holds a lock until the commit, and the server keeps a fixed number of them. A large project's restore ran out half way, rolled back, and said only "out of shared memory". It now says why, and the two ways round it: raise max_locks_per_transaction, or --no-transaction to commit as it goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… target Declarative schemas keep the desired state as SQL files (Supabase's supabase/schemas). diff --source-dir loads them into a throwaway database on the target's server — on top of the target's platform schemas, so a reference to auth.users or a call to auth.uid() resolves, as --prove does — and compares that with the target: schema, RLS and extensions unless --check says otherwise. Files load in name order, and a statement needing something a later file creates is tried again once the rest have run. What the platform owns and the copy cannot hold (pg_cron lives in one database per server) is not reported as extra. The throwaway database is dropped afterwards, whatever happens. New config field ignoreExtensions: extensions not reported as extra. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| // so a sampled row need not find its parent sampled too; then the COPY, | ||
| // whose rows follow on the same stream until it ends. | ||
| writer.stdin.write(`\\set ON_ERROR_STOP 0\nSET session_replication_role = replica;\n\\set ON_ERROR_STOP 1\n${into}\n`) | ||
| reader.stdout.pipe(writer.stdin) |
| * loads. Verdicts are read from the target, and the throwaway database must | ||
| * be gone afterwards. | ||
| */ | ||
| import { describe, it, expect, beforeAll, afterAll } from 'vitest' |
The data half of a proof hashed every column of a sampled table, so a fix that changes a generated column's expression read as values lost, and the whole migration was refused. A generated column's values follow its expression; only the columns it is computed from are compared. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A destructive drop held back without --allow-destructive leaves its table on the target, and the proof accounted for the table but not for the sequence its serial or identity column owns, so the whole apply was refused. An unexpected sequence now names the column that owns it, and a held-back drop of that table or column accounts for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dbdiff's warnings about kinds it does not compare were findings of their own, so a database with, say, an operator class never read as converged: a re-diff after an apply still found one, and a CI gate on any finding never passed. They are now the check's notices — under its line in the summary, in the JSON beside its issues, and in the CI summary — neither counted nor scored. Publications are left out: the Realtime check compares them. The README names the kinds the schema check now covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One commit per change, each with its own tests.
queuescheck comparespgmq.meta. It creates throughpgmq.create/create_unlogged, installing pgmq if needed, and drops throughpgmq.drop_queue, which is held back as destructive (it drops the messages). Partitioned queues and a change of kind are manual. Before, a missing queue went unreported: its tables are pgmq's own, so the schema diff leaves them alone.partitionscheck comparespart_configregistrations. It registers throughcreate_parentwith the source's settings, and updates differing settings in place. A changed control column, interval or type, and target-only sets, are manual.--jsonand over MCP.--provechecks the rowsSUPAFORGE_PROVE_ROWS(1000) rows of each table, piped with\copy. A kept table that loses rows, or whose values change in a column whose type didn't, fails the proof. Deletes and truncates in the fixes are expected. Tables the fixes rewrite on the copy are named.migrate createrecords the target environment and a digest of its structure.migrate runrefuses the first pending migration written against that environment if the target has changed since, unless--force.max_locks_per_transactionor--no-transactionas the ways round it.diff --source-dirsupabase/schemas) with the target. The files are loaded into a throwaway database on the target's server, on top of its platform schemas, with out-of-order statements retried. What the copy can't hold (pg_cron) isn't reported as extra. The throwaway database is always dropped.--proveand generated columns--proveand held-back dropsLive-tested against Supabase and plain PostgreSQL:
--prove: a 2,500-row retype proved with rows kept and the rewrite named.--forceand when unchanged.--source-dir: loaded out-of-order files, converged, and dropped the throwaway database.--provewith a generated column, queues and--source-dir.🤖 Generated with Claude Code