Skip to content

Detect and report schema (DDL) changes - #14

Merged
emoss08 merged 1 commit into
claude/gtc-doctorfrom
claude/gtc-ddl
Aug 12, 2026
Merged

emoss08 merged 1 commit into
claude/gtc-doctorfrom
claude/gtc-ddl

Conversation

@emoss08

@emoss08 emoss08 commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Stacked PR 2 of 3 — base is #13 (claude/gtc-doctor), so review only the top commit; GitHub retargets this to main once #13 merges. The stacking is for a clean diff, not a code dependency.

What does this PR do?

PostgreSQL's logical decoding has no ALTER TABLE event — which is exactly why Debezium needs a schema-history topic. But pgoutput re-describes a published table whenever its shape changed, immediately before that table's next row change. Diffing consecutive descriptions turns that into a first-class feature with no DDL triggers, no history topic, no superuser — it works on RDS.

I verified this behavior against a real PostgreSQL before building on it.

Detected, with the effect of the change rather than the statement text:

Detected Kind Breaking?
New column column_added no — additive
Column removed column_dropped yes
Column type altered column_type_changed yes
Primary key / identity index replaced key_changed yes
REPLICA IDENTITY altered replica_identity_changed yes
Table renamed or moved schema table_renamed yes

Renames are told apart from new tables by the relation OID, which survives them. Type OIDs resolve to names (int4 → int8), falling back to oid:<n> for user-defined types.

Reported through every channel the product already uses: WARN logs for breaking changes (so existing log alerting catches them), cdc_schema_changes_total{schema,table,kind}, a rolling history at GET /api/schema, a new dashboard Schema page, and — with REDIS_URL — a <prefix>:schema notification stream gated by CDC_SCHEMA_EVENTS.

Type of change

  • New feature / sink

Checklist

  • go test ./... passes (including -race)
  • gofmt / go vet are clean
  • Tests added or updated for changed behavior (diff cases incl. identical-relation and first-relation no-ops, recorder history/eviction/copy semantics; the integration test performs real ALTER TABLEs and asserts both the API and the notification stream)
  • README.md / CLAUDE.md updated (new "Schema changes (DDL)" section, CDC_SCHEMA_EVENTS, endpoint table, delivery-semantics note)
  • If this touches delivery semantics: yes, and it is explicit — see below

Notes for the reviewer

Schema notifications are deliberately best-effort, unlike row events. A reconnect re-describes tables with nothing to diff against, so a missed change genuinely cannot be re-derived on redelivery — at-least-once is not achievable here. Rather than imply otherwise, ports.SchemaObserver returns no error by design, a failed publish increments cdc_schema_publish_errors_total and logs, and row delivery's at-least-once guarantee is untouched.

Two further limitations follow from the mechanism and are documented in the README:

  • Detection is tied to the next row change, so a table altered at 02:00 and first written at 09:00 reports at 09:00.
  • Several statements landing between two row changes arrive as one combined change — the net effect, not a statement log.

Also: relation messages carry no WAL position of their own (the LSN came through as 0/0), so a change is stamped with the commit LSN of the transaction it arrived in, and documented as where it was detected rather than where the DDL committed.

🤖 Generated with Claude Code

https://claude.ai/code/session_019dEj24Ta2o9QxzW9ZrSPtb


Generated by Claude Code

PostgreSQL's logical decoding does not stream DDL, but pgoutput re-describes
a published table whenever its shape changed, right before that table's next
row change. Diffing consecutive relation descriptions turns that into a
first-class feature with no DDL triggers, no schema-history topic and no
superuser:

- Detects columns added, dropped and retyped, primary key / replica identity
  changes, and renames (the table OID survives a rename, which is how a
  rename is told apart from a new table). Type OIDs resolve to names.
- Classifies each change: adding a column is additive, everything else can
  break consumers holding documents for the table.
- Reports through every channel the product already uses — WARN logs for
  breaking changes, cdc_schema_changes_total{schema,table,kind}, a rolling
  history at GET /api/schema, a new dashboard page, and (with REDIS_URL) a
  "<prefix>:schema" notification stream gated by CDC_SCHEMA_EVENTS.

Schema notifications are deliberately best-effort, unlike row events: a
reconnect re-describes tables with nothing to diff against, so a change
cannot be re-derived on redelivery. ports.SchemaObserver therefore returns
no error and a failed publish is counted in cdc_schema_publish_errors_total
instead of stalling replication. Row delivery keeps at-least-once unchanged.

Relation messages carry no WAL position of their own, so a change is stamped
with the commit LSN of the transaction it arrived in, and documented as the
position where the change was detected rather than where the DDL committed.

The integration test now performs real DDL against PostgreSQL and asserts
both the API and the notification stream see it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dEj24Ta2o9QxzW9ZrSPtb
@emoss08 emoss08 mentioned this pull request Aug 12, 2026
6 tasks done
@emoss08
emoss08 merged commit 119a652 into main Aug 12, 2026
7 checks passed
@emoss08
emoss08 deleted the claude/gtc-ddl branch August 12, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants