From a171b53dac940181b07375d04306da2136ab2928 Mon Sep 17 00:00:00 2001 From: Jeffery Lofoneh Asamani Date: Fri, 18 Sep 2026 22:51:28 +0000 Subject: [PATCH 1/2] =?UTF-8?q?feat(findings):=20collation=5Fversion=5Fmis?= =?UTF-8?q?match=20=E2=80=94=20the=20collation=20library=20changed=20under?= =?UTF-8?q?=20the=20data?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The version of libc/ICU that defines text sort order is recorded in the catalog (pg_database.datcollversion, pg_collation.collversion, PG15+). When the library changes underneath — an OS upgrade, a new base image, a restore onto another host — every btree over text sorted by it may be silently out of order: lookups miss rows and UNIQUE stops catching duplicates. Postgres warns at connect time and repairs nothing; nothing in pgbot read the versions. A gauge collector reads this database's mismatches (NULL versions — C/POSIX — cannot drift and are excluded). The finding is critical when the database default drifted, since every unqualified text index uses it, and warn for a named collation. The remediation is REINDEX then REFRESH COLLATION VERSION, and the caveat says why the order matters: refreshing first silences the warning and leaves the indexes corrupt. The integration test forges a stale datcollversion as superuser, runs the real collector, checks the row and the finding, and confirms REFRESH clears it. New `collation` section in --json; SchemaVersion 1.3.0 (additive). --- CHANGELOG.md | 12 +- README.md | 9 +- docs/findings/README.md | 1 + docs/findings/collation_version_mismatch.md | 125 ++ internal/collect/collation.go | 51 + .../collect/collation_integration_test.go | 103 + internal/collect/collector.go | 1 + internal/collect/sql/collation.sql | 30 + internal/findings/catalog.go | 7 + internal/findings/catalog_test.go | 6 + internal/findings/collation_test.go | 50 + internal/findings/findings.go | 50 +- internal/model/context.go | 18 + internal/model/schema_version.go | 5 +- schema/pgbot-context-1.3.0.json | 1742 +++++++++++++++++ 15 files changed, 2203 insertions(+), 7 deletions(-) create mode 100644 docs/findings/collation_version_mismatch.md create mode 100644 internal/collect/collation.go create mode 100644 internal/collect/collation_integration_test.go create mode 100644 internal/collect/sql/collation.sql create mode 100644 internal/findings/collation_test.go create mode 100644 schema/pgbot-context-1.3.0.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 061c899..3416331 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ All notable changes to pgbot are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/), and the project aims for [Semantic Versioning](https://semver.org/). The `--json` contract is versioned -separately by `model.SchemaVersion` (currently 1.2.0). +separately by `model.SchemaVersion` (currently 1.3.0). ## [Unreleased] @@ -21,6 +21,16 @@ separately by `model.SchemaVersion` (currently 1.2.0). `pgbot ask "why is it slow?"`. ### Added +- **`collation_version_mismatch` finding** (PG15+). The collation library + (libc or ICU) that defines text sort order changed version under the data — + an OS upgrade, a new base image, a restore onto a different host — so every + btree over text sorted by it may be silently out of order: lookups miss rows + and `UNIQUE` stops catching duplicates. Read from `pg_database.datcollversion` + and `pg_collation.collversion` against the library's actual version; critical + when it is the database default, warn for a named collation. The remediation + is REINDEX **then** `REFRESH COLLATION VERSION`, in that order — the caveat + says why. New `collation` section in `--json`; `SchemaVersion` → **1.3.0** + (additive; a 1.2.0 consumer parses it unchanged). - **`$PGSERVICE` as a connection fallback** (#25). When no connection string is passed and neither `$DATABASE_URL` nor `$PGBOT_DATABASE_URL` is set, pgbot now checks `$PGSERVICE` too, so a diff --git a/README.md b/README.md index 81f9de0..cf8a649 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Provider notes

-> **Status: beta.** The `--json` contract is versioned (currently `1.2.0`, JSON +> **Status: beta.** The `--json` contract is versioned (currently `1.3.0`, JSON > Schema published in [`schema/`](schema/)) and breaking changes to it are > treated as breaking changes to the tool. The human-readable report is **not** > a stable interface — parse `--json`, not the terminal output. @@ -816,20 +816,21 @@ All from SQL — connections, cache-hit ratio, TPS and rollback ratio, WAL and I rates, checkpoints, locks and blocking chains, replication lag, replication-slot WAL retention and logical-subscription health, top queries (`pg_stat_statements`), table/index sizes, dead tuples and vacuum activity, -unused and missing indexes, and non-default settings. Counters +unused and missing indexes, non-default settings, and collation version drift +(PG15+). Counters (`pg_stat_database`, `pg_stat_wal`, IO) are **double-sampled** to produce live rates; the rest are point-in-time reads trended against the baseline. ## The `--json` contract `--json` (and `--format=json`) is the interface to build on — a versioned, -PII-free document (`schema_version`, currently `1.2.0`) whose machine-checkable +PII-free document (`schema_version`, currently `1.3.0`) whose machine-checkable JSON Schema is published in [`schema/`](schema/). Every section carries an `exactness` label — `sampled`, `cumulative`, `scraped`, or `unavailable` — so a consumer never mistakes a cumulative total for a live rate. Versioning policy: additive fields bump the minor version and are not breaking — -a `1.1.0` consumer parses `1.2.0` output unchanged; breaking changes to the +a `1.2.0` consumer parses `1.3.0` output unchanged; breaking changes to the contract are treated as breaking changes to the tool. `pgbot advise --json` has its own schema ([`schema/pgbot-advise-1.0.0.json`](schema/pgbot-advise-1.0.0.json)). diff --git a/docs/findings/README.md b/docs/findings/README.md index dc2aa7d..cbf1e69 100644 --- a/docs/findings/README.md +++ b/docs/findings/README.md @@ -23,6 +23,7 @@ Lost durability, corruption, wraparound, replication — things that end in an o - **[index_invalid](index_invalid.md)** · Critical — a failed CREATE INDEX CONCURRENTLY left an invalid index — critical if it's still maintained on writes, warn if it's failed-build debris - **[sync_rep_degraded](sync_rep_degraded.md)** · Critical — fewer synchronous standbys connected than the config requires - **[archiving_disabled](archiving_disabled.md)** · Warn — archive_mode is off — no continuous WAL archive for PITR +- **[collation_version_mismatch](collation_version_mismatch.md)** · Warn — the collation library changed version under the data — text indexes may be silently out of order - **[connection_saturation](connection_saturation.md)** · Warn — connections approaching max_connections - **[idle_in_transaction](idle_in_transaction.md)** · Warn — sessions idle inside an open transaction, holding locks and the xmin horizon - **[int4_identity_column](int4_identity_column.md)** · Warn — a sequence-backed int2/int4 column that will wrap (int4 at 2.1B) regardless of current value diff --git a/docs/findings/collation_version_mismatch.md b/docs/findings/collation_version_mismatch.md new file mode 100644 index 0000000..f334078 --- /dev/null +++ b/docs/findings/collation_version_mismatch.md @@ -0,0 +1,125 @@ +--- +id: collation_version_mismatch +severity: warn +critical_when: "the database's default collation is the one that changed" +dimension: risk +object: db +scope: infra +requires: [PG15+] +thresholds: [] +related: [] +--- + +# collation_version_mismatch + +**Severity:** warn (critical when the database's default collation is the one that changed) · **Dimension:** risk · **Object identity:** `db:` (see [configuration](../configuration.md)) · **Requires:** PostgreSQL 15+ + +## What pgbot observed + +The collation version the catalog recorded no longer matches what the server's +collation library reports now — for the database default +(`pg_database.datcollversion` vs `pg_database_collation_actual_version()`) or for +a named collation (`pg_collation.collversion` vs `pg_collation_actual_version()`). +Only this database is checked; collations whose provider records no version +(`C`, `POSIX`) cannot drift and are never reported. + +Critical when the **database default** drifted, because every text index that +does not name a collation uses it. Warn when only named collations drifted. + +## Why it matters + +Text sort order is not defined by Postgres — it is defined by the OS's libc or +by ICU, and a btree index is only valid for the order that library produced when +the index was built. When the library changes underneath (an OS upgrade, a new +container base image, a restore onto a different host; glibc 2.28 changed the +order for most locales), the index is **silently out of order**: equality +lookups miss rows that exist, range scans skip them, and `UNIQUE` constraints +stop catching duplicates. Nothing errors. Postgres emits a `WARNING` at connect +time that nobody reads, and repairs nothing. + +## How to verify it yourself + +```sql +SELECT datname, + datcollversion AS recorded, + pg_database_collation_actual_version(oid) AS actual +FROM pg_database +WHERE datname = current_database(); +``` + +```sql +SELECT n.nspname || '.' || c.collname AS collation, + c.collversion AS recorded, + pg_collation_actual_version(c.oid) AS actual +FROM pg_collation c +JOIN pg_namespace n ON n.oid = c.collnamespace +WHERE c.collversion IS NOT NULL + AND c.collversion IS DISTINCT FROM pg_collation_actual_version(c.oid); +``` + +A fresh `psql` session to the database prints the same thing Postgres sees: +`WARNING: database "app" has a collation version mismatch`. + +## How to fix it + +Rebuild first, then tell Postgres the new version is the right one. In that order. + +1. **Reindex everything that sorts text with the affected collation.** For the + database default that is every btree over a `text`/`varchar`/`char` column + without an explicit `COLLATE`; the simple, safe answer is the whole database, + online: + + ```sql + REINDEX DATABASE CONCURRENTLY app; + ``` + + For a named collation, reindex the indexes whose columns use it. If you would + rather check than rebuild, `amcheck` can verify btree order first + (`CREATE EXTENSION amcheck; SELECT bt_index_check('index_name', true);`) — an + index that passes is fine, one that fails must be rebuilt. + +2. **Record the new version** so the warning stops and pgbot clears the finding: + + ```sql + ALTER DATABASE app REFRESH COLLATION VERSION; + -- or, for a named collation: + ALTER COLLATION public.de_phonebook REFRESH VERSION; + ``` + +Refreshing *before* reindexing only updates the catalog: the warning goes away +and the indexes stay corrupt. On a managed provider, check whether the provider +handled this as part of a major-version upgrade before doing it yourself; it is +still your indexes. + +## When to ignore it + +Only once the reindex is done and the refresh is scheduled, or when you have +verified with `amcheck` that every affected index is in order. A suppressed +critical still renders in the report; it only drops out of the exit code. + +```toml +[[ignore]] +finding = "collation_version_mismatch" +object = "db:app" +reason = "reindexed after the glibc upgrade on 2026-09-10; REFRESH COLLATION VERSION in the next window" +expires = "2026-10-01" +``` + +## What pgbot cannot see + +- Whether the sort order **actually changed** between the two versions for your + locale — Postgres records versions, not orderings. A mismatch is a "may be + corrupt", not a "is corrupt"; the only proof either way is `amcheck` or a + rebuild. +- Which indexes use the affected collation. It reports the collation; mapping it + to indexes is the reindex step. +- Other databases in the cluster. Each records its own `datcollversion`; run + `--all-databases` to check them all. +- PostgreSQL 14 and older, which do not record a database-level version. + +## Related + +- [checksum_failures](checksum_failures.md) — the other silent-corruption signal, + from the storage side rather than the collation library. +- [index_invalid](index_invalid.md) — an index Postgres already knows is unusable; + a collation mismatch is one it still trusts. diff --git a/internal/collect/collation.go b/internal/collect/collation.go new file mode 100644 index 0000000..c716306 --- /dev/null +++ b/internal/collect/collation.go @@ -0,0 +1,51 @@ +package collect + +import ( + "context" + _ "embed" + "time" + + "github.com/pgrundev/pgbot/internal/conn" + "github.com/pgrundev/pgbot/internal/model" +) + +//go:embed sql/collation.sql +var sqlCollation string + +// collation = catalog objects whose recorded collation version no longer matches +// the library the server runs against. PG15+, when pg_database began recording +// datcollversion. Empty list = healthy. +type collationCollector struct{} + +type collationRow struct { + Kind string `db:"kind"` + Name string `db:"name"` + Provider string `db:"provider"` + Recorded string `db:"recorded"` + Actual string `db:"actual"` +} + +func (collationCollector) Name() string { return "collation" } +func (collationCollector) Kind() Kind { return KindGauge } +func (collationCollector) Available(caps conn.Capabilities) bool { + return caps.VersionNum >= 150000 // pg_database.datcollversion + pg_database_collation_actual_version() +} + +func (collationCollector) Sample(ctx context.Context, t *conn.Target, _ conn.Capabilities) (any, error) { + return queryMany[collationRow](ctx, t, sqlCollation) +} + +func (collationCollector) Assemble(c *model.Context, _ conn.Capabilities, s sampled, _ time.Duration, _ Options) { + rows, ok := s.A.([]collationRow) + if s.Err != nil || !ok { + c.Collation = &model.Collation{Section: unavail(s.Err, "collation versions need PostgreSQL 15+")} + return + } + col := &model.Collation{Section: model.Section{Exactness: model.ExactnessScraped}} + for _, r := range rows { + col.Mismatches = append(col.Mismatches, model.CollationMismatch{ + Kind: r.Kind, Name: r.Name, Provider: r.Provider, Recorded: r.Recorded, Actual: r.Actual, + }) + } + c.Collation = col +} diff --git a/internal/collect/collation_integration_test.go b/internal/collect/collation_integration_test.go new file mode 100644 index 0000000..9bc3774 --- /dev/null +++ b/internal/collect/collation_integration_test.go @@ -0,0 +1,103 @@ +package collect_test + +import ( + "context" + "os" + "testing" + "time" + + "github.com/jackc/pgx/v5" + "github.com/pgrundev/pgbot/internal/collect" + "github.com/pgrundev/pgbot/internal/conn" + "github.com/pgrundev/pgbot/internal/findings" + "github.com/pgrundev/pgbot/internal/model" +) + +// A collation version mismatch can't be produced by upgrading glibc inside a +// test, but the catalog state it leaves behind can: pg_database.datcollversion +// is what Postgres compares against the library at connect time, and a superuser +// can set it directly. Forge a stale version, run the real collector, check the +// collected row and the finding, then confirm ALTER DATABASE … REFRESH COLLATION +// VERSION — the step the remediation ends with — clears it. +func TestIntegration_collationVersionMismatch(t *testing.T) { + su := os.Getenv("PGBOT_TEST_SUPERUSER_DSN") + if su == "" { + t.Skip("set PGBOT_TEST_SUPERUSER_DSN (a superuser DSN) to run the collation fixture") + } + ro := dsn(t) + ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) + defer cancel() + admin, err := pgx.Connect(ctx, su) + if err != nil { + t.Fatalf("admin connect: %v", err) + } + t.Cleanup(func() { admin.Close(context.Background()) }) + + var vnum int + var db string + var recorded *string + if err := admin.QueryRow(ctx, `SELECT current_setting('server_version_num')::int, datname, datcollversion + FROM pg_database WHERE datname = current_database()`).Scan(&vnum, &db, &recorded); err != nil { + t.Fatal(err) + } + if vnum < 150000 { + t.Skip("pg_database.datcollversion is PG15+") + } + if recorded == nil { + t.Skip("this database's collation records no version (C/POSIX) — nothing can drift") + } + refresh := `ALTER DATABASE ` + pgx.Identifier{db}.Sanitize() + ` REFRESH COLLATION VERSION` + if _, err := admin.Exec(ctx, `UPDATE pg_database SET datcollversion = '0.0-pgbot-test' WHERE datname = current_database()`); err != nil { + t.Fatalf("forge a stale datcollversion: %v", err) + } + t.Cleanup(func() { _, _ = admin.Exec(context.Background(), refresh) }) + + target, err := conn.Connect(ctx, ro) + if err != nil { + t.Fatalf("connect: %v", err) + } + defer target.Close() + run := func() *model.Context { + c, err := collect.Run(ctx, target, collect.Options{Interval: 200 * time.Millisecond, ASHHz: 0}) + if err != nil { + t.Fatalf("run: %v", err) + } + if c.Collation == nil || c.Collation.Exactness != model.ExactnessScraped { + t.Fatalf("collation section must be collected on PG15+, got %+v", c.Collation) + } + return c + } + + c := run() + var row *model.CollationMismatch + for i := range c.Collation.Mismatches { + if c.Collation.Mismatches[i].Kind == "database" { + row = &c.Collation.Mismatches[i] + } + } + if row == nil { + t.Fatalf("the forged database mismatch must be collected, got %+v", c.Collation.Mismatches) + } + if row.Name != db || row.Recorded != "0.0-pgbot-test" || row.Actual == "" || row.Actual == row.Recorded { + t.Fatalf("collected row must mirror the catalog: %+v", *row) + } + var f *model.Finding + for _, x := range findings.Compute(c) { + if x.ID == "collation_version_mismatch" { + f = &x + break + } + } + if f == nil || f.Severity != model.SeverityCritical || f.Object != "db:"+db { + t.Fatalf("a drifted database default must fire critical on db:%s, got %+v", db, f) + } + + if _, err := admin.Exec(ctx, refresh); err != nil { + t.Fatalf("refresh: %v", err) + } + for _, m := range run().Collation.Mismatches { + if m.Kind == "database" { + t.Fatalf("the database mismatch must clear after REFRESH COLLATION VERSION, got %+v", m) + } + } +} diff --git a/internal/collect/collector.go b/internal/collect/collector.go index 44ea663..843243f 100644 --- a/internal/collect/collector.go +++ b/internal/collect/collector.go @@ -104,6 +104,7 @@ var registry = []Collector{ progressCollector{}, archiverCollector{}, checksumsCollector{}, + collationCollector{}, standbyCollector{}, } diff --git a/internal/collect/sql/collation.sql b/internal/collect/sql/collation.sql new file mode 100644 index 0000000..c34eb90 --- /dev/null +++ b/internal/collect/sql/collation.sql @@ -0,0 +1,30 @@ +-- Collation version drift (PG15+): the version of the collation library the +-- catalog recorded when this database (datcollversion) or a collation object +-- (collversion) was created, against what the running server's libc/ICU reports +-- now. A difference means the library that defines text sort order changed under +-- the data — every btree over text sorted by it may be silently out of order +-- until REINDEXed. Scoped to current_database(): pg_collation is per-database +-- and the database row is this one's. NULL versions (C/POSIX, or a provider that +-- reports none) cannot drift and are excluded. +SELECT 'database' AS kind, + d.datname AS name, + CASE d.datlocprovider WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' WHEN 'b' THEN 'builtin' + ELSE d.datlocprovider::text END AS provider, + d.datcollversion AS recorded, + coalesce(pg_database_collation_actual_version(d.oid), '') AS actual +FROM pg_database d +WHERE d.datname = current_database() + AND d.datcollversion IS NOT NULL + AND d.datcollversion IS DISTINCT FROM pg_database_collation_actual_version(d.oid) +UNION ALL +SELECT 'collation', + n.nspname || '.' || c.collname, + CASE c.collprovider WHEN 'c' THEN 'libc' WHEN 'i' THEN 'icu' WHEN 'b' THEN 'builtin' + ELSE c.collprovider::text END, + c.collversion, + coalesce(pg_collation_actual_version(c.oid), '') +FROM pg_collation c +JOIN pg_namespace n ON n.oid = c.collnamespace +WHERE c.collversion IS NOT NULL + AND c.collversion IS DISTINCT FROM pg_collation_actual_version(c.oid) +ORDER BY 1, 2; diff --git a/internal/findings/catalog.go b/internal/findings/catalog.go index 97a1f46..c5c66ed 100644 --- a/internal/findings/catalog.go +++ b/internal/findings/catalog.go @@ -305,6 +305,12 @@ var catalog = map[string]Meta{ Requires: []string{"PG12+"}, Related: []string{"checksum_failures"}, }, + "collation_version_mismatch": { + Severity: "warn", CriticalWhen: "the database's default collation is the one that changed", + Dimension: "risk", ObjectClass: "db", + Scope: "infra", + Requires: []string{"PG15+"}, + }, "pgaudit_silent": { Severity: "warn", CriticalWhen: "", Dimension: "risk", ObjectClass: "setting", @@ -552,6 +558,7 @@ var summaries = map[string]string{ "checksum_failures": "Postgres read a page whose checksum didn't match — likely corruption", "ignore_checksum_failure_on": "ignore_checksum_failure is on — corrupt pages are returned, not caught", "checksums_disabled": "data checksums are off, so this class of corruption is silent", + "collation_version_mismatch": "the collation library changed version under the data — text indexes may be silently out of order", "pgaudit_silent": "pgaudit is installed but pgaudit.log selects no classes — the audit trail does not exist", "pgaudit_logs_parameters": "pgaudit.log_parameter=on writes bind parameters (passwords, PII) into the server log", "pgaudit_double_logging": "pgaudit and log_statement=all record every statement twice — duplicate log volume", diff --git a/internal/findings/catalog_test.go b/internal/findings/catalog_test.go index fa8a73d..5f49021 100644 --- a/internal/findings/catalog_test.go +++ b/internal/findings/catalog_test.go @@ -28,6 +28,12 @@ func TestCatalog_matchesEmitted(t *testing.T) { "checksum_failures": { Checksums: &model.Checksums{Failures: []model.ChecksumFailure{{Database: "app", Count: 3}}}, }, + "collation_version_mismatch": { + Server: model.ServerInfo{Database: "app"}, + Collation: &model.Collation{Mismatches: []model.CollationMismatch{ + {Kind: "database", Name: "app", Provider: "libc", Recorded: "2.31", Actual: "2.36"}, + }}, + }, "pgaudit_silent": { Server: model.ServerInfo{Extensions: []string{"pgaudit"}}, Settings: &model.Settings{Params: map[string]string{"pgaudit.log": "none"}}, diff --git a/internal/findings/collation_test.go b/internal/findings/collation_test.go new file mode 100644 index 0000000..b6972e2 --- /dev/null +++ b/internal/findings/collation_test.go @@ -0,0 +1,50 @@ +package findings + +import ( + "strings" + "testing" + + "github.com/pgrundev/pgbot/internal/model" +) + +func TestCollationVersionMismatch(t *testing.T) { + dbDrift := &model.Context{ + Server: model.ServerInfo{Database: "app"}, + Collation: &model.Collation{Mismatches: []model.CollationMismatch{ + {Kind: "database", Name: "app", Provider: "libc", Recorded: "2.31", Actual: "2.36"}, + }}, + } + f := has(Compute(dbDrift), "collation_version_mismatch") + if f == nil || f.Severity != model.SeverityCritical { + t.Fatalf("a drifted database default must fire critical, got %+v", f) + } + if f.Object != "db:app" { + t.Errorf("object must be the database, for suppression keying; got %q", f.Object) + } + if len(f.Caveats) == 0 || !strings.Contains(f.Caveats[0], "AFTER reindexing") { + t.Errorf("must carry the reindex-before-refresh caveat: %v", f.Caveats) + } + if len(f.Evidence) != 1 || !strings.Contains(f.Evidence[0], "2.31") || !strings.Contains(f.Evidence[0], "2.36") { + t.Errorf("evidence must name both versions: %v", f.Evidence) + } + + named := &model.Context{ + Server: model.ServerInfo{Database: "app"}, + Collation: &model.Collation{Mismatches: []model.CollationMismatch{ + {Kind: "collation", Name: "public.de_phonebook", Provider: "icu", Recorded: "153.14", Actual: "153.120"}, + }}, + } + if f := has(Compute(named), "collation_version_mismatch"); f == nil || f.Severity != model.SeverityWarn { + t.Errorf("a named collation drifting must fire warn, got %+v", f) + } + + for _, healthy := range []*model.Context{ + {Collation: &model.Collation{}}, + {Collation: &model.Collation{Section: model.Section{Exactness: model.ExactnessUnavailable}}}, + {}, + } { + if has(Compute(healthy), "collation_version_mismatch") != nil { + t.Errorf("no mismatches must not fire: %+v", healthy.Collation) + } + } +} diff --git a/internal/findings/findings.go b/internal/findings/findings.go index f8e7ee9..c13b914 100644 --- a/internal/findings/findings.go +++ b/internal/findings/findings.go @@ -126,7 +126,8 @@ var knownIDs = map[string]bool{ "sync_rep_degraded": true, "replica_lag_time": true, "recovery_conflicts": true, "replica_disconnected": true, "checksum_failures": true, "ignore_checksum_failure_on": true, "checksums_disabled": true, - "archiving_failing": true, "archiving_stalled": true, "archiving_disabled": true, + "collation_version_mismatch": true, + "archiving_failing": true, "archiving_stalled": true, "archiving_disabled": true, "replication_slot_inactive": true, "subscription_worker_down": true, "query_slowdown": true, "pgss_entries_evicted": true, "work_mem_low": true, "checkpoints_forced": true, "connections_overprovisioned": true, "fsync_off": true, @@ -194,6 +195,7 @@ func ComputeWithTunables(c *model.Context, tun Tunables) []model.Finding { int4IdentityColumn(c, add) walArchiving(c, add) checksumFindings(c, add) + collationVersionMismatch(c, add) failoverReadiness(c, add, tun) replicationSlotRisk(c, add) subscriptionDown(c, add) @@ -1590,6 +1592,52 @@ func checksumFindings(c *model.Context, add func(model.Finding)) { } } +// collationVersionMismatch flags catalog objects whose recorded collation version +// no longer matches the running library: libc or ICU changed under the data (an +// OS upgrade, a new base image, a restore onto a different host), so any btree +// over text sorted by that collation may be silently out of order — lookups miss +// rows and UNIQUE constraints stop catching duplicates. Critical when it is the +// database default (every unqualified text index); warn for a named collation. +func collationVersionMismatch(c *model.Context, add func(model.Finding)) { + if c.Collation == nil || len(c.Collation.Mismatches) == 0 { + return + } + var ev []string + dbDefault := false + for _, m := range c.Collation.Mismatches { + actual := m.Actual + if actual == "" { + actual = "unknown" + } + if m.Kind == "database" { + dbDefault = true + ev = append(ev, fmt.Sprintf("database default (%s): recorded %s, library now %s", m.Provider, m.Recorded, actual)) + continue + } + ev = append(ev, fmt.Sprintf("collation %s (%s): recorded %s, library now %s", m.Name, m.Provider, m.Recorded, actual)) + } + n := len(c.Collation.Mismatches) + sev, score := model.SeverityWarn, 60.0 + title := fmt.Sprintf("%d collation(s) changed version under this database", n) + if dbDefault { + sev, score = model.SeverityCritical, 88 + title = "the database's default collation changed version — text indexes may be corrupt" + } + add(model.Finding{ + ID: "collation_version_mismatch", Object: "db:" + c.Server.Database, Severity: sev, + Title: title, + Detail: "The library that defines this collation's sort order (libc or ICU) is a different version from the one the catalog recorded when the collation was created. If the order changed — glibc 2.28 did for most locales — every btree index over text using it is silently out of order: equality lookups miss rows, range scans skip them, and UNIQUE constraints stop catching duplicates. Postgres warns at connect time but repairs nothing.", + Evidence: ev, + Remediation: "REINDEX every index over text columns using the affected collation (REINDEX DATABASE CONCURRENTLY for the default), then record the new version with ALTER DATABASE … REFRESH COLLATION VERSION or ALTER COLLATION … REFRESH VERSION so the warning stops.", + Caveats: []string{ + "Refresh the version only AFTER reindexing — REFRESH COLLATION VERSION updates the catalog and silences the warning; it repairs nothing.", + "A version change does not prove the sort order changed for your locale, but the only way to know is to reindex; treat the indexes as suspect until then.", + }, + Impact: impact(model.DimRisk, score, fmt.Sprintf("%d collation version mismatch(es)", n), "datcollversion/collversion ≠ the library's actual version"), + Confidence: 1.0, + }) +} + // walArchiving flags the WAL-archiving / PITR failure modes. On a detected // managed provider the backup mechanism is usually outside archive_command, so // every archiving finding is downgraded to info with wording that says pgbot diff --git a/internal/model/context.go b/internal/model/context.go index 32d76a5..27897f4 100644 --- a/internal/model/context.go +++ b/internal/model/context.go @@ -56,6 +56,7 @@ type Context struct { Archiver *Archiver `json:"archiver,omitempty"` // WAL archiving health (A15) Checksums *Checksums `json:"checksums,omitempty"` // data-checksum failures cluster-wide (A16) Standby *StandbyStatus `json:"standby,omitempty"` // standby-side recovery conflicts (A17) + Collation *Collation `json:"collation,omitempty"` // collation version drift (PG15+) Deltas *Deltas `json:"deltas,omitempty"` // vs baseline; nil on first run // Set (with Deltas nil) when a stats reset / restart between runs makes any // comparison fiction — e.g. serverless scale-to-zero. See T2. @@ -460,6 +461,23 @@ type ChecksumFailure struct { LastFailure *time.Time `json:"last_failure,omitempty"` } +// Collation lists this database's catalog objects whose recorded collation +// version no longer matches what the server's libc/ICU reports now (PG15+): the +// database default (pg_database.datcollversion) and per-collation entries +// (pg_collation.collversion). Empty = healthy. +type Collation struct { + Section + Mismatches []CollationMismatch `json:"mismatches,omitempty"` +} + +type CollationMismatch struct { + Kind string `json:"kind"` // database | collation + Name string `json:"name"` // the database, or schema.collation + Provider string `json:"provider"` // libc | icu | builtin + Recorded string `json:"recorded_version"` + Actual string `json:"actual_version"` // "" when the library reports none +} + // Archiver is WAL archiving health from pg_stat_archiver. HasArchiveCommand is // only whether archive_command/library is set — never the value (credentials). type Archiver struct { diff --git a/internal/model/schema_version.go b/internal/model/schema_version.go index 8428fc2..f84d7af 100644 --- a/internal/model/schema_version.go +++ b/internal/model/schema_version.go @@ -10,4 +10,7 @@ package model // // 1.2.0: additive only — IndexStat gains columns/method/unique/primary (feeding // index/code correlation). A 1.1.0 consumer still parses 1.2.0 output. -const SchemaVersion = "1.2.0" +// +// 1.3.0: additive only — Context gains the `collation` section (collation version +// drift, PG15+). A 1.2.0 consumer still parses 1.3.0 output. +const SchemaVersion = "1.3.0" diff --git a/schema/pgbot-context-1.3.0.json b/schema/pgbot-context-1.3.0.json new file mode 100644 index 0000000..27663d2 --- /dev/null +++ b/schema/pgbot-context-1.3.0.json @@ -0,0 +1,1742 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://pgbot.dev/schema/pgbot-context-1.3.0.json", + "$ref": "#/$defs/Context", + "$defs": { + "Activity": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "total": { + "type": "integer" + }, + "active": { + "type": "integer" + }, + "idle": { + "type": "integer" + }, + "idle_in_transaction": { + "type": "integer" + }, + "waiting": { + "type": "integer" + }, + "by_state": { + "additionalProperties": { + "type": "integer" + }, + "type": "object" + }, + "wait_events": { + "additionalProperties": { + "type": "integer" + }, + "type": "object" + }, + "longest_xact_sec": { + "type": "number" + }, + "longest_active_sec": { + "type": "number" + }, + "connections": { + "items": { + "$ref": "#/$defs/ConnGroup" + }, + "type": "array" + }, + "autovacuum_workers": { + "type": "integer" + }, + "autovacuum_max_age_sec": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "total", + "active", + "idle", + "idle_in_transaction", + "waiting", + "by_state", + "longest_xact_sec", + "longest_active_sec" + ] + }, + "Archiver": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "archived_count": { + "type": "integer" + }, + "last_archived_wal": { + "type": "string" + }, + "last_archived_time": { + "type": "string", + "format": "date-time" + }, + "failed_count": { + "type": "integer" + }, + "last_failed_wal": { + "type": "string" + }, + "last_failed_time": { + "type": "string", + "format": "date-time" + }, + "stats_reset": { + "type": "string", + "format": "date-time" + }, + "has_archive_command": { + "type": "boolean" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "archived_count", + "failed_count", + "has_archive_command" + ] + }, + "BlockingRow": { + "properties": { + "blocked_pid": { + "type": "integer" + }, + "blocking_pids": { + "items": { + "type": "integer" + }, + "type": "array" + }, + "wait_event": { + "type": "string" + }, + "wait_seconds": { + "type": "number" + }, + "blocked_query": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "blocked_pid", + "blocking_pids", + "wait_seconds", + "blocked_query" + ] + }, + "ChecksumFailure": { + "properties": { + "database": { + "type": "string" + }, + "count": { + "type": "integer" + }, + "last_failure": { + "type": "string", + "format": "date-time" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "database", + "count" + ] + }, + "Checksums": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "failures": { + "items": { + "$ref": "#/$defs/ChecksumFailure" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness" + ] + }, + "Collation": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "mismatches": { + "items": { + "$ref": "#/$defs/CollationMismatch" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness" + ] + }, + "CollationMismatch": { + "properties": { + "kind": { + "type": "string" + }, + "name": { + "type": "string" + }, + "provider": { + "type": "string" + }, + "recorded_version": { + "type": "string" + }, + "actual_version": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "kind", + "name", + "provider", + "recorded_version", + "actual_version" + ] + }, + "ConnGroup": { + "properties": { + "app_name": { + "type": "string" + }, + "user": { + "type": "string" + }, + "state": { + "type": "string" + }, + "count": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "app_name", + "user", + "state", + "count" + ] + }, + "Context": { + "properties": { + "schema_version": { + "type": "string" + }, + "profile": { + "type": "string" + }, + "collected_at": { + "type": "string", + "format": "date-time" + }, + "fingerprint": { + "type": "string" + }, + "server": { + "$ref": "#/$defs/ServerInfo" + }, + "window": { + "$ref": "#/$defs/Window" + }, + "health": { + "$ref": "#/$defs/Health" + }, + "activity": { + "$ref": "#/$defs/Activity" + }, + "locks": { + "$ref": "#/$defs/Locks" + }, + "queries": { + "$ref": "#/$defs/Queries" + }, + "tables": { + "$ref": "#/$defs/Tables" + }, + "indexes": { + "$ref": "#/$defs/Indexes" + }, + "wal": { + "$ref": "#/$defs/WAL" + }, + "io": { + "$ref": "#/$defs/IO" + }, + "replication": { + "$ref": "#/$defs/Replication" + }, + "settings": { + "$ref": "#/$defs/Settings" + }, + "limits": { + "$ref": "#/$defs/Limits" + }, + "horizon": { + "$ref": "#/$defs/VacuumHorizon" + }, + "sequences": { + "$ref": "#/$defs/Sequences" + }, + "progress": { + "$ref": "#/$defs/Progress" + }, + "archiver": { + "$ref": "#/$defs/Archiver" + }, + "checksums": { + "$ref": "#/$defs/Checksums" + }, + "standby": { + "$ref": "#/$defs/StandbyStatus" + }, + "collation": { + "$ref": "#/$defs/Collation" + }, + "deltas": { + "$ref": "#/$defs/Deltas" + }, + "delta_suppressed_reason": { + "type": "string" + }, + "events": { + "items": { + "$ref": "#/$defs/Event" + }, + "type": "array" + }, + "wait_profile": { + "$ref": "#/$defs/WaitProfile" + }, + "findings": { + "items": { + "$ref": "#/$defs/Finding" + }, + "type": "array" + }, + "config_warnings": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema_version", + "collected_at", + "fingerprint", + "server", + "window", + "findings" + ] + }, + "Delta": { + "properties": { + "id": { + "type": "string" + }, + "subject": { + "type": "string" + }, + "severity": { + "type": "string" + }, + "before": { + "type": "number" + }, + "after": { + "type": "number" + }, + "pct_change": { + "type": "number" + }, + "first_observed": { + "type": "string", + "format": "date-time" + }, + "note": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "id", + "subject", + "severity", + "before", + "after" + ] + }, + "Deltas": { + "properties": { + "against": { + "type": "string", + "format": "date-time" + }, + "yesterday_hour": { + "type": "string", + "format": "date-time" + }, + "changes": { + "items": { + "$ref": "#/$defs/Delta" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "against", + "changes" + ] + }, + "Event": { + "properties": { + "kind": { + "type": "string" + }, + "object": { + "type": "string" + }, + "before": { + "type": "string" + }, + "after": { + "type": "string" + }, + "occurred_after": { + "type": "string", + "format": "date-time" + }, + "occurred_before": { + "type": "string", + "format": "date-time" + }, + "confidence": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "kind", + "confidence" + ] + }, + "Finding": { + "properties": { + "id": { + "type": "string" + }, + "object": { + "type": "string" + }, + "severity": { + "type": "string" + }, + "title": { + "type": "string" + }, + "detail": { + "type": "string" + }, + "evidence": { + "items": { + "type": "string" + }, + "type": "array" + }, + "objects": { + "items": { + "type": "string" + }, + "type": "array" + }, + "remediation": { + "type": "string" + }, + "impact": { + "$ref": "#/$defs/Impact" + }, + "confidence": { + "type": "number" + }, + "caveats": { + "items": { + "type": "string" + }, + "type": "array" + }, + "related": { + "items": { + "type": "string" + }, + "type": "array" + }, + "safety": { + "$ref": "#/$defs/Safety" + }, + "suppressed": { + "type": "boolean" + }, + "suppression_reason": { + "type": "string" + }, + "suppression_rule": { + "type": "string" + }, + "severity_remapped": { + "type": "string" + }, + "cluster_scoped": { + "type": "boolean" + }, + "preexisting": { + "type": "boolean" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "id", + "severity", + "title", + "detail", + "impact", + "confidence" + ] + }, + "Health": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "connections": { + "type": "integer" + }, + "tps": { + "type": "number" + }, + "commits_per_sec": { + "type": "number" + }, + "rollbacks_per_sec": { + "type": "number" + }, + "rollback_ratio": { + "type": "number" + }, + "cache_hit_ratio": { + "type": "number" + }, + "cache_blocks_sampled": { + "type": "integer" + }, + "deadlocks_per_min": { + "type": "number" + }, + "temp_bytes_per_sec": { + "type": "number" + }, + "tuples_returned_per_sec": { + "type": "number" + }, + "tuples_written_per_sec": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "connections" + ] + }, + "HorizonHolder": { + "properties": { + "source": { + "type": "string" + }, + "holder": { + "type": "string" + }, + "xmin_age": { + "type": "integer" + }, + "age_s": { + "type": "number" + }, + "detail": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "source", + "holder", + "xmin_age" + ] + }, + "IO": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "checkpoints_timed": { + "type": "integer" + }, + "checkpoints_requested": { + "type": "integer" + }, + "buffers_written_per_sec": { + "type": "number" + }, + "backend_fsyncs": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "checkpoints_timed", + "checkpoints_requested", + "backend_fsyncs" + ] + }, + "Impact": { + "properties": { + "score": { + "type": "number" + }, + "dimension": { + "type": "string" + }, + "estimate": { + "type": "string" + }, + "basis": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "score", + "dimension", + "estimate", + "basis" + ] + }, + "IndexStat": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "index": { + "type": "string" + }, + "scans": { + "type": "integer" + }, + "bytes": { + "type": "integer" + }, + "definition": { + "type": "string" + }, + "columns": { + "items": { + "type": "string" + }, + "type": "array" + }, + "method": { + "type": "string" + }, + "unique": { + "type": "boolean" + }, + "primary": { + "type": "boolean" + }, + "partial": { + "type": "boolean" + }, + "expression": { + "type": "boolean" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "index", + "scans", + "bytes" + ] + }, + "Indexes": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "total": { + "type": "integer" + }, + "scanned": { + "type": "integer" + }, + "unused": { + "items": { + "$ref": "#/$defs/IndexStat" + }, + "type": "array" + }, + "largest": { + "items": { + "$ref": "#/$defs/IndexStat" + }, + "type": "array" + }, + "redundant": { + "items": { + "$ref": "#/$defs/RedundantIndex" + }, + "type": "array" + }, + "unindexed_fks": { + "items": { + "$ref": "#/$defs/UnindexedFK" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "total", + "scanned" + ] + }, + "Limits": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "connections_used": { + "type": "integer" + }, + "connections_max": { + "type": "integer" + }, + "max_xid_age": { + "type": "integer" + }, + "max_mxid_age": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "connections_used", + "connections_max", + "max_xid_age" + ] + }, + "Locks": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "blocked_count": { + "type": "integer" + }, + "chains": { + "items": { + "$ref": "#/$defs/BlockingRow" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "blocked_count" + ] + }, + "NarrowIdentityColumn": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "column": { + "type": "string" + }, + "type": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "column", + "type" + ] + }, + "PartitionRollup": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "partitions": { + "type": "integer" + }, + "total_bytes": { + "type": "integer" + }, + "live_tuples": { + "type": "integer" + }, + "seq_scans": { + "type": "integer" + }, + "index_scans": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "partitions", + "total_bytes", + "live_tuples", + "seq_scans", + "index_scans" + ] + }, + "Progress": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "operations": { + "items": { + "$ref": "#/$defs/ProgressOp" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness" + ] + }, + "ProgressOp": { + "properties": { + "pid": { + "type": "integer" + }, + "operation": { + "type": "string" + }, + "relation": { + "type": "string" + }, + "phase": { + "type": "string" + }, + "pct": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "pid", + "operation" + ] + }, + "Queries": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "enabled": { + "type": "boolean" + }, + "total_exec_ms": { + "type": "number" + }, + "pgss_dealloc": { + "type": "integer" + }, + "pgss_count": { + "type": "integer" + }, + "pgss_max": { + "type": "integer" + }, + "top": { + "items": { + "$ref": "#/$defs/QueryStat" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "enabled" + ] + }, + "QueryStat": { + "properties": { + "queryid": { + "type": "integer" + }, + "query": { + "type": "string" + }, + "calls": { + "type": "integer" + }, + "total_ms": { + "type": "number" + }, + "mean_ms": { + "type": "number" + }, + "max_ms": { + "type": "number" + }, + "rows": { + "type": "integer" + }, + "cache_hit": { + "type": "number" + }, + "wal_bytes": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "queryid", + "query", + "calls", + "total_ms", + "mean_ms", + "max_ms", + "rows", + "wal_bytes" + ] + }, + "QueryWaits": { + "properties": { + "query_id": { + "type": "integer" + }, + "sample_text": { + "type": "string" + }, + "count": { + "type": "integer" + }, + "share": { + "type": "number" + }, + "lock_share": { + "type": "number" + }, + "io_share": { + "type": "number" + }, + "top_type": { + "type": "string" + }, + "top_event": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "query_id", + "count", + "share", + "lock_share", + "io_share" + ] + }, + "RedundantIndex": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "index": { + "type": "string" + }, + "covered_by": { + "type": "string" + }, + "bytes": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "index", + "covered_by", + "bytes" + ] + }, + "ReplicaRow": { + "properties": { + "client_addr": { + "type": "string" + }, + "application_name": { + "type": "string" + }, + "state": { + "type": "string" + }, + "sync_state": { + "type": "string" + }, + "sync_priority": { + "type": "integer" + }, + "replay_lag_sec": { + "type": "number" + }, + "write_lag_bytes": { + "type": "integer" + }, + "flush_lag_bytes": { + "type": "integer" + }, + "replay_lag_bytes": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "client_addr", + "state", + "sync_state", + "write_lag_bytes", + "flush_lag_bytes", + "replay_lag_bytes" + ] + }, + "Replication": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "is_replica": { + "type": "boolean" + }, + "replicas": { + "items": { + "$ref": "#/$defs/ReplicaRow" + }, + "type": "array" + }, + "receiver_lag_sec": { + "type": "number" + }, + "slots": { + "items": { + "$ref": "#/$defs/ReplicationSlot" + }, + "type": "array" + }, + "subscriptions": { + "items": { + "$ref": "#/$defs/Subscription" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "is_replica" + ] + }, + "ReplicationSlot": { + "properties": { + "name": { + "type": "string" + }, + "type": { + "type": "string" + }, + "active": { + "type": "boolean" + }, + "database": { + "type": "string" + }, + "retained_bytes": { + "type": "integer" + }, + "wal_status": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "name", + "type", + "active", + "retained_bytes" + ] + }, + "Safety": { + "properties": { + "blocking_caveats": { + "items": { + "$ref": "#/$defs/SafetyGuard" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "blocking_caveats" + ] + }, + "SafetyGuard": { + "properties": { + "id": { + "type": "string" + }, + "kind": { + "type": "string" + }, + "action": { + "type": "string" + }, + "text": { + "type": "string" + }, + "verify": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "id", + "kind", + "action", + "text", + "verify" + ] + }, + "SequenceUsage": { + "properties": { + "schema": { + "type": "string" + }, + "sequence": { + "type": "string" + }, + "last_value": { + "type": "integer" + }, + "ceiling": { + "type": "integer" + }, + "pct_used": { + "type": "number" + }, + "owned_by": { + "type": "string" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "sequence", + "last_value", + "ceiling", + "pct_used" + ] + }, + "Sequences": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "items": { + "items": { + "$ref": "#/$defs/SequenceUsage" + }, + "type": "array" + }, + "narrow_identity": { + "items": { + "$ref": "#/$defs/NarrowIdentityColumn" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness" + ] + }, + "ServerInfo": { + "properties": { + "version_num": { + "type": "integer" + }, + "version_text": { + "type": "string" + }, + "database": { + "type": "string" + }, + "provider": { + "type": "string" + }, + "in_recovery": { + "type": "boolean" + }, + "via_pooler": { + "type": "boolean" + }, + "started_at": { + "type": "string", + "format": "date-time" + }, + "uptime_seconds": { + "type": "integer" + }, + "extensions": { + "items": { + "type": "string" + }, + "type": "array" + }, + "capabilities": { + "items": { + "type": "string" + }, + "type": "array" + }, + "has_pg_monitor": { + "type": "boolean" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "version_num", + "version_text", + "database", + "uptime_seconds", + "extensions", + "capabilities", + "has_pg_monitor" + ] + }, + "Settings": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "overrides": { + "additionalProperties": { + "type": "string" + }, + "type": "object" + }, + "params": { + "additionalProperties": { + "type": "string" + }, + "type": "object" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "overrides" + ] + }, + "StandbyStatus": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "confl_tablespace": { + "type": "integer" + }, + "confl_lock": { + "type": "integer" + }, + "confl_snapshot": { + "type": "integer" + }, + "confl_bufferpin": { + "type": "integer" + }, + "confl_deadlock": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "confl_tablespace", + "confl_lock", + "confl_snapshot", + "confl_bufferpin", + "confl_deadlock" + ] + }, + "Subscription": { + "properties": { + "name": { + "type": "string" + }, + "worker_running": { + "type": "boolean" + }, + "last_msg_age_sec": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "name", + "worker_running" + ] + }, + "TableStat": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "total_bytes": { + "type": "integer" + }, + "live_tuples": { + "type": "integer" + }, + "dead_tuples": { + "type": "integer" + }, + "dead_ratio": { + "type": "number" + }, + "seq_scans": { + "type": "integer" + }, + "index_scans": { + "type": "integer" + }, + "mods_since_analyze": { + "type": "integer" + }, + "updates": { + "type": "integer" + }, + "hot_updates": { + "type": "integer" + }, + "last_analyze": { + "type": "string", + "format": "date-time" + }, + "last_autoanalyze": { + "type": "string", + "format": "date-time" + }, + "analyze_scale_override": { + "type": "number" + }, + "analyze_threshold_override": { + "type": "number" + }, + "autovacuum_count": { + "type": "integer" + }, + "autovacuum_disabled": { + "type": "boolean" + }, + "vacuum_scale_override": { + "type": "number" + }, + "vacuum_threshold_override": { + "type": "number" + }, + "last_vacuum": { + "type": "string", + "format": "date-time" + }, + "last_autovacuum": { + "type": "string", + "format": "date-time" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "total_bytes", + "live_tuples", + "dead_tuples", + "dead_ratio", + "seq_scans", + "index_scans", + "mods_since_analyze" + ] + }, + "Tables": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "db_size_bytes": { + "type": "integer" + }, + "top": { + "items": { + "$ref": "#/$defs/TableStat" + }, + "type": "array" + }, + "partitioned": { + "items": { + "$ref": "#/$defs/PartitionRollup" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "db_size_bytes" + ] + }, + "UnindexedFK": { + "properties": { + "schema": { + "type": "string" + }, + "table": { + "type": "string" + }, + "constraint": { + "type": "string" + }, + "columns": { + "type": "string" + }, + "child_bytes": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "schema", + "table", + "constraint", + "columns", + "child_bytes" + ] + }, + "VacuumHorizon": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "holders": { + "items": { + "$ref": "#/$defs/HorizonHolder" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness" + ] + }, + "WAL": { + "properties": { + "exactness": { + "type": "string" + }, + "reason": { + "type": "string" + }, + "bytes_per_sec": { + "type": "number" + }, + "records_per_sec": { + "type": "number" + }, + "buffers_full": { + "type": "integer" + }, + "dir_bytes": { + "type": "integer" + }, + "dir_files": { + "type": "integer" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "exactness", + "buffers_full" + ] + }, + "WaitBucket": { + "properties": { + "type": { + "type": "string" + }, + "count": { + "type": "integer" + }, + "share": { + "type": "number" + }, + "events": { + "items": { + "$ref": "#/$defs/WaitEvent" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "type", + "count", + "share" + ] + }, + "WaitEvent": { + "properties": { + "event": { + "type": "string" + }, + "count": { + "type": "integer" + }, + "share": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "event", + "count", + "share" + ] + }, + "WaitProfile": { + "properties": { + "available": { + "type": "boolean" + }, + "reason": { + "type": "string" + }, + "samples": { + "type": "integer" + }, + "window_seconds": { + "type": "number" + }, + "buckets": { + "items": { + "$ref": "#/$defs/WaitBucket" + }, + "type": "array" + }, + "by_query": { + "items": { + "$ref": "#/$defs/QueryWaits" + }, + "type": "array" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "available", + "samples", + "window_seconds" + ] + }, + "Window": { + "properties": { + "sample_seconds": { + "type": "number" + }, + "stats_reset_at": { + "type": "string", + "format": "date-time" + }, + "postmaster_start_at": { + "type": "string", + "format": "date-time" + }, + "window_age_seconds": { + "type": "integer" + }, + "stats_window_days": { + "type": "number" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "sample_seconds" + ] + } + }, + "description": "pgbot inspect --json — the versioned Context contract for agents and scripts." +} From c00eb3aea85d44f1adecc70234d4baaa19defe83 Mon Sep 17 00:00:00 2001 From: Jeffery Lofoneh Asamani Date: Fri, 18 Sep 2026 23:35:28 +0000 Subject: [PATCH 2/2] test(collect): skip the collation fixture before reading datcollversion on PG14 The version check shared a query with datcollversion, which doesn't exist before PG15, so PG14 errored instead of skipping. --- internal/collect/collation_integration_test.go | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/internal/collect/collation_integration_test.go b/internal/collect/collation_integration_test.go index 9bc3774..4779a9f 100644 --- a/internal/collect/collation_integration_test.go +++ b/internal/collect/collation_integration_test.go @@ -34,15 +34,18 @@ func TestIntegration_collationVersionMismatch(t *testing.T) { t.Cleanup(func() { admin.Close(context.Background()) }) var vnum int - var db string - var recorded *string - if err := admin.QueryRow(ctx, `SELECT current_setting('server_version_num')::int, datname, datcollversion - FROM pg_database WHERE datname = current_database()`).Scan(&vnum, &db, &recorded); err != nil { + if err := admin.QueryRow(ctx, `SELECT current_setting('server_version_num')::int`).Scan(&vnum); err != nil { t.Fatal(err) } if vnum < 150000 { t.Skip("pg_database.datcollversion is PG15+") } + var db string + var recorded *string + if err := admin.QueryRow(ctx, `SELECT datname, datcollversion FROM pg_database WHERE datname = current_database()`). + Scan(&db, &recorded); err != nil { + t.Fatal(err) + } if recorded == nil { t.Skip("this database's collation records no version (C/POSIX) — nothing can drift") }