insertMany/upsertMany chunk by row count, silently overflowing Postgres's parameter limit on wide tables

[smiggleworth]

Summary

insertMany and upsertMany chunk only by row count. A 1,000-row batch on a 70-column table generates 70,000 bind parameters, overflowing PostgreSQL's 65,535-parameter protocol limit and returning the misleading bind message has 4464 parameter formats but 0 parameters error.

Triage

Confirmed, High. The documented/default 1,000-row chunk fails against a real wide PostgreSQL table; narrow-table tests do not exercise the protocol boundary.

Required implementation details

• Compute the union of supplied row properties before batching.
• Set the effective chunk size to min(configuredChunkSize, floor(65535 / propertyCount)).
• Apply the same calculation to both insertMany and upsertMany.
• Fail clearly before executing SQL if a single row shape could itself exceed the parameter limit.
• Preserve the user-configured chunk size whenever it is already below the protocol-derived bound.

Acceptance criteria

• Verify the issue author is smiggleworth.
• insertMany derives an effective row chunk from both configured chunk size and row width, never producing more than 65,535 bind values.
• upsertMany uses the identical protocol-aware bound.
• Existing empty-batch, returning, configured-chunk, and no-implicit-transaction behavior remains intact.
• A fast 70-column/1,000-row regression proves both operations split into 65,520- and 4,480-parameter statements.
• A real PostgreSQL regression proves both the insert and upsert succeed for the same 70,000-parameter naive batch and affect all 1,000 rows.
• Public method documentation states that configured row chunks are capped by PostgreSQL's bind-parameter limit.
• Guardrail (prevent this class of bug): Keep both the deterministic generated-query bound and the real wide-table PostgreSQL test so future batch rewrites cannot regress to row-count-only chunking.
• Full ORM release gate, real PostgreSQL 16/17/18 hosted matrix, and exact-head CI pass before merge.

Definition of done

• All acceptance criteria above are checked with evidence and the release-prepared exact-head PR is ready for squash merge as @askrjs/orm@0.0.2.

Severity

High — the documented default can cross a hard wire-protocol limit on realistic wide tables and produces a corrupted, non-actionable error.

---

Found via a real-PostgreSQL scale-boundary pass.

[smiggleworth]

Acceptance evidence for 1f0c648a13a824ae6fddeb147f573c4796419eeb:

• Authorship verified: this issue was opened by smiggleworth.
• Red commit 26f5b31 proves both generated statements contain 70,000 parameters and the real insert fails with PostgreSQL 08P01 / bind message has 4464 parameter formats but 0 parameters.
• Green commit 488412d caps both operations at min(configuredChunkSize, floor(65535 / propertyCount)) and documents the cap on both public methods.
• The deterministic regression proves both insert and upsert produce exact parameter counts [65520, 4480] and no statement crosses 65,535.
• The real PostgreSQL 18 regression proves both operations succeed for all 1,000 rows on a 70-column table.
• Existing unit suite, coverage, types, package, and performance gates pass.
• Exact-head hosted run 31903030248 passes package CI plus the real PostgreSQL 16, 17, and 18 matrices: https://github.com/askrjs/askr-orm/actions/runs/31903030248

[smiggleworth]

Release closure:

• PR fix: harden PostgreSQL failure boundaries #17 squash-merged as bf6d7fd057855cbcc8e726136b0dd88bf686dd56.
• Publish run 31903111492 passed the full package and PostgreSQL 16/17/18 gates before publishing: https://github.com/askrjs/askr-orm/actions/runs/31903111492
• Annotated tag v0.0.2 resolves to the exact squash commit.
• npm serves @askrjs/orm@0.0.2 with integrity sha512-B+JJ7B8zFVZo2yPoQyEv0xcnfb/rtpuvDC1Et8LBu3c879v1gvCvedsl6nfLnnUTj9RPiwLcuPlTaUH0dMnjPQ==.
• A clean registry-only consumer connected the published package to PostgreSQL 18, terminated a checked-out transaction backend, caught DatabaseError { category: "connection", code: "57P01" }, then inserted and upserted all 1,000 rows of a 70-column table.
• The clean consumer reported zero runtime audit vulnerabilities. Its temporary PostgreSQL container was stopped and removed after verification.