fix(graph): settle disposal, bound storage, and progress buffered staging

[jmagar]

Problem

A forced live Labby refresh stalled during private graph disposal before publication. SQLx 0.8.6 can finish pool shutdown before a connection returning through its asynchronous release path enters the idle queue. Waiting only for the connection count to become zero leaves that late idle connection undrained.

Resolution

Keep private graph reservations independently polled while ordered embedding buffers consume earlier results. A third live run exposed a gate deadlock: a suspended speculative graph completion owned the writer needed by the preceding batch checkpoint. Staging now uses an owned task with abort-on-drop cancellation.

Separate the 256 MiB logical journal/side-effect budget from the 1 GiB physical SQLite aggregate budget; canonical rows, indexes and transaction sidecars require storage overhead. A second live Labby attempt failed cleanly with the original 256 MiB physical cap while its logical journal was below 100 MiB. Sparse boundary tests cover all sidecars and byte-count diagnostics.

Repeat graceful pool shutdown until the private pool reaches zero connections, retaining the owner lock throughout. Log the original generation failure through the existing bounded, redacted diagnostic projection before disposal.

Skip bulk UPSERT updates when every mutable column is identical, preserving NULL transitions, actual changes, and creation timestamps. The serial private graph writer now has one connection and a bounded 64 MiB page cache; DELETE/FULL durability remains explicit.

Approach and reviewer considerations

The shared settlement helper covers explicit disposal and cancellation/drop. The regression blocks after_release across shutdown, then releases the connection into the closed pool. No transport changes, schema changes, configuration changes, or timeout-based unlocking.

Verification

• The barrier regression fails against the previous implementation and passes with this fix.
• Graph suite: 120 passed, 1 ignored.
• Graph Clippy with all targets and warnings denied passed.
• Services build, formatting, generated contracts/docs, monolith, secrets, and layering checks passed.
• Focused Rust, simplicity, and security lifecycle review completed with no outstanding findings.
• Live code.claude.com page completed cleanly in 6.459 seconds, 24 points and 64 graph nodes, with no warnings on the disposal fix.
• A second Labby run reported the physical-cap failure cleanly in 288.841 seconds and cleaned its unpublished generation; no successful large-repository benchmark is claimed yet.
• The ordered-buffer writer regression fails with inline staging and passes with independently owned staging. Caller cancellation releases the owned writer.
• Six focused services tests passed, including all three concurrent graph integration tests with CI’s 8 MiB stack setting. Services Clippy with all targets and warnings denied passed; lifecycle review found no issues.
• Working concurrency head 0a5d894e8 passed all CI checks. Live Labby generation 26 completed without degradation in 411.463 seconds with 2,305 documents and 35,560 points; graph staging progressed concurrently and activation used bulk application. Compared with the earlier baseline, this input has 62 additional documents, so no identical-input speedup is claimed.
• Verified durable activation receipt, all cleanup debt complete, older/failed generations zero Qdrant points, and current generation 35,560 points.
• The unchanged-row regression fails before the optimization and passes after it; genuine alias reassignment and NULL transitions remain applied. The dedicated private-pool regression verifies pool bound, cache, and DELETE/FULL settings. All three real concurrent source tests pass with the tuned pool.
• Final tuning head de412c30a live Labby generation 27 completed without degradation in 349.126 seconds: 2,305 documents, 35,561 points; same 2,362 inventory paths as gen26 with one content hash changed. Phase totals: preparation/embedding/graph staging 282.080s, atomic publication 66.398s, completion 0.648s. Bulk graph activation dropped from 47.611s to 16.120s; total run dropped by 62.337s (15.15%). These are two live forced refreshes, not proof of provider-cold performance.
• Final-head code.claude.com page smoke completed in 2.735s with 24 points, 64 graph nodes, and no warnings.
• Final-head graph query returned 11 nodes/10 edges with no warnings. Durable generation27 receipt exists; all source cleanup debt completed; generation26 zero points and generation27 exactly35,561. Inventory contains only code/docs categories and no PNG/SVG/JSON/binary/lockfile types. Expected content-redaction notices remain non-degraded.
• Final exact-head CI passed: workspace 6,587 passed and 9 skipped, with all required platform and CodeQL checks green. Merged as dcbd50b8683ce2b752c2e623a4fe8a6e3b0c04ee; its tree matches tested head de412c30a. Deployed axon:main-dcbd50b86 on Tootie; readiness reports SQLite, Qdrant, and TEI ready, Docker healthy with zero restarts.

Risk

Disposal retains ownership while SQLite connections remain. Physical cleanup stays asynchronous and uses durable recovery records. The original failed live generation was never published; the prior committed generation remained selected.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.