[BUG] xlsx: batch not atomic under IPC failure at scale — partial write left on disk after reported failure

[cyc1216]

Version: 1.0.138 (Homebrew build, --no-self-contained), macOS 26.5.2 arm64.

Summary: docs for batch say it's "atomic by default: any failed item rolls back the whole batch." At large scale (tens of thousands of items in one batch call), we saw the opposite: the CLI process reported failure while the resident kept processing in the background and wrote most (but not all) of the changes to disk — leaving the workbook in a partially-modified, inconsistent state despite the CLI reporting total failure.

What we did

A real-world xlsx (6 sheets, 51,824 formula cells with empty cached values — <f>...</f> with no <v>) needed those formulas re-evaluated. We enumerated every formula cell per sheet with:

officecli query file.xlsx "SheetName!cell:has(formula)" --json

built one combined batch of 51,824 set items (re-setting each cell's own formula to trigger write-time auto-evaluation), and ran:

officecli batch file.xlsx --input fix_all.json --json

Observed

The command failed:

Resident for file.xlsx is running but the batch could not be delivered (main pipe busy or unresponsive). Retry, or run 'officecli close file.xlsx' and try again.

(subprocess exit code 3)

The resident process (__resident-serve__ file.xlsx in ps) kept running at 100%+ CPU for several minutes after the CLI process had already exited with the error, then disappeared on its own. Checking the file afterward, per sheet:

Sheet	Cells fixed	Total formulas
A	1824	1896
B	1822	1896
C	4224	4424
D	8970	9480
E	5394	5688
F	26520	28440
Total	48754	51824

~94% of the batch had actually been written to disk, despite the CLI reporting total failure and no rollback occurring.

Separately, we noticed per-item cost within a single batch call grows superlinearly with batch size — 200 / 2,000 / 10,000-item synthetic batches averaged 1.19ms / 1.91ms / 4.29ms per item respectively, while a single unrelated set on files pre-populated to those same sizes cost a near-constant ~0.4–0.5s regardless of file size (ruling out file size as the driver). This growth is likely what pushes very large batches past whatever timeout produces the "pipe busy or unresponsive" failure in the first place.

Expected

One of:

• batch actually honors "atomic by default" — on any failure (including this IPC/timeout scenario), the file on disk is unchanged from before the batch started, or
• if true atomicity isn't feasible when the CLI loses its connection to the resident mid-batch, the CLI should not report unqualified failure — it should reflect the partial-completion state (e.g. "N/total succeeded before connection was lost") instead of an error implying nothing happened.

Notes

We don't have a small, reliably-reproducing repro yet — this surfaced on a real ~52k-cell/6-sheet workbook. It may reproduce with fewer items on a slower machine, or with more cross-sheet references driving up per-item cost. Happy to share the synthetic benchmark generator if useful.

[goworm]

Thanks for the excellent report — reproduced, confirmed, and fixed.

What actually happened: the resident's idle watchdog measured time since the last command boundary, not whether a command was still running. A single batch that outlasts the idle window (12 min by default — your 51k-item batch did, due to the superlinear per-item cost you measured) caused the resident to tear itself down mid-command: the in-progress reply was cancelled (so the CLI reported "could not be delivered" even though the batch had been delivered and was executing), while the exit-time flush still wrote the partially-applied state to disk. That is exactly the false-failure-over-partial-write you observed. We reproduced it deterministically with a short idle window and a batch that outruns it.

Fixes (on fix/resident-crash-atomic-saves, landing in the next release):

• 917cfef — the resident stays alive while a command is in flight; the idle window only counts from the completion of the last command. Your scenario now runs to completion and reports success.
• 61209f9 — resident flushes now go through a temp-file + atomic-replace writer, so a process death mid-write can no longer leave a torn/partial file: disk is either the pre-flush or the post-flush state.

Until the release, two mitigations: split very large batches into chunks (also sidesteps the superlinear growth, which we're tracking separately), and/or run officecli close <file> before reading the file from outside the CLI so the state on disk is deterministic.

Thanks again — the per-sheet completion table and the timing measurements made this one straightforward to pin down.