Restart persistence still fails after #2080: ui_messages.json missing after successful create-phase disk check

Observed failure

• CI: https://github.com/Zoo-Code-Org/Zoo-Code/actions/runs/39531510842/job/118998238304?pr=2092
• Tested checkout: fa5ee9b (PR [Chore] Speed up lint and check-types with caches and staged-only pre-commit lint #2092 merge into 6c76f90).
• [Fix] Restart-persistence E2E flakes when the final ui_messages.json write lands after VS Code quits #2080 merged 2026-10-17 23:19:56 UTC as 289538e. The exact tested checkout contains its hasTaskUiMessageText wait.
• Ubuntu runner, VS Code 1.100.0, mocked restart-persistence scenario.
• Task ID: 01a15c3d-a958-75fb-b669-77228942775c.

Timeline (UTC, 2026-10-21):

1. 02:15:08.127: create phase passes. It has awaited TaskCompleted and successfully read RESTART_PERSISTENCE_MARKER from ui_messages.json.
2. 02:15:08.147: create extension host exits.
3. 02:15:08.233: verify VS Code starts, sharing the same user-data directory.
4. 02:15:12.410: resume scheduling fails with TaskMessagesReadError / ENOENT reading ui_messages.json.
5. 02:15:42.388: verify times out after 30 seconds waiting for the resume_completed_task message.

The timeout is downstream of the failed message read. PR #2092 changes development tooling, with no task-persistence runtime changes.

Confirmed implementation weakness

safeWriteJson writes a replacement temp file, renames the existing target to a backup, then renames the replacement to the target:

A process exit between the two renames leaves the canonical path absent, even when a previous completed write contained the marker. Exception rollback cannot run after process termination. Reading the marker once proves visibility at that instant; it does not prevent a subsequent write from removing that path.

readFileWithMissingRetry only retries ENOENT once after a random 1–10 ms delay. It does not recover backup files. Longer retries cannot repair an interrupted replacement when the writer has exited.

There are independent UI-message writers, including background usage collection and checkpoint messages. The logs do not identify which writer, if any, ran after the marker check.

Deterministic fault-injection reproduction

Using the repository's bundled safeWriteJson and readFileWithMissingRetry implementations:

1. Write ui_messages.json containing RESTART_PERSISTENCE_MARKER.
2. Read the marker back successfully (the condition introduced by [Fix] Restart-persistence E2E flakes when the final ui_messages.json write lands after VS Code quits #2080).
3. Start a child process that calls safeWriteJson again for the same path.
4. Wrap fs/promises.rename to perform the real rename and exit the child with code 0 immediately after target-to-backup succeeds, before replacement-to-target.
5. Read the canonical path with readFileWithMissingRetry.

Result: 5/5 injected runs return ENOENT. Both .ui_messages.json.bak_.tmp and .ui_messages.json.new_.tmp remain, and the backup still contains the marker. This reproduces the storage failure mechanism, not the full VS Code shutdown interleaving.

Confidence and missing evidence

The crash window is reproduced. It is a strong candidate for this CI failure, but the CI interleaving is not proven. In particular, a live write/read gap during verify is another possible explanation for ENOENT. Logs do not record rename operations or directory contents.

The run has no uploaded artifacts. runTest.ts removes the scenario workspace in finally, including on failure, so the task directory, backup files, replacement files, and VS Code profile cannot be inspected afterward.

Proposed follow-up

• Preserve the scenario directory on failure and upload task files, phase results, and VS Code logs as CI artifacts.
• Add trace points for UI-message write start, backup rename, replacement rename, completion, marker check, and shutdown, including task ID and process ID.
• Add a subprocess regression test that interrupts the real writer between renames and checks the restart recovery contract.
• Evaluate a replacement protocol that keeps the committed target available through interruption, or explicit backup recovery under the shared lock. Respect the Windows open-file EPERM behavior documented in [Fix] Restart-persistence E2E flakes when the final ui_messages.json write lands after VS Code quits #2080.
• If shutdown coordination is used, it must stop new writes and drain existing writes; another marker poll or arbitrary sleep does not establish that condition.
• Test transient reader gaps separately from permanent absence after writer termination. Avoid treating a larger read retry as a complete fix.