Skip to content

fix(session): a torn write no longer destroys the whole conversation - #18

Merged
ForkedInTime merged 1 commit into
mainfrom
fix/phase3-persistence
Aug 3, 2026
Merged

fix(session): a torn write no longer destroys the whole conversation#18
ForkedInTime merged 1 commit into
mainfrom
fix/phase3-persistence

Conversation

@ForkedInTime

Copy link
Copy Markdown
Owner

Phase 3 (data integrity & persistence, 5 files, 1,779 LOC). One real data-loss path; the rest of the phase checked out clean and is recorded as cleared rather than changed.

An interrupted append lost the entire session

Session::append writes JSONL and never fsynced, so a turn was reported as saved while the bytes could still be in the page cache. load_messages then parsed with .collect::<Result<_>>(), meaning any malformed line failed the whole load.

Together those are a total-loss path: a crash or power-cut mid-append leaves a half-written final line, and the next /resume fails outright — losing the entire conversation, which is the one thing sessions exist to prevent.

  • append now fsyncs, so a completed turn is durable rather than merely buffered.
  • load_messages tolerates a torn final line — dropped with a warning, the rest loads. Losing one turn beats losing all of them.

Corruption anywhere other than the last line remains a hard error, deliberately. Silently skipping a middle line could drop a tool_use while keeping its tool_result, which the API rejects outright — a subtly broken conversation is worse than a clear error. The message now names the offending line.

Parsing moved into parse_message_lines so the behaviour is testable against an explicit file rather than the global sessions directory.

Session export was not atomic

export used a direct fs::write, which truncates first — an interrupted export left an empty file where the transcript was. Now uses the same atomic_write as the session files.

Checked and cleared, not changed

  • Compaction cannot break tool_use/tool_result pairing. snip_compact never removes blocks — it replaces ToolResult content with a placeholder and leaves structure intact. This was the sharpest risk on the phase plan (a mismatch is a hard API 400 that bricks a session) and the design is already correct.
  • memory.rs durability is SQLite's, not a hand-rolled write path.
  • memory.rs SQL is fully parameterised, LIMIT included. No injection.
  • SessionMeta::save and overwrite already use atomic_write from an earlier audit — only append and export had been missed.

QA (triple-checked)

  • 606 tests, 0 failures
  • Clippy clean under the exact CI command
  • Zero panics in production code across all five Phase 3 files
  • Phase 1 and Phase 2 suites re-run green — no regressions
  • Release builds, 19.08 MB

The torn-line fix verified by restoring the intolerant collect(), which fails 2 tests.

Phase 3 (data integrity & persistence, 5 files, 1,779 LOC). One real data-loss
path; the rest of the phase checked out clean and is recorded as cleared rather
than changed.

## An interrupted append lost the entire session

`Session::append` writes JSONL and never fsynced, so a turn was reported as
saved while the bytes could still be in the page cache. `load_messages` then
parsed with `.collect::<Result<_>>()`, meaning **any** malformed line failed the
whole load.

Together those are a total-loss path: crash or power-cut mid-append leaves a
half-written final line, and the next `/resume` fails outright — losing the
entire conversation, which is the one thing sessions exist to prevent.

Two changes:

  - `append` now fsyncs, so a completed turn is durable rather than merely
    buffered.
  - `load_messages` tolerates a torn *final* line: it is dropped with a warning
    and the rest of the history loads. Losing one turn beats losing all of them.

Corruption anywhere **other** than the last line is still a hard error, and
deliberately so. Silently skipping a middle line could drop a `tool_use` while
keeping its `tool_result`, which the API rejects outright — a subtly broken
conversation is worse than a clear error, and the message now names the line.

The parsing moved into `parse_message_lines` so this behaviour is testable
against an explicit file rather than the global sessions directory.

## Session export was not atomic

`export` used a direct `fs::write`, which truncates first — an interrupted
export left the user with an empty file where their transcript was. Now uses the
same `atomic_write` as the session files themselves.

## Checked and cleared, not changed

  - **Compaction cannot break tool_use/tool_result pairing.** `snip_compact`
    never removes blocks; it replaces `ToolResult` *content* with a placeholder
    and leaves structure intact. This was the sharpest risk on the phase plan —
    a mismatch is a hard API 400 that bricks a session — and the design is
    already correct.
  - **memory.rs durability is SQLite's.** Not a hand-rolled write path.
  - **memory.rs SQL is fully parameterised**, `LIMIT` included. No injection.
  - **`SessionMeta::save` and `overwrite` already use atomic_write** from an
    earlier audit; only `append` and `export` had been missed.

QA (triple-checked): 606 tests, 0 failures. Clippy clean under the CI gate.
Release 19.08 MB. Zero panics in production code across all five Phase 3 files.
Phase 1 and 2 suites re-run green. The torn-line fix verified by restoring the
intolerant `collect()`, which fails 2 tests.

Co-Authored-By: Arch Linux <noreply@archlinux.org>
@ForkedInTime
ForkedInTime merged commit 27e4d1b into main Aug 3, 2026
5 checks passed
@ForkedInTime
ForkedInTime deleted the fix/phase3-persistence branch August 3, 2026 23:01
ForkedInTime added a commit that referenced this pull request Aug 4, 2026
… usable (#19)

Phase 4. One defect, in the interaction between two individually-correct behaviours.

The API rejects an assistant tool_use that no following user turn answers, and nothing
validated history before sending. PR #18's torn-line recovery drops the final line — if that
line was the tool_result, recovery leaves a dangling tool_use and every subsequent request
400s with no way back. Phase 3 turned total loss into permanent breakage.

repair_dangling_tool_uses now synthesises the missing results on load, splicing into the
existing user turn where there is one and stubbing only unanswered ids.

Cleared without change: the live tool loop cannot produce an unanswered tool_use (every path
pushes a ToolResult first); cancellation is safe (persistence happens only on turn
completion). Still open: no 429 retry/backoff — 529 is handled, rate limiting is not.

611 tests, 0 failures. Clippy clean. Phase 1-3 suites re-run green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant