Skip to content

fountain v0.4.1 -> v0.6.1, so a lost turn fails cleanly and says why - #105

Closed
jhgaylor wants to merge 1 commit into
mainfrom
fountain-0.6.1-turn-fails-cleanly
Closed

fountain v0.4.1 -> v0.6.1, so a lost turn fails cleanly and says why#105
jhgaylor wants to merge 1 commit into
mainfrom
fountain-0.6.1-turn-fails-cleanly

Conversation

@jhgaylor

@jhgaylor jhgaylor commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

The status page claimed completing a turn was a race decided by machine speed,
and that a faster machine loses more often — 5 of 5 orphaned on a laptop, 2 of
2 completed on a runner (#67).
Standing the deployment up and running conversations through it says otherwise.

What was actually happening

Every conversation provisions fresh. The reattach branch is never taken by a
first dispatch. What varied was a crash:

kick_turn/4    Sprites.write(command, prompt)   <-- spritzer's one-shot exec already closed
               ** (EXIT) normal, ConversationServer dies
supervisor     restarts it
dispatch       sandbox.status == "ready" now, so -> reattach
reattach       list_sessions -> 426 -> turn_orphaned

27 conversations on v0.4.1: 27 fresh provisions, 14 crashes, 14 reattaches
— 1:1, each reattach 1–3ms after its own crash, and none without a crash
first
. The reattach and the 426 were the tail of the crash, not the cause.

The machine-speed reading did not survive either: the same laptop that gave 5
of 5 gave 8 completed to 12 orphaned over 20 back-to-back runs, and
provisioning duration does not separate the outcomes (median 14.4ms completed
against 13.0ms orphaned, fully overlapping). Those earlier samples were a coin
flip landing the same way twice.

The pin

Filed upstream as fountain#603,
fixed in v0.6.0. Two follow-ups came out of testing that fix:

  • #608 — the clean
    failure dropped the runtime's exit code. turns.exit_code was NULL on all
    17 failed turns of a 30-conversation run. Fixed in v0.6.1.
  • #609v0.6.0's
    verify_email/1 release task verified the account and then crashed on a
    PubSub broadcast the Repo-only VM has nothing to receive. Caught by just e2e, which went red on it. Fixed in v0.6.1.

So this goes straight to v0.6.1.

v0.4.1 v0.6.0 v0.6.1
crashes / reattaches 14 / 14 0 / 0 0 / 0
failed turns carrying an exit code 0 of 17 9 of 9
just verify-email works crashes works

A lost race now costs the turn, not the deployment: conversation back to
idle, turn failed with :command_exited (runtime exited N), nothing left
running.

The gate

just e2e asserted a pairing — reattach implies orphaned, no reattach
implies exit 0. That held on every run, but only because crash ⇒ reattach; it
was right by accident.

It is now three-way and stricter:

  • exit_code: 0 → completed
  • turn/failed with :command_exited → the emulator's one-shot exec
    (spritzer#18)
  • anything else stops the build, including a reattach in a fresh
    conversation — that is the #603 signature, and 44 conversations across the
    two pins produced zero of them, so seeing one means the pin regressed

A turn failing for some other reason also stops it: that reason would be new,
and worth reading before it gets documented.

Docs

status.md, data-plane.md, ci.md and the README carry the corrected
mechanism and the measurements behind it. The old reading is kept as history
rather than deleted — a claim that was wrong twice is worth being able to
trace, and the status page's credibility rests on showing its work.

No rate is asserted anywhere. The split moves with how fast conversations are
opened — 11 of 16 completed at three-second intervals against 2 of 14 packed
back to back — and pinning a number to it would be the same mistake one layer
down.

Verification

  • just e2e green from nothing at v0.6.1, including register → verify →
    first-admin → conversation gate → every seam
  • 44 conversations across v0.6.0 and v0.6.1: zero crashes, zero reattaches,
    zero new 426s
  • e2e happened to take the completed branch, so the gate was replayed against a
    captured v0.6.1 failed stream to confirm the other branch classifies —
    fountain#611 deliberately keeps :command_exited in front of the new detail
    for exactly this reason
  • just check and just site clean

Upstream comments posted on #67 and spritzer#18 correcting both.

🤖 Generated with Claude Code

The status page said completing a turn was a race decided by machine speed,
and that a faster machine loses more often. Standing the deployment up and
running 27 conversations through it says otherwise: every conversation
provisions fresh, the reattach branch is never taken by a first dispatch, and
the same laptop returns both outcomes at roughly even odds.

What actually happened was a crash. fountain wrote the prompt into the exec
session with a bare GenServer.call; spritzer's exec is one-shot, so when the
close won the race the call landed on a dead process and exited the
ConversationServer. The supervisor restarted it, the restarted server found
its own sandbox already `ready` and reattached, and list_sessions got
spritzer's 426 — an orphaned turn behind an error that named nothing real.
The reattach was the tail of the crash, not the cause. 14 crashes, 14
reattaches, each 1-3ms after its own crash, none without one first.

Filed upstream as fountain#603 and fixed in v0.6.0; the exit code it dropped
on the way out is #608, fixed in v0.6.1. Bumping straight past v0.6.0 because
its release task for verify-email verified the account and then crashed on a
PubSub broadcast that the Repo-only VM has nothing to receive (#609, also in
v0.6.1).

So the pin moves to v0.6.1 and the gate stops asserting a pairing it was only
ever right about by accident. It now accepts either legitimate shape —
completed, or failed as :command_exited — and fails on a third. A reattach in
a fresh conversation is the #603 signature and stops the build, which is
stricter than what was here before.

The docs carry the corrected mechanism and the numbers behind it, and keep
the old reading as history: a claim that was wrong twice is worth being able
to trace.

Verified: `just e2e` green from nothing at v0.6.1, and 44 conversations across
the two pins with zero crashes, zero reattaches and an exit code on every turn.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jhgaylor

jhgaylor commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #109.

This described a race — which arm of a lost-write crash you landed on, and how the turn died as a result. spritzer 0.5.0 closes it: turns now complete every time (34 of 34), so there is no longer a race to document.

#109 carries the pin bump, moves Completing a turn to Verified, and keeps the history — including this PR's own reading of it — in data-plane.md. It is built on current main, so it also does not fight #104/#106/#108.

@jhgaylor jhgaylor closed this Aug 7, 2026
@jhgaylor
jhgaylor deleted the fountain-0.6.1-turn-fails-cleanly branch August 7, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant