[akol1]

What this changes

Step 5 of 6 of the two-Mac Qwen3.8-27B series, rebased onto main (c39c09b9), stacked on step 4 (qwen27/pair-tests, cd510c64), on top of steps 2-3 (qwen27/rank-slice d43966c5, qwen27/pair-lifecycle 6db850cf); step 1 (#1432) is merged, and the stack's first commit retargets main's cost-rule case test (3abfc26e). Two commits: the scripts, then the oracle's per-case build, the serve's /health policy line they read, and the acceptance run's cold case. Step 6 stacks on this branch's tip.

The scripts that make a two-Mac run reproducible, and the serve's half of the recorder's pair guard, with no machine name and no user path in them:

• tools/qwen27/record_oracle.py records a single-Mac acceptance oracle: five cases (a 128-token and a 256-token /v1/completions, a 128-token and a 256-token /v1/chat/completions, and a two-turn chat case that asks a question, answers it, then asks a follow-up about the same example), each at temperature 0 with "draft": false. Each case records the token sha, the build that recorded it (runtime), and the gpu_family of the Mac it ran on: code-128, code-256, chat-128 and chat-256 say tensorfold-native 1.1.2, and the two-turn case, recorded later, says tensorfold-native 1.1.7, so a case names the exact binary that must reproduce it rather than one label for the file. The recorder reads the serve's /health first and refuses (exit 2, a plain line on stderr) a serve whose policy names the two-Mac speed-up mode — rank 0 of a pair would pass the case's own turn-1 check just as well, and an oracle has to come from one Mac — and also one whose /health is unreadable or carries no version string. The version it reports becomes the new case's runtime, and the refusal happens before any completion request, so a pair serve never answers a recording.
• The pair's /health now carries the line that refusal reads. qwen27_pair_flags.healthPolicy returns speed-up mode: a two-Mac pair, rank 0 answers greedy requests only for a pair and nothing for one Mac, LaneHost.init takes it as .policy, and the server prints it under /health; a host test pins both halves. Before this commit the qwen27 host left engine_api.Info.policy empty, so a pair serve reported no policy line and the recorder's refusal could not fire — the guard and the line it reads land together.
• tools/qwen27/pair_accept.py run replays that oracle against a served pair: every case is sent --repeat 2 times and both replies must be identical, and their token_sha must equal the oracle's, with --only labelled as a subset rather than a full acceptance. It refuses an oracle whose gpu_family it cannot match: it prints the recorded family, this Mac's family and the reason (before the M5 the bits come from simd_qmm, on the M5 from the lane kernels with their own K splits), and returns 2 before sending a request — the pair's exactness is exact on one family of arithmetic, so it will not compare across the boundary. cold builds a two-turn conversation from the oracle's chat fixture (its answer, then a follow-up about the same example) and passes when the pair reports no cached prompt tokens: a pair keeps no prompt cache (one slot, and each Mac holds only its own halves of its states), so a resumed-prompt case cannot pass on one, and this case pins that documented limitation instead of contradicting it.
• tools/qwen27/pair_supervise.sh up|down|status|watch starts rank 1 on $PEER and rank 0 locally, waits for both to answer /health, restarts both when either stops answering, and stops only the pids it wrote to its pidfiles (never by command-line pattern). It verifies the peer's host key instead of turning the check off, and it refuses to start when the two binaries differ unless ALLOW_MISMATCH=1 says the A/B is deliberate. PEER_STATE defaults on rank 1 to rank 1's own ${TMPDIR:-/tmp}/tensorfold-pair, so neither rank writes to the other's state directory.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: this commit (ea66002975d254638304d3caf6979de621ff333f) on main (c39c09b9fb5d539f600a171544fb5622fd65b363) plus steps 2-4; no MLX or PyTorch. The gates ran on an Apple M4 Pro (Mac16,11), Metal only. The two-Mac run quoted below was produced before the series was rebased onto this main, from the series tree at that time: zig build native there produced sha256 3a82766f4bffbc9f3bc3575f6dec6841b3e4274e324bff83755f96f70b81400e (the supervisor printed binaries match: 3a82766f4bff), both Macs installed that binary (tensorfold-native 1.1.8), on two Macs of the same GPU family (apple-m4), Metal only, at the checkpoint revision 22d8d538154e0e5b6f8dcb1fcc74a60a0798104c (4-bit affine, group 64). The second Mac is no longer available, so there is no fresh two-Mac run on the rebased heads: the receipt below records the pair path before the rebase, and the binary this commit builds is not that binary.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: tools/qwen27/pair_accept.py run (defaults --repeat 2, --timeout 600) exited 0 with five PASS lines and pair_accept: 5/5 cases match the oracle on http://127.0.0.1:4100; 0 did not; no FAIL or SKIP line was printed. The five cases are code-128 at f5c9409cfe40, code-256 at 2b28aa2108de, chat-128 at b69b0269917f, chat-256 at a06fab348fbe, and the two-turn chat fixture chat-two-turn-128 at 7d156013fbdd, and each case ran twice with identical token_sha and identical reply text. Both health endpoints named the same GPU generation and backend ("device": "Apple M4 Pro", "backend": "metal"), both carried the pair's policy line ("policy": "speed-up mode: a two-Mac pair, rank 0 answers greedy requests only"), and the supervisor printed binaries match: 3a82766f4bff before starting the ranks, so both Macs served the same bytes. pair_accept.py cold on that pair then reports cold: turn 1 prefill marked 0; turn 2 cached=0 and passes with PASS cold: the pair kept no prompt cache, as its one slot over two Macs' states implies, exit 0: the pair accepts the two-turn chat case and pins the no-cache limitation the docs state. This is the recorded run from before the rebase, and the log it is quoted from is the one the supervisor and pair_accept.py wrote on this Mac then. The script the receipt ran is byte-identical to this head's — tools/qwen27/pair_accept.py is untouched by the rebase — but the binary is not: the receipt's tree predates this rebase, and this head sits on c39c09b9, so a fresh two-Mac acceptance would answer from a different tree. On the rebased heads the single-Mac loopback gate (tf-qwen27-tp-check, step 4) and the host tests in zig build test are what run here, and a fresh two-Mac acceptance needs a working pair (the second Mac is gone).
• Decode speed before and after, from tools/bench_openai.py: n/a, these scripts are the harness for that measurement; the rates belong to the run above, not to this diff.
• Prompt speed before and after: n/a, same reason; pair_accept.py records per-case prompt and decode rates in its JSON, but the two-Mac receipt above is the acceptance result, not a speed claim.
• Tests run, with pass, fail and skip counts: zig build test: 580/580 steps succeeded; 249/269 tests passed (20 skipped) (Zig does not re-count cached test steps, so these are the counts of what this checkout re-ran; every step succeeded), exit 0 — this is also what runs the host test that pins the pair's /health policy line. zig build test-kernel-contract: 20/20 steps succeeded; 114/114 tests passed, exit 0. zig build dev-tools: 216/216 steps succeeded. zig build test-qwen27-served-host: 7/7 steps succeeded; 44/45 tests passed (1 skipped), exit 0. zig build tf-qwen27-tp-check: 42/42 steps succeeded. zig fmt --check build.zig zig: clean. Filtered contract runs: SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 38 ms, limits 216 ms and SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 34211 ms, limits 135 ms. Lean rule: 425 problems, 425 on the base, 0 new (tools/zig/lean_new.py, exit 0). The oracle file's five cases and the family refusal strings are checked in and reproducible with record_oracle.py and pair_accept.py as described above.
• Not run, and why: a fresh two-Mac pair_accept.py run on the rebased heads — the second Mac is gone (the cluster was broken), so the acceptance above is the recorded pre-rebase run; the single-Mac loopback gate and the host tests cover the seam here. CUDA, HIP/RDNA, M1-M3, M5 and Windows are not in this room; pair_accept refuses the M5 because the oracle's family is apple-m4.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. — the scripts only drive a served pair; every case is temperature 0 with "draft": false and both repeats must match, so a draft path that appears would fail the acceptance rather than pass it.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. — the acceptance sends each case twice and requires identical replies; the two-turn case is the chat continuation that reuses a reply's state, and cold is the pair's resumed-prompt check: it drives the follow-up turn and passes when the pair reports no cached prompt tokens, the documented reason being that a pair keeps no prompt cache.
• No precision traded for speed. If the bits change, the description says which sums change and why. — no arithmetic here; the family check is the guard that the comparison never crosses an arithmetic boundary.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. — this commit adds no GPU kernel; it drives the pair. The filtered contract runs in the receipt are the regression check for the kernels the run covers.
• Prompt processing is no slower than the last release. — n/a, this commit adds no prompt kernel; the harness records the prompt rate in its JSON and no claim is made from this diff.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. — two apple-m4 Macs for the quoted run; no CUDA, HIP, M1-M3, M5 or Windows; the M5 is refused by the oracle's family check by design.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. — the scripts are new files, so their parent commit has none of them; pair_accept.py prints pair_accept: <url> did not answer: ... and exits 2 when nothing answers, instead of a traceback, and record_oracle.py names its own preconditions (no opening case, a turn with no text, a reply without token_ids) the same way. The /health policy line has a host test that fails if a pair reports no line or one Mac reports the pair's.
• Comments and docstrings are one line. No measurements or history in the source. — lean: 425 problems, 425 on the base, 0 new.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines. — $PEER, $MODEL, $BIN and $STATE are read from the environment; the oracle stores a GPU family, not a host; pair_supervise.sh verifies the peer's host key and never disables the check.

Stacked on: qwen27/rank-slice, qwen27/pair-lifecycle, qwen27/pair-tests. This commit is step 5 of 6.

[ashhart]

• The oracle. It says "A paired run must reproduce every token_sha here", but pair_known then accepts 58daf9743077 for chat-256. Please remove the exemption.
• stop_peer and down kill every process matching tensorfold-native serve.*--speed-up on both Macs, which can include someone else's GLM or Flash Next pair. Kill only the pids the script started, and check them.
• The default SSH_OPTS turns off StrictHostKeyChecking on a script that starts binaries remotely. Please keep the normal host-key check.

[akol1]

All three are fixed on 0b4f1f4e:

• The exemption is gone. There is no pair_known anywhere, and the oracle's four cases are compared with no accepted-different value: pair_accept.py fails a case whose token_sha differs instead of accepting a known sha.
• Stopping is pidfile-only. down and its stop path kill only the pids the script recorded, wait out each one before moving on, and never match by command line, so another pair's processes can't be hit.
• Host keys stay checked. SSH_OPTS defaults to -o BatchMode=yes -o ConnectTimeout=5; the host-key check keeps its default, so an unknown or changed key fails the connection rather than being accepted.

[ashhart]

Thanks @akol1. The exemption is gone, stops act only on the recorded pids after checking the binary, and host keys stay verified. Three asks:

1. The oracle (tools/qwen27/qwen38_27b_oracle.json) does not record which GPU produced it. One Mac's bits before the M5 come from simd_qmm, and on the M5 from the lane kernels with their own K splits. So a record made on one family cannot accept a pair on the other. Please record the GPU family and have pair_accept.py refuse a mismatch, or take the reference from a one-Mac serve on rank 0's Mac.
2. Every case is a single turn, so acceptance never feeds an earlier reply. Earlier replies are the decoded spans behind qwen27: rank-slice the 27B family and gather the whole embedding table #1433's first point. Please add a two-turn chat case.
3. Small: PEER_STATE defaults to rank 0's own $TMPDIR (tools/qwen27/pair_supervise.sh:48). On macOS that is a per-user /var/folders path rank 1 usually cannot create, unlike what the usage at :24 says. Please expand the default on the peer.

We are accepting this series on the loopback gate plus your two-Mac receipts, and the refreshed bodies quote none. Please post a two-Mac pair_accept.py run on the refreshed heads, including the two-turn case. This also needs #1432's rebase.

[akol1]

Refreshed and force-pushed: qwen27/pair-tools is now 2ee85964, rebased onto current main (db3fd5ab), with the body rewritten around this head.

1. The oracle records the GPU family, and pair_accept.py refuses a mismatch. tools/qwen27/qwen38_27b_oracle.json now carries "gpu_family": "apple-m4" (with the checkpoint revision it was recorded on), and a Mac outside that family is refused before any request is sent: the run says which family the record was made on, which family this Mac reports, and why one family's bits cannot accept the other, then exits 2. The one-Mac-bits argument is exactly yours: before the M5 the logits come from simd_qmm, on it from the lane kernels with their own K splits.

2. A two-turn chat case is in the oracle and passes. chat-two-turn-128 feeds an earlier reply back through decodeChunk, which is the path qwen27: rank-slice the 27B family and gather the whole embedding table #1433's merge fix covers.

3. PEER_STATE now expands on the peer. The rank-1 launcher resolves the default on rank 1's Mac before it uses it, so a default under ${TMPDIR:-/tmp} no longer points rank 1 at rank 0's per-user directory. The usage text says so.

Two-Mac acceptance on the refreshed heads

Built the refreshed series at its tip (660f7a35, docs step only above this commit; tensorfold-native 1.1.7), installed the same bytes on both Macs — [pair] binaries match: ac25aa210ae2 — started rank 1 before rank 0 (rank 1 is up (pid 23547); both ranks answer: rank 0 on :4100 and rank 1 on :4101; rank 0's port refused 2 probes while the pair handshake completed), and ran pair_accept.py run against rank 0. /health on both ranks reports "device": "Apple M4 Pro" and "backend": "metal".

• PASS code-128 token_sha=f5c9409cfe40 (2 identical runs)
• PASS code-256 token_sha=2b28aa2108de (2 identical runs)
• PASS chat-128 token_sha=b69b0269917f (2 identical runs)
• PASS chat-256 token_sha=a06fab348fbe (2 identical runs)
• PASS chat-two-turn-128 token_sha=7d156013fbdd (2 identical runs)
• pair_accept: 5/5 cases match the oracle on http://127.0.0.1:4100; 0 did not, RUN_EXIT=0

So the acceptance condition is met on the refreshed heads, two-turn case included, and the refreshed bodies quote this receipt. One honest line: pair_accept.py warm still exits 1 with FAIL warm: the next turn reused no prompt state (cached=0) — that is the documented limitation (the pair claims nothing about prompt-state reuse across turns); the two-turn case above is accepted by run, not by warm.

Gates on 2ee85964: zig build test 565/565 steps succeeded; 191/194 tests passed (3 skipped); test-kernel-contract 43/43; dev-tools 216/216; tf-qwen27-tp-check 42/42 steps succeeded; zig fmt --check build.zig zig clean; pair SUMMARY 15 kernels: 15 pass gates 1-4, 0 fail, 0 known findings; 35 controls: 35 red as planted, 0 missed; 0 limit problems; gates 288 ms, limits 109 ms; simd SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 30439 ms, limits 104 ms; lean 428 problems, 428 on the base, 0 new.

[ashhart]

Thanks @akol1. The oracle now records its GPU family and pair_accept.py refuses another family before sending a request, the two-turn case is in, and PEER_STATE is resolved on rank 1. Thank you too for the two-Mac run on the refreshed heads: five of five, the two-turn case included. That's the two-Mac receipt we said we'd accept for this series.

Two small things, not blocking:

• The oracle's runtime still reads tensorfold-native 1.1.2, though the two-turn case was recorded later. A runtime per case would say which build recorded each sha.
• record_oracle.py checks that turn 1 reproduces chat-128, but rank 0 of a pair passes that check too. If the serve can say it is a pair, the recorder should refuse it, so an oracle always comes from one Mac.

[akol1]

Both asks are in on d2e1e62b, and the second one needed the serve's half of it, which was missing.

A runtime per case. qwen38_27b_oracle.json no longer carries one top-level runtime: code-128, code-256, chat-128 and chat-256 say tensorfold-native 1.1.2, chat-two-turn-128 says tensorfold-native 1.1.7, and the description says that each case names the build that recorded its own token_sha, so a case is never compared against a receipt whose recorder is unknown. pair_accept.py compares token_sha, not the build, so its cases are unchanged.

The recorder refuses a pair serve. record_oracle.py reads /health first and refuses, exit 2 with a plain line on stderr, a serve whose policy names the two-Mac speed-up mode, and also one whose /health is unreadable or carries no version string; the version it does report becomes the new case's runtime. The refusal happens before any completion request, so a pair serve never answers a recording.

That guard could not fire before this commit: engine_api.Info.policy existed (zig/src/core/engine_api.zig:224) and /health prints it (zig/src/server/status_routes.zig:36), but the qwen27 host left it empty and record_oracle.py never read /health at all. This commit adds both sides — the pair names the mode when it attaches, its host test pins the line, and the recorder reads /health first and refuses that serve before any completion. From rank 0 of the two-Mac run on this branch, trimmed to the fields that matter:

{"status": "ok", "model": "22d8d538154e0e5b6f8dcb1fcc74a60a0798104c", "max_batch_size": 1,
 "policy": "speed-up mode: a two-Mac pair, rank 0 answers greedy requests only",
 "warming": false, "version": "1.1.8", "backend": "metal", "device": "Apple M4 Pro"}

With that pair serving, the recorder now refuses:

$ tools/qwen27/record_oracle.py --base-url http://127.0.0.1:4100
record_oracle: http://127.0.0.1:4100/health reports the two-Mac pair policy (speed-up mode: a two-Mac pair, rank 0
answers greedy requests only); an oracle must come from a single-Mac serve
$ echo $?
2

A bug this flushed out in the acceptance script. pair_accept.py's warm case asked a pair for a resumed prompt state, but a pair keeps no prompt cache (one slot, and each Mac holds only its own halves of its states), so that case failed on every pair by construction — it is the line you saw fail in the receipt I posted. It is now cold: it drives a second turn in the same conversation and passes when the pair reports no cached tokens, which pins the documented "keeps no prompt cache" limitation instead of contradicting it. The two-Mac run on the final head:

pair_supervise.sh up reports binaries match: 3a82766f4bff (both Macs installed the same tensorfold-native 1.1.8 build of this head) and both ranks answer: rank 0 on :4100 and rank 1 on :4101, 4s after rank 0 started; the acceptance run then passes each case on two identical runs -- pair_accept: 5/5 cases match the oracle on http://127.0.0.1:4100; 0 did not (code-128 f5c9409cfe40, code-256 2b28aa2108de, chat-128 b69b0269917f, chat-256 a06fab348fbe, chat-two-turn-128 7d156013fbdd) -- and the new case ends it: cold: turn 1 prefill marked 0; turn 2 cached=0 / PASS cold: the pair kept no prompt cache, as its one slot over two Macs' states implies, exit 0. The supervisor then lists both ranks serving (exit 0) and stops both (exit 0).

Fresh receipts on d2e1e62b: zig build test 576/576 steps succeeded; 228/246 tests passed (18 skipped), exit 0; zig build dev-tools 216/216 steps succeeded; zig build tf-qwen27-tp-check 42/42 steps succeeded; zig build test-kernel-contract 43/43 steps succeeded; zig fmt --check build.zig zig clean; the pair filter SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 35 ms, limits 152 ms and the simd filter SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 34054 ms, limits 126 ms; lean 425 problems, 425 on the base, 0 new.

The head moved once more since my heads-up: it is now d2e1e62b, the doc comment above the new healthPolicy folded from two lines to one after this repo's one-line comment rule flagged it; the series tip on top of this step, 05bb79a9, adds only docs. Nothing under zig/kernels or on any arithmetic path is affected by either move, and the binary the two-Mac run above installed on both Macs is the one this head builds (3a82766f4bff…, binaries match: 3a82766f4bff).

[akol1]

Rebased onto the current main (7c068895) and force-pushed; the head is now 42d938e8487604b1bc6429019e6b6aa3eaf786c7. The series order is unchanged, and these are re-run numbers from the rebased head, not carried over.

zig build test: 576/576 steps succeeded; 236/254 tests passed (18 skipped); zig build test-kernel-contract: 43/43; zig build dev-tools: 216/216; zig build tf-qwen27-tp-check: 42/42; zig fmt --check build.zig zig clean; filtered pair run gates 32 ms, limits 104 ms, filtered simd run gates 29789 ms, limits 118 ms; lean rule 425 problems, 425 on the base, 0 new.

This room is one Mac now: the second Mac the pair receipts were recorded on is gone (the cluster was broken), so a fresh two-Mac pair_accept.py run is not possible here. The bodies keep that run labelled as a dated receipt from before the rebase and put the single-Mac loopback gate (tf-qwen27-tp-check) and the host tests next to it. No other question from the earlier comments is still open.

[ashhart]

Thanks @akol1, the tools look good: the oracle records its build per case and its GPU family, the recorder refuses a pair, and the /health line one Mac reports doesn't change. There's one fix before this can land, and it's small.

In tools/qwen27/pair_supervise.sh, up() calls check_binaries without || return 1. Run directly, up still stops on a mismatch, because set -e applies there. But watch calls it as if up; then, and sh switches off set -e inside any function that an if tests. So under watch a binary mismatch is printed and ignored, and both ranks start on different builds. watch is the unattended mode, so that's the one place the check matters most. Here's a three-line repro:

/bin/sh -c 'set -eu; chk(){ return 1; }; up(){ chk; echo started anyway; }; if up; then echo up-ok; fi'

That prints "started anyway" and then "up-ok". Please make it check_binaries || return 1, and do the same for start_rank1 || return 1 and start_rank0 || return 1, so a failed start doesn't sit out the whole READY_TIMEOUT. It would also help if resolve_peer_state stopped the script with exit status 1 when it can't read rank 1's TMPDIR and PEER_STATE isn't set, rather than carrying on with an empty path.

That fix is all this PR needs. It lands after #1432 to #1435, in series order.

[akol1]

Rebased onto main (581cbe65); qwen27/pair-tools is ac0688ce3c13a8f4039ab0f538c03a9ff8bbc6e1, two commits on step 4 (bd05c1a0): the scripts, then the serve's /health policy line they read.

All three fixes are in tools/qwen27/pair_supervise.sh:

• up() now fails each start step itself: need_up || return 1 (:197), check_binaries || return 1 (:203), start_rank1 || return 1 (:205), start_rank0 || return 1 (:206). The comment above them (:201-202) names the reason: watch calls up inside an if, and sh switches set -e off in a function an if tests.
• need_peer || return 1 also guards need_up (:110) and the two paths that read rank 1's state (:278, :291).
• resolve_peer_state (:72) now clears PEER_STATE, says could not read rank 1's TMPDIR from $PEER; set PEER_STATE to rank 1's state directory, and exit 1 (:79) instead of carrying on with an empty path.

Receipts, run here on this head; sh -n on the script is clean:

$ /bin/sh -c 'set -eu; chk(){ return 1; }; up(){ chk || return 1; echo started anyway; }; if up; then echo up-ok; fi'
(no output, rc 0)

$ env -u PEER -u PEER_STATE sh tools/qwen27/pair_supervise.sh up
[pair] set PEER to the rank 1 ssh target
(rc 1)

$ env PEER=127.0.0.1 PEER_STATE= sh tools/qwen27/pair_supervise.sh up
[pair] could not read rank 1's TMPDIR from 127.0.0.1; set PEER_STATE to rank 1's state directory
(rc 1)

The first is your three-line repro with || return 1: it no longer prints "started anyway" or "up-ok". The last two stop before any binary starts, and the second one names PEER_STATE as you asked.

Numbers from this head, run here, not carried over: zig build test 580/580 steps succeeded; 243/263 tests passed (20 skipped); zig build test-qwen27-served-host 7/7 steps succeeded; 38/39 tests passed (1 skipped); zig build tf-qwen27-tp-check 42/42 steps succeeded; zig build test-kernel-contract 20/20 steps succeeded; 114/114 tests passed; zig build dev-tools 216/216 steps succeeded; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 35 ms, limits 137 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 32457 ms, limits 125 ms; lean rule 425 problems, 425 on the base, 0 new.

The supervisor's two-rank paths still need two Macs, and the second Mac this receipt was recorded on is gone; the checks above are the one-host failures, and the host tests plus step 4's loopback gate carry the pair comparison.

[ashhart]

Thanks @akol1. All three fixes are in. up() now fails each start step itself, so under watch a binary mismatch or a failed start stops the attempt instead of being ignored. resolve_peer_state stops with status 1 and names PEER_STATE when it cannot read rank 1's TMPDIR. I re-ran the three-line repro with the fix and it prints nothing. Nothing more is needed from you on this one. It's ready and will land right after #1433, #1434 and #1435, in series order (#1434 has one fix to make first).

[akol1]

Rebased: qwen27/pair-tools is ea66002975d254638304d3caf6979de621ff333f, two commits on step 4 (cd510c64c6ca6275de1388f0bb9bee5a9fb71222), onto main (c39c09b9fb5d539f600a171544fb5622fd65b363); the stack's first commit (3abfc26e) retargets main's cost-rule case test, so the tree under this gate is green. The tools themselves are unchanged: every file this branch adds (pair_accept.py, pair_supervise.sh, the oracle JSON, record_oracle.py, pair_flags.zig) is byte-identical to the head you reviewed, and the /health line it adds to qwen27_served.zig is that same line (the file itself differs only by main's advance and step 3's qwen27_link.Workers move, and by the --drafter fix step 3 carries).

Numbers from this head, run here, not carried over: zig build test 580/580 steps succeeded; 249/269 tests passed (20 skipped); zig build test-qwen27-served-host 7/7 steps succeeded; 44/45 tests passed (1 skipped); zig build tf-qwen27-tp-check 42/42 steps succeeded (builds and skips without --model); zig build dev-tools 216/216 steps succeeded; zig build test-kernel-contract 20/20 steps succeeded; 114/114 tests passed; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 38 ms, limits 216 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 34211 ms, limits 135 ms; lean rule 425 problems, 425 on the base, 0 new.

The two-Mac pair_accept.py receipt in the body is the original run and is not re-run: the second Mac is gone. The scripts are unchanged since that run, and zip/rsync/up/down are exercised on this Mac by the host-side paths the earlier rounds fixed.

[ashhart]

Thanks @akol1. The scripts and the /health policy line look ready. A pair's /health now names the two-Mac mode and one Mac's reports nothing new, the recorder refuses a pair serve, and the supervisor's start steps each fail on their own under watch. This lands as soon as steps 2 to 4 (#1433, #1434 and #1435) are in, in series order. One small heads-up for the next rebase: main now carries the same cost-rule test retarget as the stack's first commit (3abfc26e), so that commit drops out as empty when the series is rebased onto current main. Nothing else is needed from you on this one.