Repository navigation
Conversation
51d0707 to
dff7857
Compare
|
Keep this gate. It's the right one:
When the seam is exact, both fixtures should pass with no exemption. |
42debf0 to
0c2df0e
Compare
|
Gate kept as you asked, and both receipt problems are gone on
|
|
Thanks @akol1 for keeping the gate strict. Any token difference, any logits difference under
Next we will run stage 3 on an M5 Ultra. There the projections take the lane kernels rather than |
053b403 to
64afee0
Compare
|
Refreshed and force-pushed:
Gates on |
|
Thanks @akol1. |
1d8db14 to
c7114a1
Compare
|
Thanks — and thank you for the cell list; it is what makes the lane path checkable from here. Both asks are in on the current head
Fresh receipts on The lane kernels are still the one thing no M4 can show, so your stage-3 M5 Ultra run — full and mlp at chunk 1 and 128, the 512-id prompt, and the decoded rounds, all with |
c7114a1 to
c1b3040
Compare
|
Rebased onto the current
The gate's own receipts in the body were re-read on the rebased final tree ( This room is one Mac now: the second Mac the pair receipts were recorded on is gone (the cluster was broken), so a fresh two-Mac |
c1b3040 to
bd05c1a
Compare
|
Rebased onto The gate's own receipts were re-run on this head against the checkpoint
Numbers from this head, run here, not carried over: The suite's own |
|
Thanks @akol1. Every point from our earlier reviews is answered on this head, and the gate's own commit is unchanged since we last looked; it rebases cleanly onto current main. Nothing more is needed from you here. This lands right after #1433 and #1434, which it sits on (#1434 has one fix to make first), and we'll run the M5 Ultra stage-3 cells we listed before it goes in. |
bd05c1a to
2e13027
Compare
|
Rebased: The loopback gate was run here at this head against the 4-bit checkpoint, and it reproduces the body's receipt table: the 8-step fixture with Numbers from this head, run here, not carried over: The M5 Ultra and M3 Ultra stage-3 cells are yours, as listed; this room has an M4 Pro only. Nothing else changes on a single Mac. |
2017980 to
46ee8b3
Compare
|
Rebased: The loopback gate was run here at this head against the 4-bit checkpoint, and it reproduces the body's receipt table: the 8-step fixture with Numbers from this head, run here, not carried over: The M5 Ultra and M3 Ultra stage-3 cells are yours, as listed; this room has an M4 Pro only. Nothing else changes on a single Mac. |
46ee8b3 to
e392c10
Compare
Step 2 of 6, stacked on step 1 (`qwen27/pair-seam`) and main (`40cb15b4`); step 3 stacks on this commit. A rank's layer program now carries one `Reduce` interface instead of assuming the whole matrix. Attention's o_proj and the MLP's down_proj seam this rank's output columns, gather the whole frame, and let the producer kernel reduce over full K on its own rank; the head seams the last row's logits and takes the pair's argmax over the whole vocab, so the picked token is the single Mac's token, not rank 0's half-vocab pick. On one Mac `reduce` is null and every seam is compiled out of the program: the single-Mac program, its digests and its tests are the ones that ran before. The rank-local K-split is fixed while the slice is. A projection's split is keyed on the checkpoint tensor's rows, not the view's rows: `Triple` carries `whole` (the matrix's row count) and `prepare` derives `tensorRows` from it, while the M5 and prompt tables keep the checkpoint rows they always keyed. `in_proj_qkv` loads as three blocks of one tensor (10,240 rows), so its gate/up-style stack keeps main's 4 K slices on 6,144 rows instead of reading the q block's 2,048 rows and splitting into 8; every width main served on one Mac keeps main's split, and host tests "a rank's row slice keeps the checkpoint matrix's K split" and "every one-Mac stack keeps main's K split" cover the eight stacks (qkv 10,240 rows: 4, q 6,144: 4, against the 2,048-row view's 8). `Part` drops its unused `k0`, `Triple` its `k`/`k0`, and `Shard`/the family's private `cols`/`attentionCols`/`gdnCols`/`downCols` and the column-window code `prepare`/`packTiles` used them for are gone, with `config_test.zig`'s column-span assertions deleted and the row fields left as they were. `simd_qmm.call` takes the rank's `whole` width and `chunks(n, whole)` picks the chunk fence from it, so a rank's K tree is its own full matrix's while a one-Mac stack passes 0 and keeps exactly main's fence (the core test pins `chunks` on 6,144/12,288 and 5,120/10,240 at 8, and on 6,144 and 5,120 alone at 16, which is the pair's own-width split the `gn` forwarding is there to avoid). `quant_gpu.strided` forwards the `gn` of the linear it serves. `q27_attn_merge` takes the rank's head count (buffer 11) and strides a row by that many heads instead of 24 (`G * 4`), so the non-prompt multi-row rounds a served chat feeds every earlier reply through merge the rank's own rows; `attention.zig` binds `g.heads`. The buffered form is only what a pair runs. One Mac has its 24 heads at compile time, and main's kernel used them there (`G * 4`, and `PG * 4` in the prompt merge), so each site now has a second instance: `q27_attn_merge_single` and `q27_attn_prompt_merge_single` carry main's body verbatim — same reduction, same float arithmetic, main's indexing, no `heads` or `kv_heads` buffer — and the encoder takes a trailing `split` and picks `io[4]`/`prompt[1]` for a pair and `io[10]`/`prompt[2]` for one Mac (`decode_round.zig` passes `r.reduce != null`). The two instances compute the same values, so only the emitted code differs. A function constant would need `MTLFunctionConstantValues`, which the runtime library never creates (`PipelineBuild` would fail the contract's first gate), so the second compiled instance is the route that needs no new Metal machinery. The config gains its shard parse (`config_test.zig` covers it), the embedding table stays whole on every rank — only the head is sliced — with `embedBound` reading the table's bound rather than the family's `vocab_size`, and the session keeps the pair's pick as the round's token and surfaces `error.PairFailed` from the head seam's sentinel rather than returning `maxInt(u32)`. The seam's `Reduce` is threaded through `forward.encodeTp`/`session`, and the prompt attention kernel's signature follows the seam's reopened encoder.
…ith the peer Step 3 of 6, stacked on steps 1-2 (`qwen27/pair-seam`, `qwen27/rank-slice`) and main (`40cb15b4`); step 4 stacks on this commit. The serving path learns the pair: a rank reads its rank-local shard, the serve loop routes every seam through the link, and the head's pick is the reply. A pair refuses what it cannot keep: vision, a rank count the shard does not match, and a drafter whose state only rank 0 holds, and it fails closed when its watchdog cannot be started. The slot accounting understands that both ranks hold the frame. A pair also keeps three serving rules. Its prefill chunk is capped at the seam window's rows (`pair.chunk`), because each chunk stages a seam and the window holds `batch_rows` of them; the host test pins that the served chunk times the widest seam fits `pairBytes()`. A sampled or grammar-constrained request is refused at the head of `prefillFn` (`error.PairGreedyOnly`) instead of being answered greedily and failing on the second token, with a name in `words()` and a host test. And a pair never resumes a slot in place: rank 1 resets the slot at each round's `begin` (`link.follow`), so `attach` passes the switch `Metal.init` read through `lb.slotResume(init, pair)`, which keeps `TF_QWEN27_SLOT_RESUME=0` off on one Mac and turns resume off for a pair (`lb.resumesInPlace`), or the ranks would part at the resumed offset; the host test pins that composition on both paths. A start-up step that fails after the watchdog is up is undone: the two threads live in `qwen27_link.Workers`, so `attach` sets their `quit`, stops following and joins the watchdog, and a host already started is stopped; `close` joins both before the pair it watched is freed, so neither thread reads a pair that `open` has already freed. `qwen27_link.zig` is the pair's link over the existing relay, and holds the watchdog and rank 1's follower in one `Workers`; `qwen27_pair.zig` holds the rank-local shard and the seam transport for the serve loop; main's `qwen27_costs.zig` times the window both ranks share; and `qwen27_served.zig`/`qwen27_slots.zig`/`qwen27_lanes.zig` the accounting a shard adds. GLM-5.3's transport is not touched: `pair.zig`/`pair.metal` are as on main, and no `glm_dsa` file changes in this series.
Step 4 of 6, stacked on steps 1-3 (`qwen27/pair-seam`, `qwen27/rank-slice`, `qwen27/pair-lifecycle`) and main (`40cb15b4`); step 5 stacks on this commit. `tf-qwen27-tp-check` is built by `zig build test` and launched by the suite. It needs `--model`, so CI's run prints `SKIP qwen27-tp-check: no --model DIR` and carries on, and a host with the checkpoint gets both stages. Stage 1 loads the model, then rank 1 through `loadShard`, and requires the same greedy ids, logits bytes and prompt-state digest; stage 2 starts two rank-local shards joined by `pair.Loopback`, a host-memory link, and requires both ranks to emit the same ids as the single Mac, with `pair_logits_byte_equal` and `pair_first_logits_difference: null` reported, and `bad_pick_at` and the seam give-up counters naming any loss. `--stage 1|2` narrows it and `--dump` prints each side's top two. `--decode-prompt` feeds the prompt through `Session.decodeChunk` in `--chunk N`-row rounds, an earlier reply's arithmetic rather than the prompt kernels: the multi-row round a served chat gives every earlier reply, which runs the tree merge (`q27_attn_merge_single` on one Mac) and not the prompt kernels' merge. Both stages use the same feed, and the receipt reports `prompt_arithmetic: "decode"` next to the chunk. `--chunk` may then reach 512, the served chunk (`model.promptRows()`); the pair's stages keep 128, the rows the seam window holds (`pair.stageBytes`), so `--chunk 512 --stage 2` is refused by name rather than panicking at the seam. Stage 1 at chunk 128 is where main's merge stride is exercised: it takes `q27_attn_merge` at `chunk` rows a round, so the single-Mac logits bytes are the reference the pair has to match. `zig/qwen27_core.zig` exports the pair and `zig/build/qwen27.zig` registers the program; `--forced-file` replays one side's picks on the other side for divergence localisation, and `--taps` hashes five layers' post-prompt residuals.
e392c10 to
2d224ac
Compare
|
Rebased onto
Run here, on this head, not carried over: This room has M4 Pro hardware only ( The PR body is refreshed with these same receipts. |
What this changes
Step 4 of 6 of the two-Mac Qwen3.8-27B series, rebased onto
main(94b90c9db), stacked on step 3 (qwen27/pair-lifecycle,eb8f417cf) and step 2'sqwen27/rank-slice(1af209fce); step 1 (#1432) is merged and its content arrived in3f0a715c. Step 5 stacks on this commit.The test suite gains the pair-vs-single gate.
zig build tf-qwen27-tp-checkbuilds it (zig/build/qwen27.zig:111), and the suite launches it; it needs--model, so a host without the checkpoint printsSKIP qwen27-tp-check: no --model DIR, so this host has no checkpoint to checkand the suite passes rather than fails. Stage 1 loads the checkpoint once, then rank 1 throughloadShard, and requires the same greedy ids, the same logits bytes and the same prompt-state digest; stage 2 starts two rank-local shards joined bypair.Loopback(a host-memory link with the pair's seam ABI) and requires both ranks to emit stage 1's ids, reportingpair_logits_byte_equal,pair_first_logits_differenceandbad_pick_at, and the merged top two when a mismatch names one.--stage 1runs the single-Mac parity only,--stage 2the loopback pair only, and--stage 3(the default) both.--taps 0,1,2,3,4hashes five layers' post-block residuals on every side so a divergence names its layer,--prompt-file/--forced-filereplay ids and picks, and--dumpprints each side's top two and state digests as[dump]/[state]lines (a side whose top two is unavailable printstop2=errorin place of the line's ids).--chunk Nfeeds a prompt in N-row rounds throughprefill, and--decode-promptfeeds the same rounds throughdecodeChunk— an earlier reply's arithmetic rather than the prompt kernels, which is the path a served chat's follow-up takes.Nmay reach 512,model.promptRows(); a Mac whose prompt side fuses takes all 512 rows a round, elsewhere a round is 128 rows and a wider chunk is fed in those rounds. The pair's stages keep the 128 rows the seam window allows, and a wider--chunkwith--stage 2is itself refused (error.PairChunkWiderThanTheSeamWindow) rather than silently narrowed. The review point was that--chunkhad to reachdecodeChunkand that the round size had to reach 512: an earlier head of this PR fed--decode-promptone row a round whatever--chunksaid, so the earlier-reply path had no round-size control. Both are wired now, and the--decode-promptreceipts below show--chunk 512and--chunk 1reaching the same tokens, withprompt_arithmetic: "decode"in both.zig/tests/qwen27/tp_report.zigholds the digests and the dump lines the receipt quotes:stateDigest/committedDigestover both KV caches, the GDN states and the slot offsets,tapDigestsover five post-block planes, andmergeTop2, which shifts a rank's local top two to the global vocabulary for the mismatch report. The receipt itself is JSON on stdout withrank0_tokens,rank1_tokens,reference_tokens,ranks_token_equal,reference_token_equal,first_difference,single_top2,pair_merged_top2,pair_logits_compared,pair_logits_byte_equal,pair_first_logits_difference,bad_pick_at, the twoseam_gave_upcounters, both prompt-state digests,chunk,prompt_arithmetic, and the memory triple (rank0_alone_resident_bytes,both_ranks_resident_bytes,second_rank_increment_bytes) againstdevice_max_working_set.A rank's verify never runs a tail.
forward.zig'sfinishruns the head seam first and hands the reopened encoder back to the tail (zig/src/families/qwen27/forward.zig:230,:241); when the seam ishead == .noneit is skipped and the tail still runs, which is the one-Mac path. Because the head writes only this rank's vocabulary columns while the tail's matcher reads the whole row,decode_round.zig:221refuses a round that has both witherror.TailOnRankbefore the cache checks or any GPU work;zig/qwen27_bank_boundary_host.zig:53pins that refusal with model-free stubs, and from step 3 on a pair refuses--drafterat the front door, so the two never meet.Receipt
Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: this commit (
2d224ac7f46bf5aaf2e6539d2dff06835b47f5de) onmain(94b90c9db65dcb71bbc39715901d6fb1f00407d0) plus steps 2-3; no MLX or PyTorch; macOS 27.0.1 on an Apple M4 Pro (Mac16,11), Metal only (applegpu_g16, no tensor units), Apple metal version 32023.921; checkpointTensorFold/Qwen3.8-27B-MLX-4bit, revision22d8d538154e0e5b6f8dcb1fcc74a60a0798104c, 4-bit affine, group 64, loaded from one directory on both ranks. The receipts below were re-run on this commit on 2026-10-30; the earlier table was recorded on the pre-rebase tree, and every row of the table reproduced these values when run here against the checkpoint above.Exactness, from
tools/bench_concurrent.py --alone --serialortoken_shaagainst"draft": false: the loopback pair caught a single-Mac vs pair token difference at the first token witherror.PairTokensDifferFromSingleMacwhen the head merge was deliberately made to stride one Mac's 24 heads instead of the rank's 12; with the merge fixed, the stage-2 runs below reportpair_logits_byte_equal: trueandpair_first_logits_difference: null, and nobad_pick_at. The two 512-row (--ids 512)--decode-promptreceipts below differ in exactly thechunkkey (512and1): the chunk reachesdecodeChunk, the tokens and the prompt-state digest are the same, andprompt_arithmeticisdecodein both. A 512-row run through the prompt kernels reaches the same[1011,220]withprompt_arithmetic: "prompt", so the option changes which arithmetic runs, not the picks.Decode speed before and after, from
tools/bench_openai.py: n/a, no served pair in this commit; the pair's decode rate is step 5's receipt.Prompt speed before and after: n/a, this commit adds a test gate; the chunk cap it pins is a correctness bound (the seam window), not a throughput change.
Tests run, with pass, fail and skip counts:
zig build test:584/584 steps succeeded; 8771/8824 tests passed (53 skipped)(run here with a cold local cache, so these are the counts of the whole suite; every step succeeded), exit 0; that run launches the gate, and its log printsSKIP qwen27-tp-check: no --model DIR, so this host has no checkpoint to check(exit 0, so a host without the checkpoint passes instead of failing).zig build test-kernel-contract:20/20 steps succeeded; 141/141 tests passed, exit 0.zig build dev-tools:220/220 steps succeeded.zig build test-qwen27-served-host:7/7 steps succeeded; 48/49 tests passed (1 skipped), exit 0.zig build tf-qwen27-tp-check:42/42 steps succeeded(build only; the suite is what launches it with--model).zig fmt --check build.zig zig: clean. Filtered contract runs:SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 34 ms, limits 121 msandSUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 30072 ms, limits 119 ms. Lean rule:lean: 424 problems, 424 on the base, 0 new(tools/zig/lean_new.py, exit 0).The gate's own receipts, all with
--modelon the checkpoint above:Every run below exits 0 on
--model "$CHECKPOINT"(the 4-bit MLX checkpoint named in the Environment bullet) except the last, which is the refusal; each line is a receipt this host printed, not a summary of one.tf-qwen27-tp-check --model "$CHECKPOINT" --ids 4 --steps 8shard1_tokens_equal: true,shard1_logits_byte_equal: true,shard1_first_logits_difference: null,shard1_prompt_state_equal: true; stage 2 (the loopback pair):ranks_token_equal: true,reference_token_equal: true,first_difference: null,bad_pick_at: null,seam_gave_up_rank0: 0,seam_gave_up_rank1: 0, tokens[828,318,951,2445,8,310,279,1001]on both ranks and on the single Mac… --ids 4 --steps 8 --dumppair_logits_compared: true,pair_logits_byte_equal: true,pair_first_logits_difference: null, and[logits] steps_compared=8 first_differing_step=null— the full 248,320-wide row was hashed on every decoded step, not only the two picks… --ids 512 --steps 2 --chunk 512 --stage 1[1011,220],shard1_tokens_equal: true,shard1_logits_byte_equal: true,prompt_arithmetic: "prompt"… --ids 512 --steps 2 --chunk 512 --decode-prompt --stage 1decodeChunk's 128-row rounds: tokens[1011,220],shard1_tokens_equal: true,prompt_arithmetic: "decode", and a differentsingle_mac_prompt_statethan the prompt-kernel run — the digest shows the other arithmetic actually ran… --ids 512 --steps 2 --chunk 1 --decode-prompt --stage 1chunkkey: the 128-row rounds and the 1-row rounds reach the same tokens and the same prompt-state digest… --ids 128 --steps 2 --chunk 128 --decode-prompt --dumppair_logits_compared: true,pair_logits_byte_equal: true,pair_first_logits_difference: null,bad_pick_at: null, tokens[68,220],[audit] x_rank0=514 x_rank1=514 mismatched_exchanges=0 first_bad=nullA wider chunk under a pair stage is refused by name instead of panicking:
--ids 512 --steps 2 --chunk 512 --stage 3exits 1 witherror: PairChunkWiderThanTheSeamWindow.Not run, and why: a two-Mac
pair_accept.pyrun (step 5's scripts); this gate's loopback link is the same seam ABI over host memory, and the two-Mac receipt is step 5's, recorded before the series was rebased onto this base (the second Mac is gone now). CUDA, HIP/RDNA, M1-M3, M5 and Windows are not in this room.Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. — the gate compares greedy ids only and both sides run with no drafter;--decode-promptis the earlier-reply arithmetic a resumed chat takes, and its receipts above match the prompt-kernel rounds token for token.pair_logits_byte_equalis the assertion that the pair's bits are one Mac's bits, not merely its tokens.SUMMARYline is in the receipt. — no GPU kernel changes in this commit; the filtered contract runs in the receipt are the regression check for the pair's four records core: add the row-split pair seam and split simd_qmm over K #1432 put onmain.applegpu_g16); no CUDA, HIP, M1-M3, M5 or Windows.tf-qwen27-tp-checkstep (no such step, the same shape core: add the row-split pair seam and split simd_qmm over K #1432's body reported) and no gate source; the gate fails a pair whose merged logits or ids drift from the single Mac ([logits] first_differing_step,bad_pick_at,error.PairTokensDifferFromSingleMac), and the planted wrong stride in the exactness bullet above is the one this comparison catches. Without--modelthe built gate prints its SKIP line and exits 0, which the suite does on hosts without the checkpoint.lean: 424 problems, 424 on the base, 0 new.--model DIRand names no host or user path.Stacked on:
qwen27/rank-slice,qwen27/pair-lifecycle. This commit is step 4 of 6.