[akol1]

What this changes

Step 2 of 6 of the two-Mac Qwen3.8-27B series, rebased onto main (94b90c9db). Step 1 was merged as #1432 and its content is already on main (the squash merge 3f0a715c is an ancestor of the base), so this branch carries no step-1 commit; step 3 stacks on this commit.

The branch is a single commit. main already carries the cost-rule retarget this branch used to bring itself: main's 5c9706f3 moved the 27B window timing into zig/src/native/qwen27_costs.zig and pointed the case test at the file that owns the timing, and that retarget is an ancestor of 94b90c9db, so the leg is green here and the rank slice is the whole branch.

The 27B family becomes rank-aware behind one interface: forward.Reduce (zig/src/families/qwen27/forward.zig:23) carries a seam's part, stage, gather and failed, plus an optional argmax that defaults to null; a one-Mac model therefore compiles the head seam out and executes exactly the layer program it executed before this series. projection.prepare derives a rank's row slice from the checkpoint matrix (gn), so the slice keeps the checkpoint's K split, tile and prompt tables (zig/src/families/qwen27/projection.zig:61); prompt rows take promptSplit of the whole matrix, so the pair's prompt reduction is the reduction tree one Mac uses rather than a half-width tree. The width the seam carries is the tensor's own: the fused gate-up keeps the member split instead of recomputing it from the stacked width, a local qkv block keys neither m5Split nor promptSplit, and simd_qmm_bits.call(..., whole) keeps the split rank's chunk count through twinExact as well.

The split is by output rows with the full K on the rank that owns them: 12 of 24 attention heads, 2 of 4 KV heads, 8 of 16 linear-attention q/k heads, 24 of 48 value channels and 3,072 of 6,144 z rows, 8,704 of the MLP's 17,408 rows, 2,560 of the 5,120 output rows for o_proj/down_proj at full K, and 124,160 of the 248,320-row lm_head. o_proj and down_proj compute a rank's own output rows at full K, and the other rank's rows come over the seam as whole bf16 rows: stage copies this rank's columns of a frame row and gather copies each rank's half to its own offset, so no two partial sums meet and no row is computed twice. Embeddings and norms stay whole on every rank, so a token id means the same thing on both, and the embedding gather's bound stays the whole 248,320-row table (config.embedBound). The head seam is argmax over each rank's 124,160-logit half and pick over the pair: no rank computes or gathers the whole 248,320-row logit vector, the pick is the better of the two half-vocab winners, and a non-finite logit on either half fails closed through the sentinel instead of letting the other half's winner stand in for the whole vocabulary's. session.zig keeps the merged top two for its cross-block tie test and reads the pick's maxInt(u32) sentinel as error.PairFailed, never as a logit.

Two host-path details the maintainer asked for: the prompt attention merge strides the rank's own head count — q27_attn_merge takes constant uint& heads (zig/kernels/metal/qwen27/attention_mpp.metal:270) and indexes ((int64_t)node * (int64_t)heads + h) * D, with attention.zig:408 setting it from the rank's geometry; the prompt merge writes the rank's KV node rows (kv_heads in Q27PromptArgs, attention_prompt.metal:170) instead of assuming the model's four. The in_proj_qkv 2,048-row trap is keyed on the tensor's rows, not on a split table, so a whole 6,144-row qkv block stays whole; projection.zig's tests pin it: "every one-Mac stack keeps main's K split", "a rank's row slice keeps the checkpoint matrix's K split", and "stacked gate-up keeps the member K split rather than recomputing it from stacked width". config_test pins the rank spans against the checkpoint's real shapes — 12 heads, 2 KV heads, 8,704 intermediate rows, 124,160 vocab rows, 2,560 output rows, q rows from 6,144 — while rank 0's spans start at 0, and synthetic_graph runs the same graph on both ranks. The qkv blocks start at 1,024 rather than 2,048, which is the trap the maintainer's comment names: a block boundary keyed on the stacked tensor's rows instead of the tensor's own.

One Mac keeps main's program, which is the review's point. The split merge takes the rank's head count through a buffer uniform — q27_attn_merge reads constant uint& heads (attention_mpp.metal:267, the argument at :270) — and a uniform argument is a host-side dispatch difference, so one shader cannot be both. attention.zig therefore registers a second instance beside the split one: q27_attn_merge_single (attention_mpp.metal:330) is q27_attn_merge minus the heads argument, and q27_attn_prompt_merge_single (attention_prompt.metal:175) writes PG * 4 where the split prompt merge reads PG * args.kv_heads (:170); the one-Mac prompt merge is the kernel whose comment says a single Mac's 24 query heads are known at compile time. encode chooses io[if (split) 4 else 10] and prompt.?[if (split) 1 else 2], and split is r.reduce != null — the same condition decode_round.zig:474 and forward.zig:127 already test — so an unpaired pass never binds the split instance and never sets the heads uniform (:408 runs only when split). Upstream creates no MTLFunctionConstantValues anywhere under zig/src or zig/kernels, so a function constant is not a mechanism this codebase has; a separate pipeline instance is.

The two instances are main's kernels, not lookalikes: compiled with the flags zig build gives these sources (metal -x metal -std=metal4.1 -fmetal-math-mode=safe -fmetal-math-fp32-functions=fast -c) and read with metal-objdump, q27_attn_merge from the parent tree and q27_attn_merge_single normalize to 329 IR lines each with 0 differing lines; q27_attn_prompt_merge and q27_attn_prompt_merge_single normalize to 125 lines each with exactly one line differing, the Q27PromptArgs pointer's dereferenceable(28) against dereferenceable(32). That difference is the four bytes of the int kv_heads member the split kernel reads; the single kernel never loads it, so its body is main's text with PG * 4 in the store. One Mac's prompt path keeps its arithmetic and only loses a uniform load.

A rank's verify never runs a tail. forward.zig's finish runs the head seam first and hands the reopened encoder back to the tail (zig/src/families/qwen27/forward.zig:230, :241); when the seam is head == .none it is skipped and the tail still runs, which is the one-Mac path. Because the head writes only this rank's vocabulary columns while the tail's matcher reads the whole row, decode_round.zig:221 refuses a round that has both with error.TailOnRank before the cache checks or any GPU work; zig/qwen27_bank_boundary_host.zig:53 pins that refusal with model-free stubs, and from step 3 on a pair refuses --drafter at the front door, so the two never meet.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: this commit (1af209fce92b1bcf4ccd848f99e97a301bb4306f) on main (94b90c9db65dcb71bbc39715901d6fb1f00407d0); no MLX or PyTorch; macOS 27.0.1 on an Apple M4 Pro (Mac16,11), Metal only (applegpu_g16, no tensor units), Apple metal version 32023.921. No checkpoint in this commit's gates: config_test and synthetic_graph build the graph from a synthetic config, and the model gates arrive in step 4.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a at this commit, nothing serves yet; the split is exact by construction and pinned by the host tests above — a rank's slice keeps the checkpoint matrix's K split, the fused gate-up keeps its member split, and every one-Mac stack keeps main's split (0 new problems in the lean run is the same diff's shape check, not exactness). The filtered contract runs below cover the kernels the split talks to; the limit checks pass (0 limit problems in each run's SUMMARY).
• Decode speed before and after, from tools/bench_openai.py: n/a, no serving path at this commit; the pair's decode rate needs two Macs and arrives with step 4's receipts.
• Prompt speed before and after: n/a; prompt projections keep the checkpoint matrix's split, so their reduction tree is unchanged, and a throughput number without a second Mac would be a loopback artifact.
• Tests run, with pass, fail and skip counts: zig build test: 576/576 steps succeeded; 8723/8775 tests passed (52 skipped) (run here with a cold local cache, so the counts are the whole suite; every step succeeded), exit 0. zig build test-cost-rule: 3/3 steps succeeded; 30/30 tests passed, exit 0 (run here with a cold local cache). zig build test-kernel-contract: 20/20 steps succeeded; 141/141 tests passed, exit 0. zig build dev-tools: 219/219 steps succeeded. zig build test-qwen27-bank-boundary-host: 7/7 steps succeeded; 4/4 tests passed, exit 0 (this commit's new test runs there). zig build test-qwen27-served-host and zig build tf-qwen27-tp-check: no such step at this commit (steps 3 and 4 add them). zig fmt --check build.zig zig: clean. The filtered contract runs: SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 34 ms, limits 1085 ms and SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 30082 ms, limits 123 ms. Lean rule: lean: 424 problems, 424 on the base, 0 new (tools/zig/lean_new.py, exit 0).
• Not run, and why: the model gates (they need a checkpoint and tf-qwen27-tp-check, which step 4 adds); a two-Mac run (the serving link is step 3 and the scripts step 5); the unfiltered kernel corpus (it stops at UNSUPPORTED metal moss image: the family runs on tensor units (M5) alone; its kernels unchecked here, so the runs above use --only); the series' two-Mac acceptance receipt, which was recorded before the series was rebased onto this base and cannot be re-run now that the second Mac is gone, and is quoted as dated in steps 4-5; CUDA, HIP/RDNA, M1-M3, M5 and Windows are not in this room.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. — the split rides the lane rounds the family already executes; no serving path is added in this commit.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. — no serving path at this commit; step 3 refuses drafts and step 4's fixtures are greedy, so there is nothing drafted to compare yet.
• No precision traded for speed. If the bits change, the description says which sums change and why. — the split moves bf16 rows and adds in fp32 once per seam; no bf16 partial sum is created, and a one-Mac model's Reduce is null, so its arithmetic is main's.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. — the two qwen27 attention kernels this commit edits have no contract record on main either; the edit is their head-stride argument, and it is pinned by the one-Mac split tests and by step 4's stage-3 fixtures rather than by a registration this series does not add. The pair's four records are already on main (they arrived with core: add the row-split pair seam and split simd_qmm over K #1432's squash, 3f0a715c).
• Prompt processing is no slower than the last release. — n/a, prompt projections keep the checkpoint matrix's split, so their reduction tree is unchanged; a pair throughput comparison belongs with step 4.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. — Metal on an M4 Pro (applegpu_g16); no CUDA, HIP, M1-M3, M5 or Windows.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. — config_test's shard spans and projection.zig's split tests fail on the parent commit (no Shard, no whole argument), and the model-requiring paths skip without a checkpoint as before.
• Comments and docstrings are one line. No measurements or history in the source. — lean: 424 problems, 424 on the base, 0 new.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines. — no user path, hostname or address is in the diff.

[ashhart]

The embedBound fix is a real bug fix with a test. Thank you.

Two places change the arithmetic, so the pair can't match the single Mac yet:

• The row-parallel split of o_proj/down_proj reduces across ranks. See core: add the row-split pair seam and split simd_qmm over K #1432 for an exact layout that keeps each sum on one rank.
• The rank-local K-split. projection.prepare picks the K-split from split(weight.shape[0], k), and rowView sets shape[0] to the rank's rows. So a rank's q/k/v, gate/up and z/b/a get a different split from the single Mac's. For example, gate goes from 17,408 to 8,704 rows and sk from 2 to 4; q goes from 12,288 to 6,144 and sk from 4 to 8. On M5, the m5Split/promptSplit tables are keyed on the global shapes, so a rank misses them. A different sk is a different fp32 order. Please carry the global matrix's split into the rank slice.

Stage 1 compares loadShard(ranks=1) against load on the same new code. It needs a comparison against main's binary too.

[akol1]

Refreshed head 7d57101a carries the global split into the rank slice, and the arithmetic now matches one Mac's:

• No cross-rank reduction. Each output projection is split by output rows; rank r owns 2,560 of the 5,120 hidden columns (8,704 of 17,408 for down_proj) and reads the full K. Every sum stays on one rank, and pair_gather copies whole bf16 rows with no arithmetic between them, so no cross-rank partial sum and no tie policy exists.
• The K split comes from the checkpoint matrix. projection.prepare and Linear.gn carry the global width into quant_gpu, so a rank's q/k/v, gate/up and z/b/a keep the split one Mac uses instead of recomputing it from the slice's rows: gate-up stays at 2 for 8,704 of 17,408 (not 4), q stays 4 and 8 for 6,144 of 12,288 (not 8 and 16), and a local 2,560-row down slice is keyed on neither the m5Split nor the promptSplit table. projection.zig and config_test.zig pin both directions.
• Stage 1 against main's binary. tp_check --stage 3 still compares loadShard(ranks=1) against load on the new code, and the separate cross-check in qwen27: pin the pair against the single Mac in the test suite #1435's receipt compares against main's own tensorfold-native built from 5897e0bc: it reproduces the same 256-token sha (a06fab34…) for the same 31-id chat prompt.

[ashhart]

Thanks @akol1. Carrying the checkpoint matrix's width (gn) into a rank's slice keeps the 4-bit K splits one Mac uses. The embedding-bound fix and config_test are good. On a local merge with main, fmt and lean are clean, config_test.zig passes 17/17, and the qwen27 host steps we ran pass. Four asks:

1. q27_attn_merge still strides 24 heads a row (zig/kernels/metal/qwen27/attention_mpp.metal:320,323). You fixed the same stride in the prompt merge. This kernel serves every multi-row round that is not a prompt chunk (attention.zig:343,410-413). A served chat feeds each earlier reply that way (qwen27_slots.zig:81,122, decodeChunk). On a rank, rows 1 and up land at twice their offset and read another row's gate. Please pass the rank's head count, as Q27PromptArgs now does.
2. One Mac's split changes. in_proj_qkv now loads as three blocks (gpu_weights.zig:222), and projection.zig:61 keys its split on the q block's 2,048 rows: 8 K slices, where main gives 4 (:145 pins 8). Widths other than 4-bit g64 on the M5 use that split (:71). Please key it on the tensor's rows. Add a host test that every one-Mac stack keeps main's split.
3. Before the M5, other widths take simd_qmm_bits, which chunks by the rank's width (quant_gpu.zig:217, simd_qmm_bits.zig:68). That gives 16 chunks for q, qkv and zba where one Mac takes 8. Please pass gn through, or refuse those widths on a pair by name, with a test.
4. Small: Part.k0/k, Triple.k/k0 and Shard.cols/*Cols are unused outside tests. Please drop them. Also, no contract record names q27_attn_prompt_merge, so please correct the body.

This also needs #1432's rebase.

[akol1]

Refreshed and force-pushed: qwen27/rank-slice is now b3d9b7e7, rebased onto current main (db3fd5ab), with the body rewritten around this head.

1. q27_attn_merge strides the rank's head count. It now takes constant uint& heads and indexes ((int64_t)node * (int64_t)heads + h) * D, with attention.zig setting it from the rank's geometry — the same argument Q27PromptArgs carries. The prompt merge writes the rank's KV node rows (kv_heads) instead of assuming the model's four. The path is: every multi-row round that is not a prompt chunk, including a served chat's earlier reply through decodeChunk.

2. The in_proj_qkv split is keyed on the tensor's own rows. A whole 6,144-row qkv block keeps its member split; the trap is keyed on rows rather than on the stacked width. projection.zig's tests pin it: "every one-Mac stack keeps main's K split", "a rank's row slice keeps the checkpoint matrix's K split", and "stacked gate-up keeps the member K split rather than recomputing it from stacked width". config_test pins the rank spans against the real shapes (12 heads, 2 KV heads, 8,704 intermediate rows, 124,160 vocab rows, 2,560 output rows, q rows from 6,144) and synthetic_graph runs the same graph on both ranks.

3. gn is carried through instead of refusing those widths. projection.prepare derives the slice from the checkpoint matrix, and simd_qmm_bits.call(..., whole) keeps the split rank's chunk count — the same through twinExact — so a rank's q, qkv and zba take the checkpoint matrix's K split (8 chunks) rather than the half width's 16.

4. Dropped Part.k0/k, Triple.k/k0 and Shard.cols/*Cols; they are not used outside tests. The body no longer claims a contract record for q27_attn_prompt_merge: the checklist now says the two attention kernels this commit edits have no record on main either, and this series does not add one.

Gates on b3d9b7e7: zig build test 557/557 steps succeeded; 1153/1172 tests passed (19 skipped); test-kernel-contract 43/43; dev-tools 215/215; zig fmt --check build.zig zig clean; pair SUMMARY 15 kernels: 15 pass gates 1-4, 0 fail, 0 known findings; 35 controls: 35 red as planted, 0 missed; 0 limit problems; gates 300 ms, limits 146 ms; simd SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 34032 ms, limits 121 ms; lean 428 problems, 428 on the base, 0 new. tf-qwen27-tp-check is not a step at this commit; step 4 adds it, and this head is rebased on #1432's new one.

[ashhart]

Thanks @akol1. All four are in on b3d9b7e7:

1. q27_attn_merge takes the rank's head count and the prompt merge the rank's KV heads. On one Mac those are 24 and 4, so main's indexing is unchanged.
2. The split keys on the tensor's own rows. in_proj_qkv's three blocks key on 10,240 rows (4 slices, 2 on the M5), as main's single tensor did, and the "every one-Mac stack keeps main's K split" table matches main's split() and m5Split() for all eight stacks.
3. gn reaches simd_qmm_bits through strided and the fused runs, and the twin check is keyed on it.
4. The unused fields are gone, and the body no longer claims a record.

Keeping b and a whole in a rank's zba, so its 3,168 rows stay on the tiled projection, is a nice touch. The M5 Ultra run set out on #1432 covers this step on the lane kernels, which no M4 can show.

[akol1]

Thanks — all four hold, and this is the commit where the two kernels you flagged stop being shared. On the rebased head bb00f2b0:

1. q27_attn_merge takes constant uint& heads (attention_mpp.metal:270) and the prompt merge writes PG * args.kv_heads (attention_prompt.metal:170), with attention.zig:408 setting heads from the rank's geometry. On one Mac the dispatch takes the other branch: a.io[if (split) 4 else 10] and a.prompt.?[if (split) 1 else 2], binding q27_attn_merge_single / q27_attn_prompt_merge_single, which are main's kernels. The AIR check and the exact dereferenceable(28)→(32) difference are in the core: add the row-split pair seam and split simd_qmm over K #1432 reply — the two single instances normalize to 329/329 and 125/125 IR lines, 0 and 1 differing lines.
2. The split keys on the tensor's own rows: in_proj_qkv still keys on 10,240 rows, and projection.zig's "every one-Mac stack keeps main's K split" table is unchanged. The qkv blocks start at 1,024, not 2,048, which is the trap your comment names.
3. gn still reaches simd_qmm_bits through strided and the fused runs, and twinExact is keyed on it; the one-Mac A/B below is that path on this Mac.
4. The unused fields are gone, and the body no longer claims a record.

Fresh receipts on bb00f2b0 (rebased onto dd0a9cc5): zig build test 568/568 steps succeeded; 6199/6233 tests passed (34 skipped), exit 0; zig build dev-tools 215/215; zig build test-kernel-contract 43/43; the pair filter SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 41 ms, limits 201 ms, the simd filter SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 40831 ms, limits 383 ms; lean 425 problems, 425 on the base, 0 new.

On the lane kernels: you are right that no M4 can show them, so what I can put behind them is the simd contract above (all four gates green on this Mac) and the one-Mac cold-prefill A/B at the series tip — the same before/after shape as your prefill cells, one GPU class down. Your M5 cells remain the receipt for the lane path itself.

[akol1]

Rebased onto the current main (7c068895) and force-pushed; the head is now 55865b017d4b4818290bea895247c3a63883f47c. The series order is unchanged, and these are re-run numbers from the rebased head, not carried over.

zig build test: 568/568 steps succeeded; 345/362 tests passed (17 skipped); zig build test-kernel-contract: 43/43; zig build dev-tools: 215/215; zig fmt --check build.zig zig clean; filtered pair run gates 33 ms, limits 108 ms, filtered simd run gates 30186 ms, limits 104 ms; lean rule 425 problems, 425 on the base, 0 new.

This room is one Mac now: the second Mac the pair receipts were recorded on is gone (the cluster was broken), so a fresh two-Mac pair_accept.py run is not possible here. The bodies keep that run labelled as a dated receipt from before the rebase and put the single-Mac loopback gate (tf-qwen27-tp-check) and the host tests next to it. No other question from the earlier comments is still open.

[ashhart]

Hi @akol1, a status update on #1433.

We read step 2's diff and found nothing that changes one Mac's output on the default path. The single-Mac merge
kernels are identical to main's.

#1432 landed tonight, slightly changed from your base (simd_qmm is now split over K). As a result, this branch
now conflicts with main in three places:

• core/contract/registry.zig: both sides extend the Metal record list. Keep both.
• core/simd_qmm.zig: your new chunks() tests need checking against main's split.
• qwen27/quant_gpu.zig: main keeps the simd projection block, with a new argument to row.simd.call, where your
  branch removes it.

Could you rebase onto current main? Once it's rebased, we'll run the M5 cells (chat, the multi-turn decodeChunk
case, a long prompt, prefill and the stream sweep) and the M3 Ultra cells, against main.

[akol1]

Rebased onto main (581cbe65, the head when you wrote; main has since taken 7a9fc417, which only changes tools/glm_index_emulate/emulate.cpp, and GitHub reports these branches conflict-free against it). qwen27/rank-slice is now d844977ee8b13ce440678da64d1fc5ffb74226a1, one commit on 581cbe65.

The three conflicts and how they resolved:

1. registry.zig: nothing left to merge. git diff 581cbe65 d844977e -- zig/src/core/contract/registry.zig is empty. core: add the row-split pair seam and split simd_qmm over K #1432's four pair records are on main in zig/src/core/contract/metal_pair_records.zig, and registry.all_metal takes them via ++ @import("metal_pair_records.zig").records; step 2 re-adds none, so "keep both" is main's list with no branch edit. No record lost.
2. simd_qmm.zig: main's call(p, n, k, rows, scalar, paired, whole) and chunks(n, whole) / splits(n) are kept as on main; step 2 adds six lines and one test. The test checks the pair form and main's form against main's split: chunks(3168,6240)==splits(6240), chunks(6240,0)==splits(6240), chunks(6144,12288)==8, chunks(5120,10240)==8, chunks(6144,0)==16, chunks(5120,0)==16 (zig/src/core/simd_qmm.zig:181, test "chunks follow output width only and tiles fit the reduction buffer").
3. quant_gpu.zig: main's simd projection block stays where main has it, and step 2 threads the new argument through it: row.simd.call(&k.simd.?, raw.n, raw.k, rows, scalar, paired, gn) with const gn = if (p.gn != 0) p.gn else p.n;, and const sk = if (k.prompt) projection.promptSplit(gn, p.k) orelse p.slices else p.slices;. On one Mac p.gn is 0, so gn == p.n and chunks(n, gn) == splits(n), the same split main's 0 selects; on a rank gn is the checkpoint width, so both ranks reduce with the same tree.

Numbers from this head, run here, not carried over: zig build test 572/572 steps succeeded; 8494/8545 tests passed (51 skipped); zig build test-kernel-contract 20/20 steps succeeded; 138/138 tests passed; zig build dev-tools 215/215 steps succeeded; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 36 ms, limits 127 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 32552 ms, limits 128 ms; lean rule 425 problems, 425 on the base, 0 new. tf-qwen27-tp-check is not a step at this commit; step 4 adds it.

The M5 cells (chat, the multi-turn decodeChunk case, a long prompt, prefill and the stream sweep) and the M3 Ultra cells are unblocked from this side: the branch is one commit on 581cbe65, and the rebase only touched the three files above. This room has M4 Pro hardware only, so those cells remain yours to run; the one-Mac default path is the diff you already read.

[akol1]

Rebased: qwen27/rank-slice is 579f9b45a116e6e7c5f750c95639de768bd99b4a, a single commit on main's 40cb15b4ebbbae4aa5ef028b264363bfd5a18524 (the tip when I rebased). That base carries the cost-rule retarget this branch used to have to bring itself: main's 5c9706f3 moved the 27B window timing into qwen27_costs.zig and left the case test asserting the old location, so zig build test-cost-rule was 29/30 on c39c09b9 (and zig build test with it), and 40cb15b4 points the pin at the file that owns the timing. So the rank slice is the whole branch, and it is the commit you reviewed.

The step-2 patch is unchanged: against the head you reviewed, every difference is main's advance (c39c09b9 to 40cb15b4, which includes that retarget), and git range-diff reports no hunk change for the commit itself. The three conflicts from your note therefore stay resolved exactly as answered earlier, and the M5 and M3 Ultra cells you planned carry over to this head unchanged.

Numbers from this head, run here, not carried over: zig build test 572/572 steps succeeded; 8522/8573 tests passed (51 skipped) (cold local cache, so the whole suite ran; every one of the 572 steps succeeded); zig build test-kernel-contract 20/20 steps succeeded; 138/138 tests passed (the retargeted case is part of zig build test; on a cold local cache zig build test-cost-rule is 3/3 steps succeeded; 30/30 tests passed here, 29/30 on c39c09b9); zig build dev-tools 215/215 steps succeeded; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 33 ms, limits 130 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 32425 ms, limits 125 ms; lean rule 425 problems, 425 on the base, 0 new. test-qwen27-served-host and tf-qwen27-tp-check are not steps at this commit; steps 3 and 4 add them.

This room has M4 Pro hardware only, so the M5 and M3 Ultra cells remain yours; the one-Mac default path is still the diff you already read.

[akol1]

Rebased: qwen27/rank-slice is 663bd7a0604a3042819e6886ea6768eb43b510b1, a single commit on main's 40cb15b4ebbbae4aa5ef028b264363bfd5a18524 (the tip when I rebased). That base carries the cost-rule retarget this branch used to have to bring itself: main's 5c9706f3 moved the 27B window timing into qwen27_costs.zig and left the case test asserting the old location, so zig build test-cost-rule was 29/30 on c39c09b9 (and zig build test with it), and 40cb15b4 points the pin at the file that owns the timing. So the rank slice is the whole branch, and its patch is the one you reviewed.

The step-2 patch is unchanged: git range-diff against the head you reviewed reports no hunk change for the commit itself, so the only thing that moved is main's advance, which includes that retarget. The three conflicts from your note therefore stay resolved exactly as answered earlier, and the M5 and M3 Ultra cells you planned carry over to this head unchanged.

Numbers from this head, run here, not carried over: zig build test 572/572 steps succeeded; 8522/8573 tests passed (51 skipped) (cold local cache, so the whole suite ran; every one of the 572 steps succeeded); zig build test-kernel-contract 20/20 steps succeeded; 138/138 tests passed (the retargeted case is part of zig build test; on a cold local cache zig build test-cost-rule is 3/3 steps succeeded; 30/30 tests passed here, 29/30 on c39c09b9); zig build dev-tools 215/215 steps succeeded; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 33 ms, limits 133 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 32394 ms, limits 125 ms; lean rule 425 problems, 425 on the base, 0 new. test-qwen27-served-host and tf-qwen27-tp-check are not steps at this commit; steps 3 and 4 add them.

This room has M4 Pro hardware only, so the M5 and M3 Ultra cells remain yours; the one-Mac default path is still the diff you already read.

[ashhart]

@akol1, a heads-up. #1726 landed just before this: on M5 the 27B now serves one stream through the lane loop by default, and the forward gained a tail hook (a verified head's readback work, encoded in the final command buffer). It conflicted with your head seam in forward.zig, and with the Stages fields in decode_round.zig.

We resolved it this way:

• On one Mac (reduce null) the tail runs exactly where Qwen3.8-27B on M5: the lane loop for one stream by default, and measured shared row prices (#1621) #1726 put it, so the default path is unchanged.
• On a rank (reduce set) the head seam runs first, then the tail encodes on the encoder the seam hands back.
• With head == .none the seam is skipped as before, but the tail still runs.

We're re-running the 27B with DFlash2 on an M3 Ultra and an M5, main against this, before it lands. Could you check one thing on your pair, since we can't: whether a round with a tail behaves on both ranks (drafted equal to "draft": false)? The tail reads the logits, and on a rank those are its vocabulary slice. If it shouldn't run there, a refusal when both are set would be the safe follow-up.

[akol1]

Rebased onto main at 94b90c9db; qwen27/rank-slice is now 1af209fce92b1bcf4ccd848f99e97a301bb4306f, the single commit of step 2 on that base. The base advance from 6f5dc1298 carried 30 commits; the two that touch this room are 2713b9ee0 (the qwen27 engine choice moved out of metal.zig into qwen27_engine.zig) and #1726's lane loop (q27-m5-lane-loop, merged at 24358c1e2). git range-diff against the chain you reviewed shows this step's commit differs from the one you read in exactly two places: the tail-on-rank guard and its model-free host test are new (your question, answered below), and the seam block in forward.zig's finish now sits under if (s.head != .none), so a seam that is head == .none is skipped instead of run, which is the shape this base's one-Mac path needs. Everything else in the patch is the one you read; where the new base moved a line underneath it, the range-diff above is the whole story.

Short answer: on one Mac the tail is exactly where #1726 put it, and on a rank the pair refuses the combination rather than guessing, so the two never meet. Matching the rule you gave: (1) a one-Mac tail runs exactly where #1726 put it — finish takes the seam's encoder and then if (s.tail) |tail| try tail.encode(tail.ptr, e); (zig/src/families/qwen27/forward.zig:241) runs the tail on that encoder, and with head == .none the seam is skipped at :220 while the tail still runs, which is the plain one-Mac path "draft": false uses. (2) On a rank the head seam runs first, and a tail, if one were set, would encode after it on the encoder the seam hands back (:230 then :241); in this patch that combination cannot be reached. (3) Both set on one rank is refused, which is the safe follow-up you offered: zig/src/families/qwen27/decode_round.zig:219 keeps error.BadVerifyTail and :221 returns error.TailOnRank when r.tail != null and r.reduce != null, before the cache-position check or any GPU work, because the head writes only this rank's 124,160 vocabulary columns while the tail's matcher reads the whole 248,320-row vocabulary. Upstream sets tail in one place only: zig/src/native/qwen27_lanes.zig:459-461, under fused (one window at 0, a draft, total > 1, no sampling, no masks), with std.debug.assert(r.tail == null) just before it; and from step 3 on the pair's front door refuses --drafter with refusePairFlags in qwen27_served.zig, so in a pair build the flags cannot produce the combination either. zig/qwen27_bank_boundary_host.zig:53 pins the refusal with undefined seam stubs and an unreachable body whose called flag must stay false; that test is a step of zig build test.

Run here, on this head, not carried over: zig build test 576/576 steps succeeded; 8723/8775 tests passed (52 skipped) (cold local cache, so the whole suite ran; every step succeeded, exit 0); zig build test-kernel-contract 20/20 steps succeeded; 141/141 tests passed; zig build test-cost-rule 3/3 steps succeeded; 30/30 tests passed (cold local cache); zig build dev-tools 219/219 steps succeeded; zig build test-qwen27-bank-boundary-host 7/7 steps succeeded; 4/4 tests passed; zig fmt --check build.zig zig clean; filtered pair run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems; gates 34 ms, limits 1085 ms; filtered simd run SUMMARY 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 30082 ms, limits 123 ms; lean: 424 problems, 424 on the base, 0 new (tools/zig/lean_new.py, exit 0). test-qwen27-served-host and tf-qwen27-tp-check are not steps at this commit; steps 3 and 4 add them.

The loopback gate that lands in step 4 has since run on this hop's descendant: the two-rank JSON reports "ranks_token_equal":true, "reference_token_equal":true, "first_difference":null, "seam_gave_up_rank0":0, "seam_gave_up_rank1":0, "global_vocab":248320, "local_vocab":124160, and no tail is in play on either rank.

This room has M4 Pro hardware only (applegpu_g16, macOS 27.0.1, metal 32023.921, no tensor units); the M5 and M3 Ultra cells in the matrix remain yours.

The PR body is refreshed with these same receipts.

[ashhart]

Thanks @akol1 for the rebase and the TailOnRank guard with its test. That answers the question exactly, and it's the version we'll land: your head with a changelog line on top.

The one-Mac check on an M5 Ultra, Qwen3.8-27B MLX 4-bit with DFlash2, main against this, eight alternating fresh servers:

• prompt tokens equal to main (0 of 20 rows differ);
• prefill +0.2 to +0.3% at 2k, 8k, 16k and 32k;
• decode after each prompt +0.3%, +0.2%, +1.2% and 0.0%;
• the stream sweep +0.1% at one stream and -1.6% at four, inside run-to-run spread (no new run sits under every main run).

A four-run pass earlier read 2.6% low at 2k. Its first main run was the outlier: main itself drifted from 159.9 to 153.8 over the run. The eight-run pass settles it.

The M3 Ultra run is waiting on that machine's Xcode tooling, which an overnight update broke. It lands once that's back.

[ashhart]

Thank you @akol1. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in 1.2.1, credited to you in the changelog and the release notes.

[ashhart]

Thanks @akol1! The second step of Qwen3.8-27B on two Macs, loading and running one rank's half, is in 1.2.1: https://github.com/ashhart/TensorFold/releases/tag/v1.2.1