[grantoverton]

On chips with tensor units, the 27B refuses every prompt pass over 128 rows through a mixed-width stack. An M5 prompt pass is up to 2,048 rows, so a checkpoint whose fused projections mix widths (such as the zba and kv stacks tf-qwen27-widths lists for the oQ4 and oQ2 checkpoints) fails any prompt longer than 128 tokens with BadCoreProjectionRows, an HTTP 500 before the first token. On an M5 Ultra, main answers prompts of 14 and 118 tokens and returns 500 on every longer one; with this change all of them answer. quantOne has the same 128-row refusal for a non-4-bit matrix on simd_qmm_bits tiles. This chunks both at 128 rows. Uniform 4-bit checkpoints never reach either path, and outputs stay byte-identical: on each chip, every prompt main answers gives main's tokens.

What this changes

One commit on main 3ab49e30:

1. quantRuns and quantOne dispatch a raw matrix 128 rows at a time (families/qwen27/quant_gpu.zig). quantRuns refused rows > 128 for the whole stack. Now its raw member takes 128-row chunks, each at its own input offset and at its rows' offset in the fused output, and its lane siblings keep their one-row launches. quantOne's non-4-bit tiled branch does the same at its own stride. Each chunk is a call these paths already make for a 128-row pass, and a row's bits do not depend on how many rows share its call (tf-qwen27-widths checks this in calls of 1 to 128 rows), so no output changes.
   Test that fails before: quant_rows_test.zig, part of zig build test. A 300-row pass through a 4/5-bit stack (every chip) and through a 5-bit matrix on simd_qmm_bits tiles (chips without tensor units) equals its rows run one at a time. On main the stack test fails with BadCoreProjectionRows on both chips below, and the tiles test fails the same way on the M3.

Exactness

Two machines: Mac Studio, M5 Ultra (64-core GPU), 96 GB, macOS 27.2 (26B123); and Mac Studio, M3 Ultra (80-core GPU), 256 GB, macOS 26.4.1 (25E253). Zig 0.17.0. tensorfold-native sha256 8c494705fae2ec2d… (main 3ab49e30) and 59751ffe2aa4ff7f… (this branch, deedab69), the same two binaries on both chips, served one after the other with the same settings: serve <checkpoint> --drafter z-lab/Qwen3.8-27B-DFlash2 --drafter-bits 4 --parallel 4 --context 32768 --prompt-cache-gib 0. The checkpoint is a local oQ4e quantization of Qwen3.8-27B (not published): 4-bit in groups of 64, with 5-bit out_proj on all 48 DeltaNet layers, 5-bit in_proj_a and in_proj_b on 23 of them, and a 6-bit lm_head, so both changed paths load. Drafter model.safetensors sha256 67fc76d68dc5a941…. Seven prompts, raw and chat, of 14, 118, 295, 296, 328, 607 and 1,521 tokens; greedy, at most 96 tokens, thinking off.

• M5 before and after. Main: the 14- and 118-token prompts answer; the five longer prompts return HTTP 500 BadCoreProjectionRows, drafted and "draft": false, 10/10. This branch: all 14 replies are HTTP 200, and the 14- and 118-token replies have main's token hashes, drafted and plain, 4/4.
• M3: before equals after. Without tensor units the attention plan caps a prompt pass at 128 rows, so main answers every prompt here. Every reply's token hash equals main's, 14/14.
• Drafted equals "draft": false, 7/7 on this branch on each chip, with partial acceptance (on the 14-token prompt, 94 of 166 drafted tokens accepted on the M5 and 82 of 122 on the M3).
• Concurrent equals solo. Four requests at once (1,521, 328, 296 and 118 tokens), twice on each chip on this branch, and on the M3 on main too: every token hash equals its solo run.
• Precision. No kernel, reduction order or accumulation changes: bf16 activations and fp32 group sums as before.
• Seams, cancellation and reset: this changes no state, cache or scheduler code, and they were not run separately.

Tests

• zig build native -Dcpu=apple_m1 -Dversion=1.0.0: pass (M3).
• zig build test test-golden -Dcpu=apple_m1 on deedab69 (M3): 587/587 steps succeeded, 0 failed. test-golden: 55/55 and 322/335 equal to server goldens, 0 unexpected differences. The qwen27 module: 139/139, the two new tests included; with the fix removed, the same module fails exactly those two (137/139). An earlier full run of the same change, before a one-line comment edit, ran 9,078 tests: 9,026 passed, 52 skipped, 0 failed.
• The qwen27 module's test binary on the M5: 138 passed, 1 skipped (the tiles test, which needs a chip without tensor units), 0 failed; with the fix removed, the stack test fails with BadCoreProjectionRows (137 passed, 1 skipped, 1 failed).
• python3 tools/zig/lean_check.py: 424 problems on this branch and 424 on main, zero added.
• git merge-tree against the open PRs that touch these files (qwen27: serve Qwen3.5-MoE (Ornith-1.5-122B) on Metal: routed experts, its DFlash2 drafter and the publisher's 4-bit weights #1125, qwen27: serve the 27B pair, refuse what a pair cannot keep, and die with the peer #1434, qwen27: pin the pair against the single Mac in the test suite #1435, qwen27: add the pair acceptance and supervision scripts #1436, qwen27: document two-Mac 27B serving and the row split's exactness #1437, qwen27 DFlash: compile the drafter head's lane pipeline at load #1734, qwen3_5_moe: the experts' prompt sort as one threadgroup (stacked on #1125) #1799): no conflicts.

Not run

• The full suite on the M5. Only the qwen27 module's tests ran there: zig build test test-golden -Dcpu=apple_m1.

• The public oQ4 checkpoint. The served runs used the local oQ4e above. With TensorFold/Qwen3.8-27B-oQ4 @fb60862 on an M5, main should return HTTP 500 and this branch should answer:

  zig-out/native/bin/tensorfold-native serve <Qwen3.8-27B-oQ4 folder> --name local-model --port 8080 \
    --drafter <Qwen3.8-27B-DFlash2 folder> --parallel 4 --prompt-cache-gib 0
  python3 -c 'import json; print(json.dumps({"model": "local-model", "max_tokens": 64, "temperature": 0, "prompt": " ".join(f"Note {i}: aisle {i % 7 + 1} received {i * 3} crates." for i in range(1, 40))}))' > long.json
  curl -s -w '\n%{http_code}\n' http://127.0.0.1:8080/v1/completions -H 'Content-Type: application/json' -d @long.json

• Speed. On the M3 every pass makes the same calls as main, and on the M5 main has no before for a pass over 128 rows (it refuses), so nothing was measured. M5 prefill after the fix: python3 tools/bench_openai.py http://127.0.0.1:8080 local-model --tokens 256 --reps 3 --temperatures 0 --label candidate --output decode.json, with the 2k, 8k, 32k and 64k cold-prefill arrays at --prompt-cache-gib 0.

• Fidelity and FP64 references. No arithmetic changes, so teacher-forced fidelity and per-class FP64 references were not rerun.

[ashhart]

Thanks @grantoverton, a clear find. A mixed-width 27B on an M5 failing every prompt over 128 tokens is a bad first
impression, and the fix is the smallest one: each chunk is a call these paths already make, so the bytes can't
move. The before/after on the M5, the M3's 14/14 equal hashes and the test that fails on main are what we'd want.
We'll run the public Qwen3.8-27B-oQ4 on our M5 Ultra and an M3 Ultra (main's 500 against the branch's answer, and
token hashes where main answers), then land it.

[ashhart]

Thank you @grantoverton. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @grantoverton! Mixed-bit Qwen3.8-27B checkpoints on M5 now answer prompts longer than 128 tokens in 1.2.1: https://github.com/ashhart/TensorFold/releases/tag/v1.2.1