[linson007]

Historical source / current-main reconciliation: upstream has merged #487 and the later runtime stack at 1482f6bbc2abf12375332725302019594150b6cb. This draft preserves the original source and scoped evidence; its old runtime flags are not current-main settings. The mergeable qualification tools, updated source audit and consolidated evidence are now in #523. New-source validation and remaining awake replacement runs are in progress.

This implements the pending-tape GDN experiment in step 3 of #508, on top of #507/#509. SPLASH_LIVE_GDN=1 separately opts eligible greedy MoE verification into shorter current-row GDN arithmetic. The flag defaults off. The local experiment is complete: correctness passes, but the performance promotion gate is not met. This remains a draft reference implementation with the flag off; it is not a request to promote or merge the runtime optimization.

Review scope

Review the additional implementation commit 46a074ee after measurement base bc09da81, plus the GDN design note. Earlier commits belong to #507/#509. The developer CLI/oracle accept --live-gdn on|off to isolate this change from MoE routing at the same threshold. The hardware package is separately reviewed in #510.

Why pending tapes matter

Current main's state cell holds the state before the current step, after consuming the previous step's retained pending tape. Current prefix length cannot limit that previous fold. Pending 8/current 1 must consume all eight pending rows, copy the correct convolution history, and leave the same committed state as full execution.

The live variants preserve all storage extents and offsets. They load all retained pending rows, skip inactive current-row convolution/norm/gate/tape work, and limit only the current recurrence loop. All eight SIMD groups still scan their recurrent-state dimensions and reach both barriers. Inactive hidden/shared rows are zero, so full-width prepared-table consumers receive defined data. Unwritten tape suffixes are never retained. Invalid counts conservatively run all eight rows.

Optional metadata is appended to GdnDecodeBuffers; ordinary buffer bindings and kernel entries stay available. gdnLiveRows is separate from the MoE switch, so the incremental experiment keeps the same prefix policy. First live graph construction at each width also prepares ordinary GDN fallback variants without submitting a second graph or changing state parity. The real-model oracle caught missing fallback preparation in the first version; the corrected path passes its sampled-request pipeline-cache invariant.

Completed correctness validation on M5 Pro

• The shader-validated GDN suite passes 128 geometry/width/norm/state/head-order/plain-or-Table16 combinations, each exercising 130 steps. It checks every prefix, pending 8/current 1 and reverse transitions, both parities, poisoned packed/tape suffixes, invalid-count fallback, zero inactive hidden rows and independently prepared tables.
• Active hidden rows, live tape fields and the entire folded state match the full shader bit for bit. The full shader also passes the existing independent CPU recurrence/prefill-gate reference and changing/permuted-lane replay tests.
• All four real-model oracle runs pass with shader validation: Qwen3.6 MoE GGUF IQ3_XXS and MLX 4-bit safetensors, each with FP32 and BF16 recurrent state and INT8 KV. These exercise restore/materialization, disk state, preemption, changing widths, masks, sampled reproducibility, thinking transitions and EOS/length limits under the existing numerical contract.
• Production/profiler/oracle builds and architecture/diff checks pass. Other GPU families and BF16 KV combinations are not claimed by these four runs; Prepare pinned Qwen hardware qualification and adaptive-policy controls #510 provides those repeatable qualification jobs.

Completed performance qualification: keep off

The portable M5 Pro evidence bundle includes 4,096 native cycle samples, all 24 ABBA width comparisons, production HTTP reports, disabled-path controls, model/source/binary hashes, exact commands and all five passing shader/oracle logs. Generated response text is removed while output hashes, usage, counters and environmental measurements remain.

At fixed threshold 0.1, GGUF native gains at B1–B4 are 0.11%, 0.55%, 0.45%, 0.66%. At threshold 0.2 they are 0.55%, 0.95%, 0.83%, 0.90%. After a full unmeasured warmup, the longer 64-cycle MLX repeats gain 0.54–1.20% across both thresholds and widths, with noise at most 0.90%. The initial, noisier MLX series is retained; no cell exceeds its noise-adjusted gate.

Production B1 HTTP code runs (2k/8k context, three trials each, 512 output tokens) complete with matching output hashes and sampled/constrained fallback probes. Gains range from -0.47% to +1.32%; none clears its threshold. The same-threshold disabled-binary control has two stable trials with 0.25–0.26% regressions, within the 2% floor, and one noisy/inconclusive trial. No noisy trial is counted as a pass.

The promotion rule is useful output throughput gain > max(2%, 2 * measured noise), with spread >5% inconclusive. The native metric is fused command wall time per committed token; the HTTP metric is complete native decode-cycle time per output token. The source leaves pending-state traffic and all fixed-shape projections intact, which limits the whole-cycle opportunity. The 2k/8k HTTP series used the earlier startup warmup, not the later sustained-warmup protocol; it supplies no positive speed claim.

Decision: step 3 has a tested implementation and a measured negative promotion decision on this M5 Pro. Keep SPLASH_LIVE_GDN=0 and leave this PR in draft. Other Apple GPU families and BF16 KV model-oracle combinations remain unexecuted; #510 prepares repeatable maintainer runs. No shared-default claim is made.

All four GitHub CI jobs passed for implementation commit c6c2abd3 (production/native/sanitizers and Python 3.12/3.13/3.14). The appended evidence commits change documentation/data only. Full invariants and reproduction commands are in dev/benchmarks/gdn-live-prefixes.md.

[linson007]

Closing the historical GDN arithmetic prototype with a measured negative promotion decision; consolidated evidence and current-main tooling are in #523.

On its frozen source, shader validation passes 128 geometry/width/norm/state/head/preparation combinations with 130 steps each, and four real-model storage/state oracles pass. All 4,096 native cycle records and HTTP/disabled controls are retained. None clears the agreed complete-cycle/serving promotion gate. The flag remains disabled in the historical branch.

Current main also adds a third packed geometry and sigmoid gate outside this prototype's coverage. No GDN pruning patch is applied in #523. The archived branch, invariants, exact commands, sources, model/build hashes and numerical logs remain available; passing current-main BF16-KV oracles is not claimed as coverage of this absent historical pruning feature.