[Bench][SM70] Make the Qwen3.8 no-MTP baseline reproducible from source

[yangzhuxinyzx]

Purpose

Deliver the accepted quality-repaired Qwen3.8 NVFP4 single-request TP4 baseline as a reproducible current-source build and bounded benchmark. User-requested integration audit, not a new performance/precision experiment.

Base: 95205a2d9952813aa7469f63ff65b8f2813c027a (main, #525). Core HC #506 and QSA/router #507 commits are already ancestors of main. Existing open #507 targets the old HC feature branch; re-merging it is unnecessary. #510 and FlashInfer/E4M3 drafts are not accepted baseline dependencies and are intentionally excluded.

This is not duplicate work: the prior kernel PRs contain the production implementations, but the validated launch depended on untracked HC/NVFP4 wrapper DSOs and a local import bootstrap. This PR publishes those build bindings, a source/binary manifest, the explicit baseline contract, and the actual quality/speed reproduction entry point.

Implementation / numerical scope

• Build HC, NVFP4 W13/W2, QSA, FlashQLA and repaired Flash-V100 from this checkout. No production kernel algorithm changed.
• Keep opaque HC communicator creation and operations in one DSO.
• Verify source hashes, complete tracked production-tree fingerprint, runtime library hashes and worker mappings.
• Publish an opt-in source-overlay bootstrap; this is not a full native wheel rebuild. Compatible standard SM70 native extensions remain explicit, fingerprinted dependencies.
• Preserve native FP32 SSM, FP16 activations/KV, checkpoint NVFP4; TP4/PP1, V2, dual prefill/full-decode graph, 262144 context, chunk8192, maxseq1, disk-mmap prefill/pinned-UVA decode, no MTP or prefix cache. Do not enable online QPN8, approximate LM-head or experimental batch sidecars.

Test plan / current results

• CPU-only source build: all six libraries compiled with Torch2.10.0+cu128, nvcc12.0.140, SM70. No shared environment install.
• CUDA_VISIBLE_DEVICES= <project-python> -m pytest -q --confcutdir=tests/benchmarks tests/benchmarks/test_sm70_qwen38_baseline.py: 11 passed. Includes tampered/incomplete bundle rejection, exact precision contract, successful and failed driver cleanup.
• pre-commit run on scoped staged files: passed. An initial standard re import was corrected to the repository's required regex.
• Environment and engine arguments compared with accepted sweep: exact match except read-only worker manifest class; fixed8192 prompt SHA matches 220ffafa68b25b7d632b762d17beb292f8ada1351ae5105954b490749015e997.
• Fresh operator acceptance passed: HC raw-bit256 graph replays/16 patterns/eight alignments, zero mismatches on every TP rank; router1280 bitwise rows; QSA72 exact cases, dynamic128 graph replays/1536 row comparisons. Built from this checkout, not frozen DSOs.
• One full-length model initialization, native FP32 SSM confirmed in all four workers, freshly built overlays mapped and hashed. Two warmed fixed8192 and261631+513 repeats each; all complete513-token outputs and warmups match the corresponding prior native-state sweep exactly. Natural arithmetic, record copy and middle-context retrieval pass (88/110/172 output tokens, natural EOS). The262143+1 boundary passes (50.97670s prefill). All workers exited and released GPUs.

Fresh unprofiled end-to-end result

Initial tested source b2042fb24b78bba131ec9aa0b1a05cdf3f54df60. Physical V100 GPUs4-7, same group as the comparison sweep. TP4/maxseq1, max262144, no MTP/prefix cache, native FP32 SSM/FP16 KV, hybrid PLE and dual graph unchanged. Greedy513-output timing is separated from official-sampling/natural-EOS health checks. Main subsequently advanced; see the separate integration gate below rather than treating the initial numbers as a newer-source measurement.

Input	Previous decode tok/s	Fresh decode tok/s	Previous prefill tok/s	Fresh prefill tok/s	Fresh TPOT ms
8192	97.826	97.72461	6936.60	6950.16	10.23284
261631	73.977	73.98534	5137.10	5169.99	13.51619

Two warmed repeats per row. Both new complete output sequences match their old reference and warmup exactly. No approximate/lower-precision path was introduced to recover speed. This reproduces the accepted approximately98tok/s short baseline; it does not claim100tok/s or reduced long-context decay. Fixed repetitive-prompt prefill is not a universal conversational/dataset speed.

Production-tree fingerprint: 110d349cd3be7f30b1cee724567b391598295f5740ab93ef15c0f44fc227aaed.
All four workers report FULL_AND_PIECEWISE, QSA specialization1, matching current-checkout imports, freshly built/mapped HC/QPN/QSA/FlashQLA/FlashV100 libraries and identical compatible native-base hashes (documented publicly).

This bounded source-reproduction gate does not certify arbitrary inputs or close the broader GDN/W13 actual-input numerical-reference audit. No independent human review or complete from-scratch native wheel rebuild is claimed.

Latest-main integration gate

During the first acceptance, main advanced to a193c285d4 through #517. We merged it normally into this owned branch: b6cb3f10896b850de6b3c6655b7e53a1316f1cae. No manual conflict resolution or runtime patch. The PR diff versus current main is still only the nine public reproduction files.

The incoming changes touch shared sampling and NVFP4 alignment as well as DFlash2-only/gated code. Therefore the initial model result is not relabeled as latest-main evidence. The builder was rerun incrementally; HC/QPN/QSA/FlashQLA/paged utility binaries stayed identical, FlashV100 rebuilt from the updated source. New production fingerprint: ca473d5f1dbd628597ea4e48c038c63b01415744dbad602958efbb3464f45175.

The14 focused shared top-k/top-p tie/capture tests pass on V100 in16.11s. A second, short-only model instance passed on actual source b6cb3f10896b850de6b3c6655b7e53a1316f1cae:

Latest-main8K metric	Run1	Run2	Mean
Pure decode tok/s	97.64778	97.72422	97.68600
Prefill time s	1.19615	1.17787	1.18701
Prefill tok/s	6848.64	6954.92	6901.78
TPOT ms	10.24089	10.23288	10.23688

Both complete513-token outputs and warmup match the original native-state reference exactly. Natural arithmetic/copy pass; all four worker manifests retain native FP32 SSM/FP16 KV/full-decode graph/new-library mappings. Result complete=true; workers and PLE exited and all four test GPUs returned to7MiB. No resident API.

No duplicate full256K/HC/QSA campaign; no DFlash2/FP32-logits/E4M3 route is enabled in our contract. The near256K result remains attributed to initial b2042, not misrepresented as a repeated b6 measurement. Final b6 CI is green (pre-commit4m49s, pre-run-check passed). The latest-main runtime has therefore reproduced the approximately98 decode/7K prefill short baseline without lowering precision.

Reproduction / artifacts

See docs/design/sm70_qwen38_reproducible_baseline.md for public build/run commands and limitations. Generated manifests, build logs and detailed results are retained task-locally, not committed; no model weights, tokens, personal absolute paths or binaries are included.

Disclosure

AI assistance: OpenAI Codex performed implementation, audit and tests at the repository owner's request. No independent human review is claimed by this report.

[yangzhuxinyzx]

Post-merge verification complete: public/main is 099d984. Its complete Git tree is exactly identical to the model-tested integration head b6cb3f1 (both tree 45e2c718d061f1cf1a70a3acff1e9fd7ad453860; git diff exits 0). The latest-source 8K result is therefore attached to identical merged source: 97.68600 decode tok/s, 6901.78 prefill tok/s, complete 513-token reference agreement, natural health and focused sampler tests passing. No extra model startup for the merge commit. All task-owned model/PLE workers have exited; no API or GPU residency remains. Existing source-overlay/native-base and bounded-quality limitations still apply.