[linson007]

Commit attribution was corrected to linson007 with a metadata-only history rewrite. File trees are unchanged; historical benchmark source hashes and CI links below retain their original recorded scope.

Qwen performance comparisons need to distinguish useful decode gains from timing noise, cache reuse, or serialized prefill. This adds an HTTP ABBA benchmark that checks the effective verification policy and requires actual native prefill/continuation progress alongside an active decoder.

The final diff is nine tooling, test, and guide files against main 1482f6bbc2abf12375332725302019594150b6cb. This benchmarks existing behavior; it introduces no production runtime, kernel, scheduler, model API, or default changes and claims no new inference speedup. Related research and returned hardware results are tracked in #508. The separate penalty-oracle correction remains #526.

Behavior to review

• Set --verify-rows all|auto independently for baseline and candidate, then check native /status.verify_rows. Dense auto may correctly report all. This compares complete product settings: all also disables the adaptive fitter.
• Keep one server resident for each complete ABBA round, with matched warmups/workload history, when two models do not fit. Paired residency remains available when capacity permits.
• For mixed workloads, prove native long-prefill/continuation progress while the original request decodes. Concurrent HTTP lifetimes or stale status cannot establish overlap. Require a fresh initial snapshot; retain/count later stale polls without accepting them as evidence.
• Check full requested output budgets, actual cache reuse and released resources. Report useful-token throughput, short/long TTFT, streaming gaps, memory/pressure, counters and hashes separately.
• Use scoped macOS wake prevention, release it on failure, and reject elapsed UTC/awake-clock discrepancies of at least one second. Throughput gains must exceed max(2%, 2*noise); spread above 5% is inconclusive. A regression pass is not a gain.

Review map and reproduction

• dev/benchmarks/serving_regression.py: serving comparisons and validity checks.
• dev/benchmarks/qualification_matrix.py and qwen-qualification-models.example.json: prepare or execute serial correctness/serving runs; preparation marks every job unexecuted.
• dev/benchmarks/trace_report.py: trace validation/summary and shared percentile calculation. The trace CLI consumes the archived experimental format; ordinary production binaries do not emit it, and --trace refuses unsupported binaries.
• dev/tests/tools/test_serving_profile.py and test_qualification_matrix.py: 20 focused regression tests.
• dev/benchmarks/serving-measurements.md and qwen-maintainer-runs.md: commands, compatible build/model requirements and measurement contracts. qwen-serving-experiments.md provides a concise research summary and archive links.

Run the focused checks:

.venv/bin/python -m unittest \
  dev.tests.tools.test_serving_profile \
  dev.tests.tools.test_qualification_matrix -v

Prepare the example matrix without loading models or starting servers:

.venv/bin/python -m dev.benchmarks.qualification_matrix \
  --models dev/benchmarks/qwen-qualification-models.example.json \
  --baseline build --candidate build --output build/qwen-qualification

Replace the example model paths and freeze compatible binaries before adding --execute. The default example prepares 36 jobs; preparation is not hardware coverage or a test pass.

Validation of narrowed head c8a09f3f

• All 20 focused tests pass. Compileall, Ruff lint/format and patch whitespace pass.
• All three CLI help commands work. The matrix CLI prepares 36 explicitly unexecuted jobs.
• Every local documentation link resolves; all 170 files removed from the final diff are preserved in the published archive snapshot.
• Retained benchmark behavior is unchanged from the previously validated 3e8911fc: the matrix, trace analyzer and two test modules are byte-identical; only the serving module's introductory documentation changes.
• Full local env -u HF_HUB_OFFLINE make -j4 check-python: PASS, 1,338 tests in 543.642 seconds with 27 skips, compileall, Ruff lint and formatting (164 files). The native build prerequisite passes.
• Fresh CI for this head is running; it is not recorded as a pass. The opt-in release hardware job is skipped. Earlier green CI and model/performance runs retain their original revision scope.

Research archive and merge scope

The full investigation is preserved at immutable commit 3e8911fc89a6ed009e7b0561bfa7c23e58ea594f, also published as tag archive/deepinfer-qwen-m5-pro-20261021 in linson007/splash. Start with the decision ledger. Historical gains, failed promotion gates, excluded captures, raw reports, traces and legacy confidence/prefill analysis tools remain there, linked from #508. Their runtime prototypes are not part of this final diff, and old measurements do not establish a speedup over current main.

Original branch history is retained. Squash merge the final nine-file diff to keep the earlier artifact commits out of main's history.

[linson007]

Rechecked against main 1482f6bbc2abf12375332725302019594150b6cb after the maintainer comment on #280/#283.

This branch already contains that exact main: 11 commits ahead, 0 behind. No merge/rebase or new timing run is needed because neither its source nor main changed since the completed validation. All four CI jobs remain green on 3e8911fc.

The #487 overlap is already accounted for: this PR adds supported all/auto qualification tools and source-pinned evidence, with no production runtime or obsolete threshold/GDN flag. The archived prototypes remain historical experiments, and their results are not current-main default qualification. #526 is a separate test-only oracle correction, not an adaptive verification implementation. The linked closure does not invalidate this tools/evidence contribution.