Repository navigation
[WIP] Sync #302 with latest main - #367
Merged
yangzhuxinyzx merged 129 commits intoAug 27, 2026
Merged
yangzhuxinyzx merged 129 commits into
yangzhuxinyzx merged 129 commits into
Conversation
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Bumps [sentence-transformers](https://github.com/huggingface/sentence-transformers) from 5.2.0 to 6.0.0. - [Release notes](https://github.com/huggingface/sentence-transformers/releases) - [Commits](huggingface/sentence-transformers@v5.2.0...v6.0.0) --- updated-dependencies: - dependency-name: sentence-transformers dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.4.0 to 12.3.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@10.4.0...12.3.0) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [black](https://github.com/psf/black) from 24.10.0 to 26.5.1. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](psf/black@24.10.0...26.5.1) --- updated-dependencies: - dependency-name: black dependency-version: 26.5.1 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
* [Model][SM70] Prefer accurate PP2 TP4 QPN8 schedules Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit a253e04) * [Doc] Record DeepSeek V4 QPN8 accuracy audit Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit bdc82e8) * [Model][SM70] Preserve bounded QPN8 speed schedules Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> --------- Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
* [Kernel] Tune sparse MLA head grouping for TP4 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit 4397e7a) * [Kernel] Fuse TP4 Q transform and KV insertion Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit 19e64ab) * [Kernel] Broadcast MXFP4 W13 input for batch one Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit 089feea) * [Kernel] Remove redundant MXFP4 top-6 sorting Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit d1e8c1c) * [Kernel][SM70] Default audited PP2 TP4 hot paths Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> * [Kernel][SM70] Align compact decode defaults Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> --------- Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
* [Kernel][SM70] Prescale DeepSeek V4 FP8 decode Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> (cherry picked from commit b8d928d) * [Kernel][SM70] Default safe FP8 prescaled M1 decode Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> --------- Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Upgrade SentenceTransformers, Pillow, and Black with a regenerated CUDA test lock; add the new pytokens dependency and repair the CrossEncoder model_kwargs API. Keep chardet 5.2 because mbstrdecoder currently requires chardet below 6. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Track PREFIX_ controls in retained SM70 benchmark environments so memory, scheduling, latency, and numerical configurations remain reproducible. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Make the SM70 sampler target use the stable Python ABI so single-arch Volta source builds package the expected .abi3.so module. Reported-by: Andrew Leech <andrew@alelec.net> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Refresh the CUDA test lock to mbstrdecoder 1.1.5 and the highest chardet release permitted by upstream metadata, 6.0.0.post1. Reported-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Repair Dependabot’s broad update on current main with consistent source inputs and generated locks, preserving the validated Torch/tokenizer/Pillow stack and synchronizing optional dependency consumers. Reported-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Repair #322 on current main, retaining only the measured FP13 kernel and adding tensor-value, CUDA-device, shape, dtype, layout, memory, quality, and rollback safeguards. The production path defaults on without model/checkpoint identity admission. Original-commit: 165ec09 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Record the source-matching Qwen3.8 exact-8K baseline, request-wall target, trace budget, bounded shard-consumer design, rejected routes, quality ladder, and rollback requirements so implementation work does not repeat known dead ends. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Prevent M2-M8 active-expert compaction from poisoning B1 direct-order graph capture by selecting the immutable one-row slot offsets. CUDA kernels, arithmetic, route order, defaults, and rollbacks remain unchanged. Reported-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> Original-commit: 052312c Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Freeze the public repository at the exact released v1.3.0 source tree before archiving public development. Ongoing work was mirrored to the personal private repository first; history is preserved without force-pushing. Target-tag: v1.3.0 Target-tree: 4373fb8 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Bumps [webcolors](https://github.com/ubernostrum/webcolors) from 24.11.1 to 25.10.0. - [Changelog](https://github.com/ubernostrum/webcolors/blob/trunk/docs/changelog.rst) - [Commits](ubernostrum/webcolors@24.11.1...25.10.0) --- updated-dependencies: - dependency-name: webcolors dependency-version: 25.10.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pandas](https://github.com/pandas-dev/pandas) from 2.2.3 to 2.3.3. - [Release notes](https://github.com/pandas-dev/pandas/releases) - [Commits](pandas-dev/pandas@v2.2.3...v2.3.3) --- updated-dependencies: - dependency-name: pandas dependency-version: 2.3.3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [wrapt](https://github.com/GrahamDumpleton/wrapt) from 1.17.2 to 2.3.0. - [Release notes](https://github.com/GrahamDumpleton/wrapt/releases) - [Changelog](https://github.com/GrahamDumpleton/wrapt/blob/develop/docs/changes.rst) - [Commits](GrahamDumpleton/wrapt@1.17.2...2.3.0) --- updated-dependencies: - dependency-name: wrapt dependency-version: 2.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Source-audited against private main b06f54c. Includes strict route/shape admission, bitwise-equivalent long-context kernel optimizations, static-gate fixes, default-on promotion, and explicit rollback. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Extends the quality-valid max-aware D256 GQA architecture to KV256K, removes the rejected raw-logit half2 experiment, preserves explicit rollback, and retains the recorded exact output hash with 19.5% prompt-throughput gain. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Default the audited non-fused shared-gate QPN8 and exact TP4 six-route MXFP4 QPN M1 paths with explicit rollback and old-extension fallback. Preserve the recorded PP2 x TP4 endpoint, rejected-path evidence, and long-context design scope. Tests: Ruff 0.16.4; clang-format 21; git diff --check; 39 passed, 4 skipped policy tests; recorded matching-extension suite 41 passed. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Record the route-hit 73.645-token combined endpoint and passing quality gate, while identifying the FP8 route change that prevents assigning its delta to MXFP4. Continue follow-up work from the merged private main. Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Keep the numerically sensitive shared-expert gate/up QPN8 route opt-in after its matched GSM8K endpoint regressed one item. Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Keep the route opt-in until its matched full-model speed and quality gate closes. Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Release per-expert conversion temporaries before pointer table construction and build strided pointer tables directly in torch-managed storage. This avoids unchecked cudaMallocAsync failures when V100 model loading exhausts driver headroom. Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…20260826-124851 [Bugfix][SM70] Audit DFlash2 output quality
…e-20260826-173751 [Bugfix][SM70] Restore DFlash2 prefill closure
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…gates-20260827-163817 [Bugfix][SM70] Repair #345 Qwen4Exp integration gates
…main-sync-20260827-164533 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> # Conflicts: # tests/benchmarks/test_benchmark_sm70_decode.py
…260827-164533 [CI][SM70] Sync #345 with latest main
…4-20260826-140311 [Model][SM70] Adapt Qwen3.8 Flash Next NVFP4
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…ill-20260827-000848 [Perf][SM70] Optimize Qwen3.8 Flash Next prefill
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…1-glm53-20260828-010541 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> # Conflicts: # csrc/ops.h # csrc/sm70_turbomind/ops/awq_sm70_gemm.cu # csrc/torch_bindings.cpp # docs/design/sm70_v100_migration_control.md # tests/model_executor/test_weights_mapper_stacked.py # tests/v1/worker/test_kv_block_zeroer.py # vllm/_sm70_ops.py # vllm/config/vllm.py # vllm/model_executor/layers/quantization/modelopt.py # vllm/model_executor/layers/quantization/nvfp4_sm70_moe.py # vllm/v1/core/kv_cache_coordinator.py # vllm/v1/core/kv_cache_utils.py # vllm/v1/kv_cache_interface.py # vllm/v1/worker/utils.py
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…0828-010541 [Bugfix][SM70] Reconcile GLM-5.3 with latest main
…826-163805 [Model] Add GLM-5.3-Flash NVFP4 support on SM70
…-20260828-012548 [Bugfix][SM70] Harden #361 default fast paths
…4-20260826-140311 [Kernel][SM70] Reach 82 tok/s on Qwen3.8 Flash Next NVFP4
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…20260828-0154 [WIP] Repair #355 against latest main
…827-152119 [Core][SM70] Add lookup-augmented DFlash2 drafting
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
The audited QPN8 functionality is already present through merged PR #313. Preserve the historical commits without replaying the stale policy or tests. Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
yangzhuxinyzx
marked this pull request as ready for review
August 27, 2026 18:26
yangzhuxinyzx
merged commit Aug 27, 2026
11b2411
into
agent/v100-dsv4-pp2tp4-qpn8-accuracy-20260826
2 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Sync superseded PR #302 to current main without replaying its stale stacked source. Its validated QPN8 schedules and shared gate/up fallback were rebuilt and merged through #313.
Test Plan
Test Result
035af5d4fbe635cf4e4650b792fac3bf71e76e32has tree6cf99a61fb720978b84134b09ddf58ffd5eb96ca, byte-identical to current main.