Skip to content

[Kernel][SM70] Pipeline batched E5M2 XQA decode - #269

Merged
yangzhuxinyzx merged 1 commit into
mainfrom
agent/v100-xqa-pv-software-pipeline-20260823-115300
Aug 27, 2026
Merged

yangzhuxinyzx merged 1 commit into
mainfrom
agent/v100-xqa-pv-software-pipeline-20260823-115300

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

Continue the SM70/V100 batched long-context XQA decode work after #268. This round profiles the remaining E5M2 conversion, shared-memory publication, and QK/PV consumer dependency chain, then admits only one narrow software-pipeline or vector-movement change with measured benefit.

  • Base: onecat/main@675a12dedcca8cc020c033f1b1d0f1751d4b8efe
  • Current head: 8808f8e239
  • Target: Qwen3.8-27B-FP8, TP4 V100, no MTP, FP8 E5M2 KV, prefix cache, Mamba align, Flash-V100
  • Dataset/sampling: SPEED-Bench low_entropy and GSM8K; temperature 1.0, top-k 20, top-p 0.95, natural EOS

Starting bottleneck

The accepted #268 B16 ragged 17.8K kernel still has 30.10% long-scoreboard stall, 0.65 eligible warps/scheduler, 41.60% issue active, and 2.19% tensor active. Occupancy is only 18.16%, but #268 proved useful work can increase without raising occupancy: it removed 41.5% of L1 global-load requests and 14.7% of instructions while leaving HBM bytes and occupancy unchanged.

Pending work and gates

  • exact-shape phase/PC attribution for K load+E5M2 conversion, shared publication, QK, V load+conversion, PV, and synchronization;
  • one minimal default-off candidate selected from counters, not an occupancy assumption;
  • CUDA 12.8 SM70 build plus SASS/register/stack/spill audit;
  • bitwise randomized-page operator A/B for uniform/ragged B4/B8/B12/B16 and boundary contexts;
  • FULL graph route proof and five-repeat B8/B16 1K/16K pure-decode endpoint A/B;
  • official-sampling GSM8K health and accuracy gate.

This PR remains Draft and will not be promoted or merged unless both correctness and end-to-end speed gates pass. Rejected variants and their measurements will remain documented.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

First exact-head audit on 8808f8e23926fb1575694504f03a6e1ac291391a: keep Draft; there is nothing to merge yet.

The commit is DCO-signed, but its tree is byte-for-byte identical to base 675a12dedcca8cc020c033f1b1d0f1751d4b8efe; GitHub reports zero changed files, zero additions, and zero deletions. Every implementation, correctness, build, endpoint, and quality gate in the PR body is still pending. No source review or approval can be given until an actual candidate diff is pushed.

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

源码审计结论:该 Draft PR 当前相对最新 main 为 0 个文件、0 行变更,唯一提交只是启动占位提交;正文列出的性能与质量门禁也尚未产出可合并实现。后续主线已吸收相关 XQA 基线工作,因此关闭为空/已失效的跟踪 PR,不制造空合并提交。若出现新的、经算子正确性与匹配性能证据验证的通用硬件/张量契约改动,请从最新 main 新开独立 PR。

@1CatAI 1CatAI locked and limited conversation to collaborators Aug 26, 2026
@yangzhuxinyzx
yangzhuxinyzx deleted the agent/v100-xqa-pv-software-pipeline-20260823-115300 branch August 26, 2026 07:55
@yangzhuxinyzx
yangzhuxinyzx restored the agent/v100-xqa-pv-software-pipeline-20260823-115300 branch August 27, 2026 14:40
@1CatAI 1CatAI unlocked this conversation Aug 27, 2026
@yangzhuxinyzx yangzhuxinyzx reopened this Aug 27, 2026
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

公开开发已恢复:本 PR 已取消归档、解锁并重新打开。为避免把旧基线上的问题直接带入最新 main,现统一作为 Draft 审计项;恢复可见性不代表已经批准合并。 该组涉及 SM70/V100 性能或数值路径,需先与当前主线及 #344–#351 去重,再用同条件的聚焦质量和性能证据决定修复后合并。

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Latest-main audit: Git three-way merge result tree is exactly current main tree 7bb5ea4b41ddc38172ac7c07d1c205210f58a29d; this PR's effective change is already present and merging cannot reintroduce its stale branch snapshot. Recording the original contribution as Merged rather than closing it as superseded.

@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 27, 2026 16:02
@yangzhuxinyzx
yangzhuxinyzx merged commit c29cf34 into main Aug 27, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant