perf: integrate fused AscendC LayerNormFwd.#1959
Draft
Fengfengst123 wants to merge 2 commits into
Draft
Conversation
Fengfengst123
requested review from
Clement-Wang26,
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
RobbieLeung,
XuZhang99,
liujinguang0125,
liutongxuan,
walsonyang,
xiao-yu-chen,
yingxudeng,
yq33victor and
zhang-minchao
as code owners
July 16, 2026 15:29
Sinle4Cat
reviewed
Jul 17, 2026
| std::strcmp(value, "FALSE") == 0 || std::strcmp(value, "off") == 0 || | ||
| std::strcmp(value, "OFF") == 0) { | ||
| return false; | ||
| } |
Fengfengst123
marked this pull request as draft
July 17, 2026 09:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
中文说明
为什么修改
Qwen3.5 Gated DeltaNet 的 gated LayerNorm/RMSNorm 路径当前使用 Triton 实现。
xllm_ops已提供融合的 AscendCaclnnLayerNormFwd算子,本 PR 将其接入 xLLM runtime,以减少 prefill 阶段的算子开销并改善 TTFT。怎么修改
layer_norm_fwd_aclnnC++ wrapper,处理输入 reshape、contiguous、optional bias/gate、输出和统计 tensor 分配。xllm_opsCMake 和 API 声明。性能
Qwen3.5-27B,BF16,TP=1,Ascend 910B3;启用 graph 和 schedule overlap,并使用相同模型、二进制、workload、NPU 和 CPU 绑核策略。
16 组交替顺序的 Triton/AscendC 配对 E2E 测试,input=2048、output=2048、parallel=4:
与 PR #1936 的 batch=1、input=2048、output=16 对比中,双方各取前 12 轮,并按 1.5×IQR 去除离群值:AscendC TTFT 为 591.911 ms,PR #1936 为 599.399 ms,AscendC 改善 1.249%。
验证
git diff --check:通过。English Description
Why
The Qwen3.5 Gated DeltaNet gated LayerNorm/RMSNorm path currently uses the Triton implementation.
xllm_opsnow provides a fused AscendCaclnnLayerNormFwdoperator. This PR integrates it into the xLLM runtime to reduce prefill operator overhead and improve TTFT.What changed
layer_norm_fwd_aclnnC++ wrapper for reshaping, contiguous inputs, optional bias/gate tensors, and output/statistics allocation.xllm_opsCMake target and API header.Performance
Qwen3.5-27B, BF16, TP=1 on Ascend 910B3, with graph mode and schedule overlap enabled. Both variants use the same model, binary, workload, NPU, and CPU affinity policy.
Across 16 alternating paired Triton/AscendC E2E rounds with input=2048, output=2048, and parallel=4:
For the batch=1, input=2048, output=16 comparison against PR #1936, the first 12 rounds from each variant were evaluated with 1.5×IQR outlier filtering. AscendC achieved 591.911 ms TTFT versus 599.399 ms for PR #1936, a 1.249% improvement.
Validation
git diff --check: passed.