Skip to content

perf: integrate fused AscendC LayerNormFwd.#1959

Draft
Fengfengst123 wants to merge 2 commits into
xLLM-AI:mainfrom
Fengfengst123:perf-aclnn-layernorm-fwd
Draft

perf: integrate fused AscendC LayerNormFwd.#1959
Fengfengst123 wants to merge 2 commits into
xLLM-AI:mainfrom
Fengfengst123:perf-aclnn-layernorm-fwd

Conversation

@Fengfengst123

@Fengfengst123 Fengfengst123 commented Jul 16, 2026

Copy link
Copy Markdown

中文说明

为什么修改

Qwen3.5 Gated DeltaNet 的 gated LayerNorm/RMSNorm 路径当前使用 Triton 实现。xllm_ops 已提供融合的 AscendC aclnnLayerNormFwd 算子,本 PR 将其接入 xLLM runtime,以减少 prefill 阶段的算子开销并改善 TTFT。

怎么修改

  • 新增 layer_norm_fwd_aclnn C++ wrapper,处理输入 reshape、contiguous、optional bias/gate、输出和统计 tensor 分配。
  • 将 wrapper 加入 NPU xllm_ops CMake 和 API 声明。
  • NPU gated LayerNorm 默认调用融合 AscendC 算子。

性能

Qwen3.5-27B,BF16,TP=1,Ascend 910B3;启用 graph 和 schedule overlap,并使用相同模型、二进制、workload、NPU 和 CPU 绑核策略。

16 组交替顺序的 Triton/AscendC 配对 E2E 测试,input=2048、output=2048、parallel=4:

指标 Triton AscendC 变化
TTFT 2189.237 ms 1961.203 ms 改善 9.833%

与 PR #1936 的 batch=1、input=2048、output=16 对比中,双方各取前 12 轮,并按 1.5×IQR 去除离群值:AscendC TTFT 为 591.911 ms,PR #1936 为 599.399 ms,AscendC 改善 1.249%。

验证

  • xLLM runtime 接入静态检查:18/18 PASS。
  • git diff --check:通过。
  • AscendC 单算子精度测试:通过。
  • 16 组 Triton/AscendC 配对 E2E 性能测试:通过,共 32 个有效服务 run。
  • 固定提示 E2E smoke:两种后端均为 5/5 非空输出。
  • 测试前清除了历史 xLLM/xllm_ops CMake、build 和 OPP 产物,并仅加载本轮新构建的 workspace OPP。

English Description

Why

The Qwen3.5 Gated DeltaNet gated LayerNorm/RMSNorm path currently uses the Triton implementation. xllm_ops now provides a fused AscendC aclnnLayerNormFwd operator. This PR integrates it into the xLLM runtime to reduce prefill operator overhead and improve TTFT.

What changed

  • Add a layer_norm_fwd_aclnn C++ wrapper for reshaping, contiguous inputs, optional bias/gate tensors, and output/statistics allocation.
  • Register the wrapper in the NPU xllm_ops CMake target and API header.
  • Use the fused AscendC operator by default for NPU gated LayerNorm.

Performance

Qwen3.5-27B, BF16, TP=1 on Ascend 910B3, with graph mode and schedule overlap enabled. Both variants use the same model, binary, workload, NPU, and CPU affinity policy.

Across 16 alternating paired Triton/AscendC E2E rounds with input=2048, output=2048, and parallel=4:

Metric Triton AscendC Change
TTFT 2189.237 ms 1961.203 ms 9.833% improvement

For the batch=1, input=2048, output=16 comparison against PR #1936, the first 12 rounds from each variant were evaluated with 1.5×IQR outlier filtering. AscendC achieved 591.911 ms TTFT versus 599.399 ms for PR #1936, a 1.249% improvement.

Validation

  • xLLM runtime integration harness: 18/18 PASS.
  • git diff --check: passed.
  • AscendC single-op accuracy validation: passed.
  • 16 paired Triton/AscendC E2E rounds: passed, with 32 valid service runs.
  • Fixed-prompt E2E smoke: 5/5 non-empty outputs for both backends.
  • Historical xLLM/xllm_ops CMake, build, and OPP artifacts were removed before testing; runtime loaded only the freshly built workspace OPP.

Comment thread xllm/core/kernels/ops_api.cpp Outdated
std::strcmp(value, "FALSE") == 0 || std::strcmp(value, "off") == 0 ||
std::strcmp(value, "OFF") == 0) {
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么会有stringcmp操作?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

嘿嘿,我把回退的代码给删掉吧

@XuZhang99 XuZhang99 changed the title perf(npu): integrate fused AscendC LayerNormFwd perf: integrate fused AscendC LayerNormFwd. Jul 17, 2026
@Fengfengst123
Fengfengst123 marked this pull request as draft July 17, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants