Skip to content

feat: add Ascend950 URMA combine - #89

Open
one-bite-one-pig wants to merge 9 commits into
LingquLab:mainfrom
one-bite-one-pig:feature/ep-urma-combine-s22-pr
Open

feat: add Ascend950 URMA combine#89
one-bite-one-pig wants to merge 9 commits into
LingquLab:mainfrom
one-bite-one-pig:feature/ep-urma-combine-s22-pr

Conversation

@one-bite-one-pig

Copy link
Copy Markdown

Summary

  • add an Ascend950PR URMA expert-parallel Combine API and device kernel
  • use 42 Pack/Receive AIVs and 22 Send AIVs with independent UDMA QPs
  • add cacheless DataAsFlag transport, deferred round credit, parallel round publication, and Q/D v3
  • add focused host validation, layout/source tests, documentation, and one retained BS128 profile report
  • preserve the default single-QP UDMA behavior outside Ascend950PR EP builds

Workloads

Parameter BS32 BS128
Hardware Ascend950PR Ascend950PR
Input/output dtype FP16 FP16
Batch size 32 128
Hidden size 5120 5120
Top-K 6 6
Rank count 8 8
Routes per rank (BS * K) 192 768
Route stride 5632 bytes 5632 bytes
Pack/Receive AIVs 42 42
Send AIVs 22 22
UDMA QPs 22 22
Doorbell batch 1 WQE 1 WQE
Transport payload per-route INT8 quantized per-route INT8 quantized

The operator accepts FP16 expert output and FP16 top-k weights, performs per-route INT8 transport quantization, and writes FP16 combined output.

Performance

Measured on eight Ascend950PR devices.

Production timing uses profiling-free strictKernelCycles with 20 warmup rounds and 100 measured rounds. Each launch is reduced using max-core -> max-rank, followed by the median across measured launches.

Workload Final median
FP16, BS32, K6, H5120, 8x Ascend950PR 38,541 cycles (38.541 us)
FP16, BS128, K6, H5120, 8x Ascend950PR 88,343.5 cycles (88.3435 us)

The retained BS128 report excludes the first-launch synchronization gate from charts and kernel KPIs. Raw gate samples remain embedded only for provenance.

Validation

Validated on eight Ascend950PR devices:

  • FP16 BS32/K6/H5120 correctness smoke passed
  • FP16 BS128/K6/H5120 correctness smoke passed
  • BS32 and BS128 strict 100-round timing passed
  • BS128 coarse and fine profile captures passed
  • coarse and fine inventories contain 80 traces each
  • all rank outputs passed correctness checks
  • exact rebased PR build passed test_tilexr_ep_kernel_sources
  • exact rebased PR build passed test_tilexr_ep_layout

Notes

The src/comm/udma changes provide the independent QPs required by the 22 Send AIVs. Builds that do not enable the Ascend950PR EP path retain the existing single-QP default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant