feat(moonep): fuse dispatch payload epoch

docs/moonep/MINDSPEED_DEBUGGING_EXPERIENCE.md

@@ -182,6 +182,40 @@ reset 等单一变量。一次同时修改 Kernel、Host、timeout 和路由，
 - 不要因某次补丁通过完整模型就跳过最小 reproducer；最小 reproducer 才能证明因果。
 - 不要删除被推翻的假设记录。保留否定证据可以防止后续重复猜测。
 
+## Dispatch V2 fused epoch 约束与验证边界
+
+一次 paired `TileXRMoonEpDispatchV2` 应只有一个 magic、一次 AICore launch 和
+一个 UDMA epoch。Hidden 与 RouteWeight 必须使用互不重叠的 source 和双 scratch；
+同一 route 在同一逻辑 QP 上按 Hidden、Weight 顺序发 WQE，ordered completion 必须
+排在该 QP 的全部 payload WQE 之后。completion、group credit、CQ reclaim 和 final
+quiet 每个 fused epoch 只执行一轮。发生上游或设备错误后可以停止 payload work，但
+不能跳过 signal-only completion、incoming wait/credit 和最终 status/quiet 收敛，否则
+健康 rank 会退化为超时发现错误。
+
+诊断仍保留独立 Hidden/Weight Profile 与 DFX，但共享阶段只能有一个 owner。paired
+以 Weight record 承载 route scan、flag wait、credit、CQ 和 quiet；Hidden record 的共享
+耗时为零，并由 kernel status 的 fused feature bit 明确标识。profiling OFF 的延迟运行与
+profiling ON 的阶段分析必须分开，不能把两个 payload record 伪装成两轮独立通信。
+
+实机验证时还要注意以下边界：
+
+- Host 环境值是 `group_credit`，不是 `group-credit`；错误拼写会在 launch 前返回 `-3`。
+- grouped/group-credit 只适用于 vector route selection。小于 64 routes、非 2 的幂
+  `NvS` 或 UB/vector 不满足条件的 shape 应使用 legacy scalar-tiled 路径；同步返回
+  `-6` 不是 UDMA 数据面失败。
+- padded zero-fill 要验证完整输出 tensor，不能只比较有效 slot。若 shape 不满足 grouped
+  条件，用 legacy 路径证明 Hidden 和 Weight 的空 slot 都为零。
+- shared-QP 不能仅凭配置名推断。应同时确认 runtime 调用 shared-QP-domain 初始化、日志
+  显示 domain 启用，且 communicator `extraFlag` 含 UDMA 与 `UDMA_SHARED_QP`；本次
+  Ascend950PR 单机 8 rank 证据对应固定 32 QP shared domain。
+- 参考形状 `S=128,K=16,H=3584,NvS=2048` 的不重叠布局仍为 30 MiB；其他 shape
+  必须使用 checked add/multiply 计算真实容量，并在扩大 workspace 绑定后重新检查全部
+  active region 与 common tail 边界。
+
+这些 Host/mock/source guard 只能证明 ABI、布局和源码协议不变量。只有相同 CANN、设备、
+拓扑上的 HCCL baseline 和 Ascend950 实机逐元素多轮结果，才能证明 UDMA 数据面；性能结论
+还必须使用同 shape、同 pair 模式、同 warmup/迭代和独立 profiling-off 构建做 A/B。
+
 ## 当前实现状态说明
 
 本文记录的是已验证经验，不代表所有修复都已经进入 `main`。截至 2026-08-12：

docs/plans/2026-08-13-moonep-dispatch-fused-epoch.md

@@ -0,0 +1,49 @@
+# MoonEP Dispatch Fused Epoch Implementation Plan
+
+## Scope
+
+Implement the approved design in
+`docs/specs/2026-08-13-moonep-dispatch-fused-epoch-design.md`. Preserve the
+public MoonEP and Python APIs. Do not modify comparison-only `reference/` code or
+add Weight-only Dispatch.
+
+## Tasks
+
+### 1. Layout And Host Contract
+
+Update the URMA layout to append Hidden and Weight active regions rather than
+overlay them. Extend launch parameters and the registered direct-Kernel ABI to
+carry both pointer pairs and active offsets. Refactor `RunDispatchUrma` so a
+paired call validates both descriptors but invokes the launcher once. Add layout
+and Host/launch tests that prove one magic and one launch.
+
+### 2. Fused WQE Accounting
+
+Extend the common batch helpers and UB WQE builder so one selected route emits
+one Hidden WQE and an optional immediately following Weight WQE on the same
+logical QP. Preserve completion WQE ordering, SQ capacity reserve, ring wrap,
+CQE final-BB accounting, and grouped staged-doorbell behavior. Add Host-testable
+helpers for WQE counts and selected indices.
+
+### 3. One-Epoch Kernel
+
+Refactor the current single-payload Kernel body into one fused execution path.
+Stage both sources before the first local barrier, scan routes once, service
+both local and remote payloads, wait once, copy both outputs, and quiet once.
+Keep communication convergence on device-detected errors and preserve sticky
+status. Ensure UB resources are reset only after their producers/consumers have
+completed.
+
+### 4. Diagnostics And Tools
+
+Version or feature-mark fused diagnostics without breaking existing record-size
+consumers. Preserve separate per-payload records and final paired Kernel status.
+Update `dispatch_hot_loop.py`, reporting tests, source guards, and API tests.
+
+### 5. Validation And Documentation
+
+Run focused C++ and Python tests, target CANN 9.1 Host/Kernel build, and inspect
+the final diff. On `141.61.49.195`, establish NPU/CANN/source/binary provenance,
+run matching HCCL Test, deploy to a task-specific directory, and execute the
+approved correctness/stability/performance ladder. Record reusable fused-epoch
+workspace and completion-order lessons in the maintained Dispatch design.

docs/specs/2026-08-13-moonep-dispatch-fused-epoch-design.md

@@ -0,0 +1,93 @@
+# MoonEP Dispatch Fused Epoch Design
+
+## Goal
+
+Execute the Hidden payload and optional FP32 route-weight payload of one
+`TileXRMoonEpDispatchV2` call in one registered AICore Kernel launch and one UDMA
+communication epoch. Hidden-only calls remain one launch. The public C, Torch,
+MindSpeed, asynchronous-event, and zero-copy contracts do not change.
+
+## Direct-Launch Contract
+
+The Host validates the mandatory Hidden descriptors and the optional paired
+route-weight descriptors, builds one disjoint registered-workspace layout,
+obtains one communicator magic, and calls `rtKernelLaunchWithFlagV2` once. The
+Kernel ABI carries both payload pointer pairs, both active layouts, separate
+diagnostic offsets, one plan status, and one communication configuration.
+
+Standalone route-weight Dispatch is not added. The mandatory primary payload
+remains FP16 or BF16 Hidden `[S,H] -> [NvS,H]`; the optional pair remains FP32
+`[S,K] -> [NvS]`.
+
+## Workspace
+
+The registered region contains, in this order:
+
+1. Hidden source `[S,H]` and two Hidden receive scratch slots `[NvS,H]`;
+2. Weight source `[S,K]` and two Weight receive scratch slots `[NvS]`;
+3. shared completion flags and signal source;
+4. separate Hidden and Weight Profile arrays;
+5. separate Hidden and Weight DFX arrays;
+6. shared Kernel status.
+
+All offsets use checked 64-bit arithmetic and 64-byte internal alignment. The
+whole registration is rounded to 2 MiB. Binding a larger registered allocation
+moves the common tail while preserving disjoint active regions. For
+`S=128,K=16,H=3584,NvS=2048`, the result remains 30 MiB after registration
+alignment.
+
+## Kernel Flow
+
+1. Validate uniform scalar, pointer, layout, route, transport, and core-count
+   arguments without partially entering the communication protocol.
+2. Cooperatively stage Hidden source and, when present, Weight source into their
+   disjoint registered regions; converge through one `SyncAll`.
+3. Load and select the route plan once. For every selected route and logical QP,
+   append the Hidden WQE followed by the optional Weight WQE. Hidden uses
+   `sourceRow=route/K` and `H*2` bytes; Weight uses `sourceRow=route` and four
+   bytes. Both target the same decoded rank and slot in different scratch
+   regions.
+4. Append the ordered completion WQE after both payload WQEs for that peer/QP.
+   Build every WQE in UB, publish complete batches to SQ through MTE3, then ring
+   doorbells only with `st_dev` after MTE3 completion.
+5. Use one completion-flag exchange, grouped credit progression, CQ recovery,
+   and final quiet for the fused epoch. Errors after protocol entry continue the
+   bounded convergence path so peers are not stranded.
+6. After all receive completions and local-core convergence, zero-fill and copy
+   Hidden output, then optional Weight output, from their respective scratch
+   slot. Output-copy UB is reused only after explicit pipeline completion/reset.
+7. Preserve sticky first-error publication in `plan.status` and write complete
+   per-payload diagnostics plus shared Kernel status.
+
+## Diagnostics
+
+Hidden and Weight retain separate Profile and DFX records for compatibility and
+payload-specific byte/output counters. A new diagnostic feature bit identifies
+a fused epoch. Shared route-selection, flag-wait, credit, CQ, and quiet work is
+owned by the final active payload record: Hidden for hidden-only calls and
+RouteWeight for paired calls. The non-owning Hidden paired record reports zero
+for shared-stage durations instead of duplicating time. Both records carry the
+same magic and failure context. Paired shared Kernel status keeps
+`payloadMode=RouteWeight`; hidden-only keeps `payloadMode=Hidden`.
+
+## Non-Goals
+
+- Weight-only public Dispatch.
+- `WRITE_WITH_NOTIFY` as a weight carrier.
+- Multi-SGE or packed Hidden/Weight WQEs.
+- Public ABI or MindSpeed adapter changes.
+- Removing required local barriers or weakening timeout/error convergence.
+
+## Verification
+
+Host and layout tests prove disjoint ranges, checked arithmetic, one magic, and
+one launch. WQE helper tests prove paired counts, address/length selection, QP
+split, signal ordering, batching, wrap, and CQ accounting. Source guards retain
+UB-only WQE construction, MTE3 SQ publication, and `st_dev` doorbells. Python
+tests preserve the one-FFI-call contract and parse fused diagnostics.
+
+The target CANN 9.1 build must compile Host and Kernel. Hardware validation on
+`141.61.49.195` starts with the matching official HCCL Test and then covers
+single-rank, two-rank, and full-host Hidden/paired exactness, repeated rounds,
+alternating plans, grouped/group-credit/shared-QP configurations where
+supported, and profiling-off/on `pair` A/B evidence.

src/moonep/dispatch/urma/common/dispatch_profile.h

@@ -11,6 +11,7 @@ constexpr uint32_t kDispatchKernelStatusMarker = 0x54584453U; // TXDS
 constexpr uint16_t kDispatchDiagnosticVersion = 3U;
 constexpr uint64_t kDispatchKernelStatusFeatureDfxEnabled = 1U << 0;
 constexpr uint64_t kDispatchKernelStatusFeatureProfilingEnabled = 1U << 1;
+constexpr uint64_t kDispatchKernelStatusFeatureFusedEpoch = 1U << 2;
 
 enum DispatchSelectMode : uint32_t {
     kDispatchSelectScalarTiled = 0,

src/moonep/dispatch/urma/common/dispatch_wqe_batch.h

@@ -23,6 +23,44 @@ constexpr uint32_t kDispatchSharedQpCoreCount = 16U;
 constexpr uint32_t kDispatchSharedQpCount =
     kDispatchQpCount * kDispatchSharedQpCoreCount;
 
+TILEXR_MOONEP_WQE_BATCH_INLINE uint32_t DispatchPayloadWqesPerRoute(
+    bool hasWeight)
+{
+    return hasWeight ? 2U : 1U;
+}
+
+TILEXR_MOONEP_WQE_BATCH_INLINE bool DispatchDataWqeCount(
+    uint64_t routeCount, bool hasWeight, uint64_t &wqeCount)
+{
+    const uint32_t perRoute = DispatchPayloadWqesPerRoute(hasWeight);
+    if (routeCount > UINT64_MAX / perRoute) {
+        wqeCount = 0U;
+        return false;
+    }
+    wqeCount = routeCount * perRoute;
+    return true;
+}
+
+TILEXR_MOONEP_WQE_BATCH_INLINE uint32_t DispatchDataTaskRouteIndex(
+    uint32_t dataTask, bool hasWeight)
+{
+    return hasWeight ? dataTask / 2U : dataTask;
+}
+
+TILEXR_MOONEP_WQE_BATCH_INLINE bool DispatchDataTaskIsWeight(
+    uint32_t dataTask, bool hasWeight)
+{
+    return hasWeight && (dataTask & 1U) != 0U;
+}
+
+TILEXR_MOONEP_WQE_BATCH_INLINE bool DispatchSignalFitsAfterData(
+    uint64_t remainingRoutes, bool hasWeight, uint32_t availableWqes)
+{
+    uint64_t remainingDataWqes = 0U;
+    return DispatchDataWqeCount(remainingRoutes, hasWeight,
+        remainingDataWqes) && remainingDataWqes + 1U <= availableWqes;
+}
+
 TILEXR_MOONEP_WQE_BATCH_INLINE bool DispatchQpCountSupported(
     uint32_t availableQpCount, bool sharedQp = false)
 {
@@ -159,9 +197,12 @@ TILEXR_MOONEP_WQE_BATCH_INLINE uint32_t DispatchQpSelectedIndex(
 
 TILEXR_MOONEP_WQE_BATCH_INLINE bool DispatchPeerWqesStreamable(
     uint64_t routeCount, uint32_t sqEntryCount,
-    uint32_t reserve = kDispatchSqPollReserve)
+    uint32_t reserve = kDispatchSqPollReserve, bool hasWeight = false)
 {
-    if (routeCount > UINT32_MAX || sqEntryCount <= reserve) {
+    uint64_t dataWqeCount = 0U;
+    if (routeCount > UINT32_MAX ||
+        !DispatchDataWqeCount(routeCount, hasWeight, dataWqeCount) ||
+        sqEntryCount <= reserve) {
         return false;
     }
     return sqEntryCount - reserve >= kDispatchWqeBatchCapacity;

src/moonep/dispatch/urma/host/dispatch_host.cpp

@@ -55,7 +55,7 @@ bool ResolveDispatchPeerConfig(DispatchPeerConfig &config)
 }
 
 bool DispatchVectorBatchShapeSupported(uint64_t routeCount,
-    uint64_t destinationCapacity)
+    uint64_t destinationCapacity, bool hasWeight)
 {
     constexpr uint64_t vectorCompareMinElements = 256U / sizeof(int32_t);
     return routeCount >= vectorCompareMinElements &&
@@ -65,17 +65,19 @@ bool DispatchVectorBatchShapeSupported(uint64_t routeCount,
         destinationCapacity <= UINT32_MAX &&
         (destinationCapacity & (destinationCapacity - 1U)) == 0U &&
         DispatchPeerWqesStreamable(routeCount,
-            TileXR::TILEXR_UDMA_SQ_BB_COUNT);
+            TileXR::TILEXR_UDMA_SQ_BB_COUNT,
+            kDispatchSqPollReserve, hasWeight);
 }
 
 int ValidateDispatchPeerConfig(const DispatchPeerConfig &config,
     const TileXR::CommArgs &commArgs, uint64_t routeCount,
-    uint64_t destinationCapacity)
+    uint64_t destinationCapacity, bool hasWeight)
 {
     if (!DispatchPeerModeUsesGroups(static_cast<uint32_t>(config.mode))) {
         return TILEXR_MOONEP_SUCCESS;
     }
-    if (!DispatchVectorBatchShapeSupported(routeCount, destinationCapacity)) {
+    if (!DispatchVectorBatchShapeSupported(
+            routeCount, destinationCapacity, hasWeight)) {
         return TILEXR_MOONEP_ERROR_NOT_SUPPORTED;
     }
     if (commArgs.rankSize > 1 &&
@@ -378,7 +380,8 @@ static int RunDispatchUrma(const TileXRMoonEpDispatchArgsV1 *args,
     }
     ret = ValidateDispatchPeerConfig(peerConfig, *commArgs,
         static_cast<uint64_t>(layout.routeCount),
-        static_cast<uint64_t>(layout.destinationCapacity));
+        static_cast<uint64_t>(layout.destinationCapacity),
+        args->routeWeightsSk != nullptr);
     if (ret != TILEXR_MOONEP_SUCCESS) {
         return ret;
     }
@@ -454,6 +457,12 @@ static int RunDispatchUrma(const TileXRMoonEpDispatchArgsV1 *args,
     params.groupWidth = peerConfig.groupWidth;
     params.zeroFillRangeCount = args->plan->e + args->plan->b;
     params.layout = layout;
+    params.hiddenInput = args->hiddenSh->data;
+    params.hiddenOutput = args->hiddenNvsh->data;
+    params.weightInput = args->routeWeightsSk == nullptr ? nullptr :
+        args->routeWeightsSk->data;
+    params.weightOutput = args->routeWeightsNvs == nullptr ? nullptr :
+        args->routeWeightsNvs->data;
 
     const bool statusResetEnqueued = resetStatus ||
         (args->flags & TILEXR_MOONEP_FLAG_RESET_STATUS) != 0;
@@ -462,21 +471,6 @@ static int RunDispatchUrma(const TileXRMoonEpDispatchArgsV1 *args,
         return TILEXR_MOONEP_ERROR_INTERNAL;
     }
 
-    params.input = args->hiddenSh->data;
-    params.output = args->hiddenNvsh->data;
-    params.mode = DispatchPayloadMode::Hidden;
-    ret = MapLaunchStatus(TileXRMoonEpLaunchDispatchUrmaKernel(params));
-    if (ret != TILEXR_MOONEP_SUCCESS || args->routeWeightsSk == nullptr) {
-        if (ret != TILEXR_MOONEP_SUCCESS && statusResetEnqueued &&
-            aclrtSynchronizeStream(stream) != ACL_SUCCESS) {
-            return TILEXR_MOONEP_ERROR_INTERNAL;
-        }
-        return ret;
-    }
-
-    params.input = args->routeWeightsSk->data;
-    params.output = args->routeWeightsNvs->data;
-    params.mode = DispatchPayloadMode::RouteWeight;
     ret = MapLaunchStatus(TileXRMoonEpLaunchDispatchUrmaKernel(params));
     if (ret != TILEXR_MOONEP_SUCCESS && statusResetEnqueued &&
         aclrtSynchronizeStream(stream) != ACL_SUCCESS) {

src/moonep/dispatch/urma/host/dispatch_host.h

@@ -12,6 +12,9 @@ int TileXRMoonEpQueryDispatchUrmaWorkspace(TileXRCommPtr comm, int64_t s,
 int TileXRMoonEpRunDispatchUrmaV1(const TileXRMoonEpDispatchArgsV1 *args,
     aclrtStream stream);
 
+int TileXRMoonEpRunDispatchUrmaV2(const TileXRMoonEpDispatchArgsV2 *args,
+    aclrtStream stream);
+
 } // namespace TileXRMoonEp
 
 #endif // TILEXR_MOONEP_DISPATCH_URMA_HOST_H