Skip to content

feat(dcu-kernel-auto-opt): harness planner, gate policy and DKAO tooling - #36

Merged
myrfy001 merged 1 commit into
HuangPuStar:next-appfrom
fms-zth:feat/dcu-kernel-auto-opt-20260911
Sep 28, 2026
Merged

myrfy001 merged 1 commit into
HuangPuStar:next-appfrom
fms-zth:feat/dcu-kernel-auto-opt-20260911

Conversation

@fms-zth

@fms-zth fms-zth commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Brings the dcu-kernel-auto-opt (DKAO) task plugin up to the state of the DCU development container, as two commits:

  1. 6a22b60 — harness planner, gate policy and DKAO tooling
  2. 0345ba7 — refresh from the container (contention/round-strategy, retest tooling, m8 baselines, 7 new test modules)

Scope is limited to metainfer/tasks/dcu_kernel_auto_opt/ — 132 files, 0 files outside the plugin, 0 deletions.

What's in it

Orchestrator / harness

  • planner.py + harness_default/{planner_catalog,planner_policy,manifest,gates}.yaml and harness_default/systemprompt/round_strategy.yaml
  • gate_policy.py, gpu_preflight.py, harness_io.py, predictions.py, validation_budget.py, variant_promote.py
  • contention.py, round_strategy.py, w8a8_baselines.py; updates to config.py, prompts.py, gen_and_opt_pipeline.py, w8a8_pipeline.py, real_pipeline.py, skill_store.py, variant_store.py

Tooling / evidence

  • tools/retest_variant.py, tools/baseline/ Triton INT8 benchmarks + TP8 M8 graph JSONs
  • reports/ best-variant tables (csv/json/xlsx) plus the down-proj diagnostic scripts
  • skills/ INT8 W8A8 GEMM skill packs (foundations / decode / prefill)

Tests: 11 new modules (planner, gate policy, GPU preflight, harness IO, predictions, validation budget, variant promote, contention, execution profile, GPU mode, round strategy, shape catalog panel, retest tool).

PR-side fixes from the previous round are preserved: init_or_resume(task_id) alignment, CPU-only-CI torch guards, hermetic dsh agent tests.

Dependency guards

Three capabilities this plugin measures are not part of this PR and do not exist on next-app yet:

Missing piece Used by Guard
metainfer.orchestrator.gpu_broker production code imports it lazily (config.py, gen_and_opt_pipeline.py — the latter already falls back via try/except) pytest.mark.skipif(importlib.util.find_spec(...) is None) on 10 tests
/opt/dtk/bin/hipprof trusted PMC entry-point validation skipif on the probe test (DCU toolchain only)
metainfer.tasks.harness_evolve tools/retest_variant.py, AHE child embedding skipif on 1 test

Same convention as the existing torch guards: the suites stay green in CI and re-enable automatically once those modules land. Follow-ups: land gpu_broker.py (+ harness_evolve) as their own PRs; no runtime path in the plugin is broken by their absence (all imports are lazy/guarded).

Tests

python -m pytest -q
Tree Result
This PR 10 failed, 1236 passed, 14 skipped
HuangPuStar/MetaInfer:next-app (baseline) 10 failed, 1055 passed, 3 skipped

The 10 failures are identical in both trees and all outside this plugin (2 plugin-registry/form checks, 2 gen_infer_framework_cpp, 2 sglang_trace_analyze, 4 tests/e2e). This PR therefore introduces no new failures and +181 passing tests.

Plugin suite on its own: 417 passed, 14 skipped (13 dependency/environment guards + 1 pre-existing PyTorch guard).

…ainer

Updates the task plugin to the latest container content
(/root/zth_agent/MetaInfer/metainfer/tasks/dcu_kernel_auto_opt), applied as an
incremental 3-way merge so the PR-side alignment fixes stay intact
(init_or_resume(task_id), CPU-only-CI torch guards, hermetic dsh tests).

Added:
- orchestrator/contention.py, orchestrator/round_strategy.py and
  harness_default/systemprompt/round_strategy.yaml
- tools/retest_variant.py, tools/baseline/tp8_m8_* (Triton m8 baselines)
- reports/: down-proj diagnostic scripts and tooling checks
- tests: test_contention, test_execution_profile, test_gpu_mode,
  test_retest_tool, test_round_strategy, test_shape_catalog_panel,
  test_ahe_child_embed

Changed: planner/gate_policy/gpu_preflight/harness_io/config/prompts,
gen_and_opt_pipeline, w8a8_pipeline, w8a8_baselines, server/routes, static
detail/shape panels, form.yaml, WORKFLOW.md, bridge/dsh.

Dependency guards (tests only):
Three features measured by this plugin live outside this PR's scope for now —
metainfer.orchestrator.gpu_broker (GPU broker leases), the DCU toolchain
(/opt/dtk/bin/hipprof PMC profiling) and the harness_evolve sibling task.
Eleven tests that require them are marked with pytest.mark.skipif via
importlib.util.find_spec / path checks (same convention as the torch guards),
so the suite stays green in CI and they re-enable automatically once those
modules land.
@fms-zth
fms-zth force-pushed the feat/dcu-kernel-auto-opt-20260911 branch from 0345ba7 to 20b8dc2 Compare September 20, 2026 13:42
@myrfy001
myrfy001 merged commit 0ce9bdb into HuangPuStar:next-app Sep 28, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants