Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ not fabricated implementation SHAs or a version-matched runtime stack.
| Original source / version | Inspected scope and local evidence | Mechanism / consumer | Evidence limits |
|---|---|---|---|
| Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration |
| [AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e4a51998eba2933ed63cfb9c10120f86ea) | Direct bounded source inspection, 2026-11-03: serving CLI/evaluator/history, seed/objective parser, launcher/DES countercheck and named unexecuted test definitions; [16-path readset and rights](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json) | mined — `calibrate-serving-simulation`: one [calibration-owned provenance reference](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md); search-policy links its seed comparator and evaluated-trial ledger | Source-derived candidate; empirical not_run. Scalar tuner scoring and DES percentiles are separate; missing SLO fields and stale no-P/D-queues text are qualified. Selected original paths MIT; no runtime, ranking accuracy, quality, service or agent-benefit result. |
| Mooncake — FAST '25 proceedings, pp. 155-170 | Direct public-paper read, retained qualification 2026-10-31; [source condition and readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md), §2.2, Table 1, Eqs. 1-2 | mined — conditional prefix-KV transfer versus saved-prefill-work comparison → `performance-ceiling-analysis` | Analytical gate only; current inputs and E2E benefit unverified. SSD mechanics excluded. Original transfer-gates extraction history remains positive; missing synthesis novelty UNKNOWN. Raw-paper redistribution permission UNKNOWN. |
| FlexGen — [ICML 2023 / PMLR 202:31094–31116](https://proceedings.mlr.press/v202/sheng23a.html), official ten-author edition | Direct scoped public-paper qualification, 2026-10-31; [condition, exact digest and readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md), §§3–4.3 and selected Appendix A.3 formulas; Figures 1–3, Algorithm 1, Eq. 1, Tables 2–3 | mined — joint weights/KV/activation placement, resident plus working-storage peaks and offline latency/throughput conditions → `performance-ceiling-analysis` | Source-derived candidate; empirical not_run. Perfect-overlap fitted model, relaxed-placement OOM and unresolved `bls < 20` versus `32 × 8` wording retained; no current runtime, quality or gain qualification. Attributed CC BY 4.0 synthesis; missing-archive novelty UNKNOWN. |
| Orca — OSDI 2022 proceedings | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md); §§3–6, Algorithm 1 | mined — iteration eligibility, selective batching, FCFS/reservation → [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md) | Historical engine/harness and normalized-median denominators; not present vLLM behavior or fairness proof |
Expand Down
4 changes: 2 additions & 2 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The primary compute and HBM resource. Most current work lives here.
| **Diagnose / Configure / Verify** | AKG static CUDA Affine loop tails: effective target/option gates, full point-loop bounds and protected input/output accesses | [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | [Tail-guard reference](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md): separation precedence, non-divisible fixture and diagnostic boundary | AKG @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff; source + FileCheck expectations only | Candidate; compiler/IR command, CUDA runtime and agent efficacy not run; no new measured coverage |
| **Optimize** | Operator-level: staged accuracy-gated tuning campaign (code-level → scheduling → instruction-level), metric→action routing; AscendC tiling design per operator class (reduction/broadcast/elewise/matmul/conversion) | `ascend-operator-tuning`, `ascendc-tiling-design` | PyPTO frontend/swimlane/incore playbooks; fullload-vs-split, four-element tiling template, chip-generation (DAV_2201/3510) routing | awesome-ascend-skills pypto-op-perf-tune + ascendc-tiling-design @ 75ec89c | 🟢 |
| **Optimize** | Casebook of past NPU optimization decisions: AI-CPU fallback, memory-bound, comm-exposed, graph-compilation, scheduling idle gaps | `ascend-optimization-casebook`, `inference-optimization-casebook` | Five bottleneck classes with symptom→diagnosis→action→evidence schema | MindIE-SD profiling-analyze + dit-perf-opt + perf-gate + dit-parallel-opt @ 8481d39b | 🟢 |
| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | [Sarathi chunk-budget case](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md): decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate official OSDI 2024 Sarathi case/readset in Reference Matrix | 🟢 (source-derived candidate procedure; empirical not_run) |
| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | [Sarathi chunk-budget case](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md): decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. [Infera provenance](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md) adds a seed-only comparator and evaluated-trial ledger example; its trace-ranking check is proposed. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate Sarathi case and bounded Infera Git readset in Reference Matrix | 🟢 (source-derived candidate procedure; empirical not_run) |
| **Optimize** | Bounded multistream overlap: dependency-DAG constrained (data/state/event/collective_order edges; shared input ≠ dependency), minimal-first, execution-mode sync contract (eager streams vs TorchAir) | `cann-attribution-overlap` | Metric separation (wall / union-of-busy / summed durations / summed duration+wait; repeated-instance product = estimate); device-memory domain qualification; formula conflicts carried unresolved | cann/oam-tools @ 8440935f (pack-mediated, wave 6) | 🟢 |
| **Verify** | Evidence taxonomy (L0 correctness → L5 production), claim-evidence matching | `dense-performance-evidence`, `rsi-evidence`, `claim-contract` | InferRSI PASS/FAIL/INCONCLUSIVE protocol | InferRSI @ 4c8e1285 | 🟢 |
| **Diagnose / Verify** | Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline | [ascend-performance-analysis](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md) | [Stage-evidence rule](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md): workflow instructions versus registered marker/keyword checks; completion is not numerical validation | canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope | Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage |
Expand All @@ -58,7 +58,7 @@ The primary compute and HBM resource. Most current work lives here.
| **Observe / Diagnose / Verify** | KV receive/save completion, delayed-free ownership, in-flight-write fences and failure reporting | `audit-vllm-kv-lifetimes` | Event ledger is proposed, not a telemetry exporter; abstract obligations require concrete connector/caller evidence | vLLM @ 7f730b4cea9a, Dot bundle | 🟢 (contract intelligence; concrete backend unverified) |
| **Diagnose / Configure / Optimize / Verify** | Runtime ACLGraph dispatch, stable input addresses, capture-resource limits and branch-specific replay/update ordering | `ascend-aclgraph-capture-replay` | Caller owns persistent buffers; DEBUG checks positional tensors only; reduced capture sets trade coverage for resources; [version/responsibility routing](../src/inferrsi/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | vllm-ascend @ 8a8e1e05ecb6, Dot bundle; wrapper/guide/tests inspected by supplier | 🟢 (conditional proposals; tests not run) |
| **Diagnose / Configure / Optimize / Verify** | Device-HBM stream-use lifetime, fragmentation/counter interpretation and gated allocator controls | `torchnpu-device-memory-lifecycle` | Lifetime recording does not insert data dependencies; expandable segments change counters/control legality; GC synchronization/comment conflict retained | TorchNPU @ 67511c53cd4c, Dot bundle; device allocator ledger | 🟢 (source-supported procedure; NPU execution not run) |
| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound | Vidur v2; Reference Matrix systems-paper row | 🟢 (prediction guidance; calibration not run) |
| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain, fidelity provenance and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound. [Infera source reference](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md) separates scalar tuner scoring from DES percentiles and retains missing-SLO/stale-document limits; no service guarantee | Vidur v2 and bounded Infera Git readset; Reference Matrix systems sources | 🟢 (prediction guidance; calibration not run) |
| **Diagnose / Verify** | Reusable-prefix compute work versus host/network KV loading cost | [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md), [Mooncake condition](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md) | Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements | Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive | 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit) |
| **Diagnose / Verify** | Offline offload feasibility across GPU, CPU RAM and disk | [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md), [FlexGen condition](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md) | Joint weights/KV/activation placement; RAM resident plus staging/working peaks, SSD state capacity and disk↔CPU cost, effective batch versus GPU batch, exposed CPU/transfer cost and block-latency objective | FlexGen official ICML 2023, §§3–4.3 and selected A.3; separate public source, foundation archive still unavailable | Source-only candidate; empirical not_run, fitted/perfect-overlap model and relaxed OOM limits; no blanket RAM/SSD, Network/OS or current-engine qualification |
| **Diagnose / Configure / Optimize / Verify** | Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups | [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | [Hypothetical pressure/migration cases](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md); cohort/fairness and destination-reservation costs | Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate | 🟢 (conditional paper-derived plans) |
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:7640b171256091efd1514a33d5e434eef3feaed238fc92fe3eeba02fee94e9ff`
- Input digest: `sha256:930a3c698dd502804d2bff47cd6c49311e51492d68497d2986110f51bf18ceae`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -126,7 +126,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md) | common | Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host ... | Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Orca — OSDI 2022 proceedings) | candidate · not_run · 1 readset(s) | no-eval |
| [assess-split-backward-schedule](../src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md) | common | Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update ... | Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Zero Bubble — [2401.10241v1](https://arxiv.org/abs/2401.10241v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [audit-fixed-workload-claim](../src/inferrsi/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md) | common | Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation ... | Auditing a performance claim against supplied fixed-workload reports and an explicit ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | N/A: SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval |
| [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md) | common | Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical ... | Building an evidence-gated serving-configuration shortlist when a profiled simulator or ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2)) | candidate · not_run · 1 readset(s) | no-eval |
| [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md) | common | Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical ... | Building an evidence-gated serving-configuration shortlist when a profiled simulator or ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2))<br>1 · Systems sources ([AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e ...) | candidate · not_run · 3 readset(s) | no-eval |
| [checkpoint-recovery-consistency](../src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md) | common | Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport ... | Assessing distributed checkpoint recovery through DLRover memory agreement, persistence ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 2 RAM/Host · Configure / Verify<br>3 SSD · Configure / Verify<br>4 Network · Diagnose / Configure / Verify<br>5 OS · Diagnose / Configure | 1 · Systems sources (C3 / DLRover FlashCheckpoint @ `e35b1ad90e2928ca6bacf5d22117f71356 ...)<br>1 · Systems sources (C4 / NVIDIA Resiliency Extension @ ...) | candidate · not_run · 1 readset(s) | no-eval |
| [choose-parallel-training-plan](../src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
Expand Down
Loading
Loading