Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ SSD/OS systems columns.
| JiusiServe/vllm-evolve @ 43cc82c | search | scheduling-algorithm evolution, search policy | `optimization-search-policy`, `dense-performance-evidence`, `inference-benchmark-design` |
| [OpenBMB/JustGoal repo + paired paper](https://github.com/OpenBMB/JustGoal/tree/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d) @ `f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d` (repository Apache-2.0 default; NOTICE BSD exceptions; paper license UNKNOWN) | evaluation | Bounded direct inspection, 2026-11-02: complete 21-page paper/appendices and selected DSA wrapper/CUDA workspace/stream contracts; [exact readset and exclusions](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json). Source-only methodology; author-reported comparisons, released failures versus repaired timing and official/stricter oracle differences retained | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md); candidate/not_run, efficacy not_evaluated; no new implementation/backend coverage or full campaign runner |
| [AutoTuneBench repo + paper v1](https://github.com/li-ch/autotunebench/tree/64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f) @ `64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f`; [arXiv:2609.18123v1](https://arxiv.org/abs/2609.18123v1) (selected repository code/data MIT; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: treatment delivery, arm-validity precedence, exported protocol/check/record gaps, weak-denominator and anomaly counterexamples; complete 15-page paper, 6 figures, 7 tables and [exact selected readset/rights](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json) | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md); source-derived candidate/not_run, efficacy not_evaluated; invalid-arm/source-median conflicts retained, no new controller or measured implementation coverage |
| [Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc98043fcf89f3589aa5addb3742777a) @ `1882c6adbc98043fcf89f3589aa5addb3742777a` (selected software/docs Apache-2.0) | evaluation | Bounded source inspection, 2026-11-03: Qwen3.5 GDN/MTP eager record/fold and checkpoint/adoption/retirement contracts, unexecuted fold-test definitions and historical completion denominators; [27-file readset and rights](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md); candidate/not_run, efficacy not_evaluated. Historical author results retain their separate execution revision; no new implementation, backend or measured resource coverage |
| Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4) | evaluation | skill_distill / skill_eval harness, no_skill/human/distilled contract, Tier A pilot | `npu-skill-factory` (evaluation contract) |
| InferMatrixLab/InferRSI PR #9 @ 4ad551a9 | reference | `skill_library/` layout + knowledge records | `npu-skill-factory` (bundle layout) |
| RRSI — arXiv 2609.24972v3 | search; mined via reviewed Pack F | bounded harness proposals; calibrated noise/cost/novelty gates; grouped-ablation limits, replacing the earlier generic constraint summary | `optimization-search-policy`; [exact ledger](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f1--rrsi) |
Expand Down
6 changes: 6 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -600,6 +600,12 @@ NOASSERTION and InferRSI's repository license.
| --- | --- | --- | --- |
| Accelerator (methodology only) | Verify: trace treatment to the eligible consumer, apply frozen arm validity before interpreting a null, and separate declared protocol, reached checks and persisted gate evidence | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md) and [source record](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json); AutoTuneBench repo + paired paper | Source-derived candidate / empirical not_run; paper measurements remain author-reported, efficacy not_evaluated. Contextual source bars are not InferRSI defaults. No new implementation, measured hardware/resource coverage, backend compatibility, or RAM/Storage/Network/OS coverage. |

## NInfer state and completion validation (source-derived)

| Resource context | Stage / scoped question | Canonical owner | Evidence boundary |
| --- | --- | --- | --- |
| Accelerator / Host / service boundary (methodology only) | Observe / Verify: freeze coupled checkpoint and same-block state oracle; distinguish zero accepted drafts from zero adopted inputs, logical commit from device retirement, and selected steady intervals from complete-cohort/termination accounting | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md) and [source record](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json); NInfer `1882c6adbc98043fcf89f3589aa5addb3742777a` | Source-derived candidate / empirical not_run; unexecuted test definitions and historical author results with a separate execution revision. No new implementation, measured hardware/resource coverage, backend compatibility, native state/model/quality/performance result or service acceptance; efficacy not_evaluated. |

## TorchNPU framework source procedures

| Resource | Stage / mechanism | Canonical owner | Evidence boundary |
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:7640b171256091efd1514a33d5e434eef3feaed238fc92fe3eeba02fee94e9ff`
- Input digest: `sha256:ba1e64967e2b3a1eef429bca60ff21c3a78d3be4cdbed077ab255dbc40643cee`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -131,7 +131,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [choose-parallel-training-plan](../src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
| [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 7 readset(s) | — |
| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 9 readset(s) | — |
| [inference-optimization-casebook](../src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
| [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
| [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |
Expand Down
4 changes: 2 additions & 2 deletions src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
procedures use their corresponding backend; shared methods use `common`.

Source snapshot: [7aa3344bf84c42467f43b3a927e1c7d13642bd55](https://github.com/InferMatrixLab/InferRSI/tree/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library).
Source snapshot: [bad68ff5c1426b050df78ffbd106ac2e370ce186](https://github.com/InferMatrixLab/InferRSI/tree/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library).

```bash
inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
Expand All @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).

```bash
python tools/build_skill_index.py --source-revision <committed-source-revision>
python tools/build_skill_index.py --source-revision 7aa3344bf84c42467f43b3a927e1c7d13642bd55 --check
python tools/build_skill_index.py --source-revision bad68ff5c1426b050df78ffbd106ac2e370ce186 --check
```

`--check` detects missing, stale and unexpected generated records without writing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bad68ff5c1426b050df78ffbd106ac2e370ce186/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "bad68ff5c1426b050df78ffbd106ac2e370ce186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json",
Expand Down
Loading
Loading