Extend VLA observation-to-action benchmark guidance

docs/reference-matrix.md

@@ -444,6 +444,7 @@ the candidate mechanisms.
 | --- | --- | --- |
 | [Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198), 2026-10-21; eight Git source inventories in the [owner metadata](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/metadata.json) | Task/quality/workload separation across text, agentic coding, work agents, TTS, full-duplex interaction and image/video; selected pinned AIPerf/realtime/FDB/video sources rechecked and the current InferRSI metric contract inspected | Extend [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [task selection](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [command templates](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md). Candidate guidance; source review and bounded smoke do not qualify all tasks or establish a performance gain. Replay scores, reference parity and task quality retain separate meanings. |
 | [NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Speech.cpp/tree/20999cc921d2b14aa58708f61756a8b2e787ba20), inspected 2026-11-01 | Selected cache-aware encoder/config, runner, SDK bounds, endpoint policy, geometry test/CMake and ASR benchmark; complete 14-file readset and Apache-2.0 / parakeet MIT attribution retained in the reference | Extend [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [cache-aware ASR stream boundaries](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md): first/steady mel geometry and decoded clock, stream-slot reset/reuse, EOU/EOF/reset and model-test SKIP boundaries. Source-derived candidate / empirical not_run; benchmark figures remain author-reported with precision, FE and trial limits. No model/device/quality/long-audio/load/service qualification. |
+| [Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) + [paper v3](https://arxiv.org/abs/2607.02501v3), inspected 2026-11-04 (repository Apache-2.0; paper CC BY 4.0) | Selected fixed CPU-observation-to-complete-CPU-action boundary, client queue/profiler denominators, precision identity, per-task quality and source/artifact lineage; [exact readset and rights](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json). Later SmolVLA reports remain separate from the August paper. | Extend registered owner `inference-benchmark-design` ([SKILL.md](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md)) with [VLA boundary and quality guidance](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md). Source-derived candidate/not_run; public sample arithmetic is distinct from episode recount and native execution. Generated actions are not control frequency; per-task loss, unequal precision, whole-device memory and unresolved measured-script/dirty-patch identity remain explicit. |
 
 ## SGLang versioned entry selection
 

docs/searchspace-matrix.md

@@ -614,3 +614,16 @@ NOASSERTION and InferRSI's repository license.
 | Accelerator / RAM / Host | Diagnose / configure / verify: tensor/alias/gradient state, training and data ownership, compiler guards and export contracts, allocator accounting, stream dependencies and graph input/output lifetimes | [pytorch operating graph](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/SKILL.md), [memory/graph route](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md), [compiler/export route](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md) | Source version `2.16.0a0` and observed CPU wheel `2.15.0+cpu` are distinct. CPU observations do not qualify pinned-source execution, CUDA/ROCm/MPS/XPU, native custom operators, memory fit or model quality. |
 | SSD / Storage / RAM | Diagnose / verify: serialization trust and state identity, checkpoint ownership, distributed restore and capture/staging/upload completion boundaries | `pytorch`: [distributed/checkpoint route](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md); existing [DCP async-save owner](../src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md) retains its separate source pin | No source-build checkpoint run, backend durability, distributed recovery or measured storage benefit is established. A state-dict round trip in a separate installed wheel does not qualify those properties. |
 | Communication / Network / Accelerator | Diagnose / verify: process/rank/device mapping, backend and collective schedule, DDP/FSDP2/DTensor state and completion | `pytorch`: [distributed/checkpoint route](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md), [source and runtime boundary](../src/inferrsi/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md) | Selected source and CPU controls do not establish NCCL or other accelerator transport, multi-device/multi-node correctness, overlap or throughput. Required backend checks remain `BLOCKED_REQUIRED_BACKEND`; no new measured/green resource cell, OS control or compatible latest PyTorch/TorchNPU stack is asserted. |
+
+## Embodied.cpp benchmark methodology
+
+The [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md)
+method uses [Embodied.cpp at 156bed15](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd)
+and [arXiv:2607.02501v3](https://arxiv.org/abs/2607.02501v3) as source context
+for observation-to-action timing, generated versus replayed actions, precision
+identity and per-task closed-loop quality. Its [operating reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md)
+and [readset](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json)
+retain the later author's measurement and source-version limits.
+
+These are comparison methods. Experiment hardware is source context rather
+than a device prerequisite or a resource mechanism for this method.

docs/skill-coverage.md

@@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
 matrices. **Do not edit by hand**; regenerate after any registry, skill,
 matrix or frozen-plan change.
 
-- Input digest: `sha256:7640b171256091efd1514a33d5e434eef3feaed238fc92fe3eeba02fee94e9ff`
+- Input digest: `sha256:2ade3ef0deed6fb22ff97c62064e29198e13f8f6bf5e750776cb205d8c14ad43`
 - Regenerate: `python tools/build_skill_coverage.py`
 - Verify without writing: `python tools/build_skill_coverage.py --check`
 
@@ -131,7 +131,7 @@ optimization is fully covered or that any procedure has measured benefit.
 | [choose-parallel-training-plan](../src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
 | [dense-performance-evidence](../src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
 | [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
-| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 7 readset(s) | — |
+| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 9 readset(s) | — |
 | [inference-optimization-casebook](../src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
 | [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
 | [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |

src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md

@@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
 IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
 procedures use their corresponding backend; shared methods use `common`.
 
-Source snapshot: [7aa3344bf84c42467f43b3a927e1c7d13642bd55](https://github.com/InferMatrixLab/InferRSI/tree/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library).
+Source snapshot: [d5fc2121d6d1628111403ba70ec3b70638ab8674](https://github.com/InferMatrixLab/InferRSI/tree/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library).
 
 ```bash
 inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
@@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).
 
 ```bash
 python tools/build_skill_index.py --source-revision <committed-source-revision>
-python tools/build_skill_index.py --source-revision 7aa3344bf84c42467f43b3a927e1c7d13642bd55 --check
+python tools/build_skill_index.py --source-revision d5fc2121d6d1628111403ba70ec3b70638ab8674 --check
 ```
 
 `--check` detects missing, stale and unexpected generated records without writing.

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json

@@ -21,8 +21,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json

@@ -23,8 +23,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json",

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json

@@ -27,8 +27,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json",

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json

@@ -26,8 +26,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json",

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json

@@ -22,8 +22,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json",

src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json

@@ -23,8 +23,8 @@
   "sources": [
     {
       "type": "git",
-      "url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
-      "commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
+      "url": "https://github.com/InferMatrixLab/InferRSI/blob/d5fc2121d6d1628111403ba70ec3b70638ab8674/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
+      "commit": "d5fc2121d6d1628111403ba70ec3b70638ab8674",
       "paths": [
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
         "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json",