Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,10 @@ jobs:
dvidia-training-benchmark --help
dvidia-skillspace-coverage --help
dvidia-episode-tape --help
dvidia-reaction --help
dvidia-training-doctor --json
- name: Test footage, visual training and loopback studio
run: python -m unittest tests.test_footage_data tests.test_footage_train tests.test_footage_pipeline tests.test_footage_benchmark tests.test_training_studio tests.test_doctor tests.test_coverage tests.test_episode_tape -v
run: python -m unittest tests.test_footage_data tests.test_footage_train tests.test_footage_pipeline tests.test_footage_benchmark tests.test_training_studio tests.test_doctor tests.test_coverage tests.test_episode_tape tests.test_reaction tests.test_reaction_metrics tests.test_reaction_convergence tests.test_reaction_benchmark -v

arm:
name: Optional arm / Linux / Python 3.12
Expand All @@ -86,6 +87,17 @@ jobs:
run: python -m unittest discover -s tests -v
- name: Train and qualify the synthetic native pilot
run: python benchmarks/native_pilot.py --output runs/native-pilot
- name: Run and verify reaction mechanical coupons
run: |
dvidia-reaction benchmark --output runs/reaction-ci --repeats 1 --convergence
dvidia-reaction inspect runs/reaction-ci
- name: Preserve reaction diagnostic evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: reaction-mechanical-coupons
path: runs/reaction-ci
if-no-files-found: ignore

container:
name: Visual CLI container
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ This adds MuJoCo 3.15.0 for the existing bounded simulation placement adapter, n

To measure a candidate before improving it, use the [sample-count and simulation benchmark](docs/benchmarks.md). It compares nested sets of whole recordings, keeps development/test footage fixed, freezes candidate weights before evaluation, and reports new-layout outcomes alongside matched open-jaw controls. It runs locally on a CPU and retains failures and qualification receipts. Install from the current source checkout to get `dvidia-training-benchmark`; the frozen v0.1.0 release predates this tool.

The [reaction framework v0.1](docs/reaction-framework.md) adds a reusable monitor and native linear-actuator/parallel-jaw fixtures for predicted clearance, contact, slip and sensor faults. Its local benchmark reports detection errors with uncertainty, continued-physics braking time/distance, contact and load outcomes, and CPU cost against a matched disabled-intervention baseline. It uses synthetic sensed state; camera perception, whole-arm protection and physical stopping accuracy need further adapters and calibration. It preserves every attempted trial and the existing frozen placement evidence.

[The first measured run](benchmarks/reaction-v0.1-20261009/README.md) preserves 78 primary trials and eight timestep checks, including late-impact diagnostic failures and payload drops. Run `dvidia-reaction benchmark --output runs/reaction --repeats 3 --convergence`, then `dvidia-reaction inspect runs/reaction` to generate and verify your own offline report.

## Public pilot on Hugging Face

[Recorded demo](https://huggingface.co/spaces/Dvidia/dvidia-training) · [Synthetic datasets](https://huggingface.co/datasets/Dvidia/dvidia-training-examples) · [Pilot models and measurements](https://huggingface.co/Dvidia/dvidia-training-pilot)
Expand Down
48 changes: 48 additions & 0 deletions benchmarks/reaction-v0.1-20261009/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Reaction v0.1 developmental measurements — 9 October 2026

[Recorded summary](summary.json) and [all primary attempt summaries](attempts.jsonl) bind these observations to source, protocol,
engine and fixture identities. These are synthetic mechanical coupons on one
CPU host, with no camera perception, learned task policy or physical qualification.

The run contains 13 authored cases × 3 repeats × 2 intervention variants:
**78 primary attempts**, plus **8 timestep-comparison attempts**. None emitted
native numerical errors. All attempted trials, diagnostic failures and drops
remain in their declared denominators.

| Observation | Result and scope |
| --- | --- |
| Detector with braking enabled | 24/24 authored positive conditions detected; 0/15 negative cases flagged. Three repeats of the same case families are correlated; this is not broad detection accuracy. |
| Modeled command delay | 20 ms from request to application. |
| Application to confirmed rest | Median 293 ms; maximum 391 ms, including the declared 50 ms quiet dwell. This is modeled mechanical motion, not hardware timing. |
| Native episode throughput | 109.2 rollout seconds / 8.812 per-trial wall seconds = 12.39× real time. Wall includes compilation, warmup, sensor priming, monitor, scoring and trace construction; rollout numerator excludes warmup/priming. |
| Primary benchmark wall | 8.872 s including JSONL writing; additional timestep work 1.228 s. Imports and final report serialization are excluded. |
| CPU monitor calls | Median of per-episode p95 values: 9.79 µs on this host. A per-episode observed percentile is not a hard real-time guarantee. |
| Contact diagnostics | Late-entry attempts exceed declared penetration/force diagnostics in both intervention variants. Finite engine state does not establish physically credible impact forces. |
| Retention | Stable-grip cases retain their load. Slip cases drop it in both variants: stopping horizontal motion does not supply a regrip/set-down recovery. |
| Timestep comparison | Detection/rest timing agrees within declared tolerances at 1 ms and 0.5 ms. Contact-force/impulse diagnostics fail agreement; precise impact forces remain unqualified. |

Anticipatory braking avoids native contact in the crossing and static-obstacle
cases; matched no-brake controls collide. The late occluded entry still contacts
the fixture. The controller runs in shadow in disabled controls, preserving its
detection result while withholding the intervention.

The recorded host is Darwin arm64 with 10 logical CPUs and no GPU use. CPU model
and total RAM were not identified in this run. Process lifetime peak RSS was
75,350,016 bytes, including imports and all trials; it is not a minimum requirement.

Reproduce from the current source with Python 3.12+ and the optional arm extra:

```sh
dvidia-reaction benchmark --output runs/reaction-new --repeats 3 --convergence
dvidia-reaction inspect runs/reaction-new
```

This committed snapshot omits full traces and compiled scene declarations. A
fresh run writes complete native traces, attempted-trial JSONL, frozen protocol,
source identities, convergence receipts, readable HTML and integrity manifest.
The CI arm job also runs the framework and retains its diagnostic artifacts.
The inspector consumes a complete run directory, rather than this summary.

These inspected scenarios are developmental/regression evidence. Changes to a
learner, controller or physics profile need fresh confirmation for qualification.
See [framework contracts](../../docs/reaction-framework.md).
Loading
Loading