Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,11 @@ controller is authored; shoe physics and physical transfer remain unqualified.

[Hardware matched robot simulation](reports/Hardware%20matched%20robot%20simulation.md)
now includes an [interactive 3D arm/gripper replay](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-11-hardware-reference/replay/)
and the [hardware support map](https://github.com/Dvidia-Inference/dvidia-training/blob/928fed79eca09a1a5081aae7720ab78c2e5b817c/docs/hardware-roadmap.md).
and the [hardware support map](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/docs/hardware-roadmap.md).
PiPER and SO-101 have runnable CPU simulation profiles with unloaded numerical
receipts; their object-task and physical adapter qualification remain open.
receipts. PiPER also has a bounded authored block-task pilot: nominal placement
2/2, low-friction placement 0/1, and both controls 0/3, with all nine attempts retained.
Learned task and physical adapter qualification remain open; SO-101 object-task work is upcoming.
YAM, OpenArm, ARX X5 and PAROL6 remain planned integration targets. The replay displays frozen native poses and geometry;
it adds no live physics, learned skill or physical-fidelity result.

Expand Down
29 changes: 25 additions & 4 deletions docs/downloads/reports/Hardware matched robot simulation.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,17 @@ The ready-made browser replay can be viewed on another supported browser without

## Hardware integration roadmap

The following list separates runnable numerical simulation from planned integration. **PiPER now has a pinned CPU simulation profile; YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.** All five still need their own object-task qualification and physical adapter evidence tied to the exact hardware/controller revision. Arm-axis counts exclude the gripper; a future five-finger hand has a separate actuation and sensing contract.
The following list separates runnable numerical simulation from planned integration. **PiPER now has a pinned CPU simulation profile and a bounded authored block-task pilot; YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.** All five still need physical adapter evidence tied to the exact hardware/controller revision. PiPER's task evidence is limited to the synthetic block protocol below; broader task qualification remains open. Arm-axis counts exclude the gripper; a future five-finger hand has a separate actuation and sensing contract.

| Priority | Target | Intended role and remaining work |
| --- | --- | --- |
| 1 | **YAM — 6 arm axes + gripper** | First candidate for a full-size shoe grasp-and-place benchmark. MIT control/models include MuJoCo and teleoperation; complete open manufacturing designs are not established. Pin arm/gripper revisions and measure their physical response. [Official SDK/models](https://github.com/i2rt-robotics/i2rt). |
| 2 | **OpenArm — 7 arm axes + gripper per side** | Later two-arm coordination. CERN-OHL-S-2.0 hardware and Apache-2.0 simulation; native LeRobot connection currently requires Linux. Keep v1/v2 profiles distinct and assess hand mounts, mass, power and remaining payload. [Hardware](https://github.com/enactic/openarm_hardware), [MuJoCo](https://github.com/enactic/openarm_mujoco), [LeRobot](https://huggingface.co/docs/lerobot/main/openarm). |
| 3 | **AgileX PiPER — 6 arm axes + gripper** | Runnable pinned CPU profile with unloaded numerical receipts; object-task and physical qualification remain open. Open SDK and MIT Menagerie model; full manufacturing openness is not established. Bind firmware, offsets, control modes and jaw calibration for a physical adapter. [SDK](https://github.com/agilexrobotics/piper_sdk), [model](https://github.com/google-deepmind/mujoco_menagerie/tree/main/agilex_piper). |
| 3 | **AgileX PiPER — 6 arm axes + gripper** | Runnable pinned CPU profile and authored rigid-block contact pilot; learned task and physical qualification remain open. Open SDK and MIT Menagerie model; full manufacturing openness is not established. Bind firmware, offsets, control modes and jaw calibration for a physical adapter. [SDK](https://github.com/agilexrobotics/piper_sdk), [model](https://github.com/google-deepmind/mujoco_menagerie/tree/main/agilex_piper). |
| 4 | **ARX X5 — 6 arm axes + gripper** | Joint position/velocity/torque research adapter and later multiple-arm control. MIT research SDK and community LeRobot plugin; exact X5 simulation and hardware licensing need qualification. [SDK](https://github.com/real-stanford/arx5-sdk), [plugin](https://github.com/villekuosmanen/lerobot-arx5). |
| 5 | **PAROL6 — 6 arm axes + separate gripper** | Community-build desktop target. GPLv3 mechanics/software and ROS/Python tools; the documented open-loop version has limit-switch sensing, so actual joint feedback requires verification. Complete electronics openness is not established. [Project](https://github.com/Source-Robotics/PAROL6-Desktop-robot-arm), [specifications](https://source-robotics.github.io/PAROL-docs/page2_2/). |

SO-101 is the accessible entry profile, with five arm axes plus a gripper. Its pinned CPU simulation is now runnable; its object-task and physical adapter qualification remain open. The UR5e/Robotiq reference is separately characterized. Gripper opening, finger geometry, slip, workspace and usable payload drive the first shoe hardware choice. Device support advances separately from learned skill and physical qualification. ([DVIDIA hardware support map](https://github.com/Dvidia-Inference/dvidia-training/blob/928fed79eca09a1a5081aae7720ab78c2e5b817c/docs/hardware-roadmap.md))
SO-101 is the accessible entry profile, with five arm axes plus a gripper. Its pinned CPU simulation is now runnable; its object-task and physical adapter qualification remain open. The UR5e/Robotiq reference is separately characterized. Gripper opening, finger geometry, slip, workspace and usable payload drive the first shoe hardware choice. Device support advances separately from learned skill and physical qualification. ([DVIDIA hardware support map](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/docs/hardware-roadmap.md))

## Match observable mechanics before calling it fidelity

Expand Down Expand Up @@ -121,7 +121,28 @@ All attempts remain in the [Robot Simulation Lab](https://huggingface.co/spaces/

PiPER simulation assets are MIT; SO-101 assets are Apache-2.0. The latter has published build/design files and LeRobot guidance, while an open PiPER SDK does not by itself establish open manufacturing designs. The models' original notices are retained, and their attribution is separate from DVIDIA software. ([SO-101 design](https://github.com/TheRobotStudio/SO-ARM100), [LeRobot guide](https://huggingface.co/docs/lerobot/so101), [PiPER SDK](https://github.com/agilexrobotics/piper_sdk))

These additions are ready for local unloaded simulation. Each still needs its own object-task environment, grasp/release controls, calibrated observations and fresh closed-loop policy benchmark. YAM, OpenArm, ARX X5 and PAROL6 remain further integration targets; bimanual and dexterous manipulation retain separate contracts. The hosted lab displays the UR5e contact replay separately from these numerical receipts, so additional model support cannot be mistaken for inherited grasp competence.
These additions are ready for local unloaded simulation. PiPER now also has the bounded authored contact pilot below; SO-101 still needs its own object-task environment and grasp/release controls. Both need calibrated observations and a fresh learned-policy benchmark before a skill claim. YAM, OpenArm, ARX X5 and PAROL6 remain further integration targets; bimanual and dexterous manipulation retain separate contracts. The hosted lab presents the UR5e contact replay, PiPER task receipts and unloaded profile receipts separately.

## PiPER contact pilot retains a low-friction failure

The next PiPER slice adds a free **40 mm rigid cube weighing 40 g** to the pinned model. Six arm targets use radians and the jaw target uses single-finger slide metres; both actual finger slides are recorded. Stock servos, force limits, arm capsule proxies, finger collision boxes and robot gravity compensation remain unchanged. The block has gravity compensation disabled and moves under native gravity/contact, with no attachment or post-reset pose writes. A 60 mm support height and declared 180° acquisition / 150° carry wrist orientations bound this environment. The TCP is an authored local site near the distal pad centers, rather than a measured hardware calibration. ([Versioned source and installation](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/docs/piper-contact-pilot.md), [Task implementation](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/src/dvidia_training/piper_task.py))

Development exposed a failed higher-support/fixed-orientation attempt near a wrist limit with table/finger contact. Low support and different acquisition/carry orientations were then selected; subsequent nominal development attempts passed while a low-friction attempt failed to lift. Those source-snapshotted development runs informed implementation and are separate from the confirmation denominator.

The frozen confirmation protocol uses seed **642103**, two nominal scenes at friction 0.8 and one stress scene at friction 0.15, with seeded start/target XY offsets bounded by 5 mm. Each scene retains placement, identical requested arm targets with the jaw open, and initial-arm hold. All nine attempts ran for the full 20 simulated seconds, with a 1 ms physics step, 1,000 commands and 1,001 recorded states each. The source and protocol were frozen before these CPU attempts ran under process-level network denial. These three synthetic scenes provide descriptive pilot counts, not an estimate of general robot reliability. ([Frozen protocol](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-11-piper-contact/protocol.json), [Full hosted attempt bundle](https://huggingface.co/spaces/Dvidia/robot-simulation-lab/tree/b07b6e0e95a19158384734d140abb36a9adb670b/tasks/piper))

| Stratum | Placement completed / attempted | Open-jaw completed / attempted | Hold completed / attempted | Numerically valid / attempted |
| --- | ---: | ---: | ---: | ---: |
| Nominal, friction 0.8 | 2/2 | 0/2 | 0/2 | 6/6 |
| Low friction 0.15 | 0/1 | 0/1 | 0/1 | 3/3 |

Success requires bilateral normal contact above 0.01 N during a table-free lift of at least 60 mm for 100 ms. Once lift qualifies, a consecutive bilateral-contact loss of 40 ms permanently disqualifies retained carry before supported arrival. Release needs both actual finger openings, zero active pad contacts, supported low linear/angular motion within 25 mm of the destination for 250 ms, and 100 mm tool retreat. Unexpected moving-arm/table/floor or nonfinger block/robot contacts disqualify the task; the stationary base/floor pair has an explicit exemption. Destination orientation is not scored. ([Scoring source](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/src/dvidia_training/piper_task.py), [Complete report](https://huggingface.co/spaces/Dvidia/robot-simulation-lab/resolve/b07b6e0e95a19158384734d140abb36a9adb670b/tasks/piper/report.html))

Nominal final position errors are **4.643–4.682 mm**; maximum lifts are 97.842–97.911 mm, and first qualifying release dwells occur at 15.112–15.132 simulated seconds. The low-friction placement establishes bilateral contact but never qualifies lift: its maximum rise is 0.132 mm and its final target error is 120.185 mm. All nine are numerically valid without captured warnings or critical contact ticks. This retained failure distinguishes gripping contact from a usable carry. ([Exact report snapshot](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-11-piper-contact/report.json))

Complete-attempt wall times span **1.386–2.285 seconds** for 20 simulated seconds on macOS 26.5.2 arm64, Python 3.12.9, MuJoCo 3.15.0 and NumPy 2.5.3, with ten reported logical CPUs and no GPU workload. Per-attempt 20 ms computation p95 spans 1.755–2.986 ms; the worst window is **28.182 ms**, leaving **one missed computation window out of 9,000**. That outlier remains in the report. Wall timing includes verification, compilation, settling, planning, physics and state capture; disk serialization, rendering, learned inference and physical sensing/actuation are excluded. Process lifetime peak RSS is about 724 MiB before final inspection, not a minimum-memory requirement. Host load is uncontrolled, and these timings do not establish real-time hardware response. ([Timing and runtime receipt](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-11-piper-contact/report.json))

The [Robot Simulation Lab](https://huggingface.co/spaces/Dvidia/robot-simulation-lab) publishes complete task attempts with the original MIT model notices. The [small research mirror](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-11-piper-contact/README.md) contains summary documents only; its original integrity file describes the full hosted bundle. Inspection checks byte integrity, declared controls, model/units, all-attempt denominators and recorded endpoint consistency. It does not authenticate the publisher or recompute contact physics. The result is a benchmark for our next compact action policy; learned skills, calibrated shoe interaction and physical robot qualification remain open.

## Release an evidence-bearing skill, not an appearance match

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# PiPER contact pilot — summary-only research mirror

This mirror preserves the exact report, protocol and original integrity bytes
from the nine-attempt confirmation run, seed 642103. It is **not a standalone
complete benchmark bundle**. Raw attempts, the HTML report and original model
notices remain in the [full Hugging Face bundle](https://huggingface.co/spaces/Dvidia/robot-simulation-lab/tree/b07b6e0e95a19158384734d140abb36a9adb670b/tasks/piper).
The integrity file describes that full bundle, including files absent here;
run the offline inspector against a complete download.

- [Summary and runtime measurements](report.json)
- [Frozen synthetic scene/control protocol](protocol.json)
- [Original complete-bundle integrity manifest](integrity.json)
- [Complete HTML report](https://huggingface.co/spaces/Dvidia/robot-simulation-lab/resolve/b07b6e0e95a19158384734d140abb36a9adb670b/tasks/piper/report.html)
- [Versioned source and local installation](https://github.com/Dvidia-Inference/dvidia-training/blob/c9a72584b4f35c38ad2616894f56da719bba0182/docs/piper-contact-pilot.md)

All nine attempts are numerically valid: nominal placement 2/2, low-friction
placement 0/1, open-jaw 0/3 and hold 0/3. Every attempt retains 20 simulated
seconds, 1,000 commands and 1,001 recorded states in the full bundle. One of
9,000 measured computation windows exceeded 20 ms; real-time physical response
is not established.

The task uses an authored controller and privileged native state. These are
synthetic rigid-block scenes, with no learned skill, camera sensing, shoe
physics, calibrated contact or physical robot qualification. Earlier source-
snapshotted development attempts informed the controller and remain separate
from this confirmation denominator. The original PiPER model uses MIT licensing;
its notices are preserved in the full bundle. Hash consistency is not publisher
authentication or recomputation of native physics.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"schema":"dvidia.piper-task-integrity/v1","files":{"protocol.json":{"bytes":5865,"sha256":"43f0825130f1b436bc78709a75027f62ad43f55a42c1af5d80d6dfeae9f9e19d"},"report.json":{"bytes":70980,"sha256":"d9148a29cef95fa242e6bc3f0d4b02e0ee56e824bd70996528b47573691916eb"},"report.html":{"bytes":4600,"sha256":"98515b44bfdbf29e73782794c890d6a98e6af258175d47ca324a81061d1c35ef"},"licenses/agilex_piper/LICENSE":{"bytes":1065,"sha256":"e32d78fd3761052ab0b2bb42fef8a7a9ad3f298d315d305333c349d2aff1f134"},"licenses/agilex_piper/README.md":{"bytes":1681,"sha256":"ab0d218758ec6c298417407962f49179e369236e8f316dbaa7671756d6f8eb77"},"attempts/nominal-00--placement.json":{"bytes":3316278,"sha256":"432c4878157436abaf3a7224028b4ddc0c269c1ebb72f5bb27d4d7c22cacfac3"},"attempts/nominal-00--replay_open_jaw.json":{"bytes":3245020,"sha256":"6e29360bfaccb0d159e1a59cee79a143ef5c6dd6ee8c53c6596d5fb050239128"},"attempts/nominal-00--hold.json":{"bytes":3051655,"sha256":"9fd5ce65a0dae6f5f10a96067e9348e20684943d80f4c4219f5593ead3a1c26f"},"attempts/nominal-01--placement.json":{"bytes":3325448,"sha256":"d2ad68dee1a5a6d9905f91700e9d0ee11601f9a078d3b110d238b643a629b6e0"},"attempts/nominal-01--replay_open_jaw.json":{"bytes":3255424,"sha256":"02ca2be312d9aa1f477522296b84be9998d1923ba34c2f6bf9f4aa19d372c03c"},"attempts/nominal-01--hold.json":{"bytes":3061802,"sha256":"12e360442110790d79f362dfe62bbd225618f713dd9348b39a79fa0cd2cc3f83"},"attempts/low_friction-00--placement.json":{"bytes":3266080,"sha256":"5f33b7f4ff0fb439f6539b9b930b4bac165be174f3c94d91f4476fccd2e4113b"},"attempts/low_friction-00--replay_open_jaw.json":{"bytes":3267631,"sha256":"621079faf79775dcbe3752aa8dea090dcd27d46b51b92422a4bf1dcc1725eee6"},"attempts/low_friction-00--hold.json":{"bytes":3062416,"sha256":"1bb3a710fb5c05897f97c0ba31cff6a08978b336d452742b25252ef7fd65ba0e"}}}
Loading
Loading