Skip to content

Add native grasp benchmarks, additional robot profiles and public simulation lab - #11

Open
Tarzelf wants to merge 7 commits into
feat/reference-3d-replayfrom
feat/reference-task-benchmark
Open

Tarzelf wants to merge 7 commits into
feat/reference-3d-replayfrom
feat/reference-task-benchmark

Conversation

@Tarzelf

@Tarzelf Tarzelf commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

What changes

The simulation library now includes pinned PiPER and SO-101 CPU profiles, plus native free-object grasp/carry/release benchmarks for UR5e/Robotiq and PiPER. Each profile declares axes, gripper mechanics, native command units, upstream model identity and original license. PiPER has six arm axes plus a gripper; SO-101 has five arm axes plus a gripper and currently has unloaded mechanics evidence.

The contact tasks use authored controllers with privileged simulator state. Their frozen protocols retain every planned attempt, matched open-jaw and hold controls, complete states, issued commands, contact outcomes and timing. The new dvidia-piper-task benchmark/inspect command runs locally after explicit model provisioning. Artifact checks validate source/model/control consistency, including partial failures and terminal commands; they do not authenticate a publisher or prove physical accuracy.

The public Hugging Face lab carries the recorded UR5e 3D replay, separate numerical-profile bundles and complete PiPER contact attempts. PiPER has no browser 3D replay yet. The hosted viewer now follows its required public download redirect only on the exact DVIDIA Space host, without credentials/referrers; other deployments remain strict.

Evidence and validation

  • UR5e frozen cohort: 6/6 placements, 0/6 open-jaw, 0/6 hold; all 18 attempts retained, including timing outliers.
  • PiPER fresh frozen cohort: 2/2 nominal placements, 0/1 low-friction placement, 0/3 open-jaw, 0/3 hold; all 9 numerically valid, full 20-second traces. Nominal final error 4.643–4.682 mm in simulator truth. One of 9,000 computation windows exceeded 20 ms; no real-time claim.
  • PiPER cohort ran from an installed wheel with networking denied. Source hashes matched the frozen implementation. PiPER/SO-101 unloaded numerical groups each retain all 6 attempts.
  • Native PiPER tests: 11 passed. PiPER benchmark/inspector tests: 11 passed, including invalid partial traces, matched control tampering, source/license mismatch and endpoint evidence. Profile tests: 12 passed.
  • Native replay tests: 23 passed against pinned assets. Offline loader-origin/byte checks passed. Public desktop and 320/390 px replay rendering passed after the hosting fix.
  • Publication tests: 12 publisher and 11 preparer tests passed. Source implementation checks run in GitHub CI; base suite previously passed 537 tests.

Scope

Stacked research change based on feat/reference-3d-replay; install the candidate branch or an explicit commit. This does not establish learned robot skills, shoe manipulation, a human-video action bridge, hardware accuracy or camera-to-actuator reaction time. Next: larger task coverage, compact learned policies, PiPER visualization and measured hardware comparison.

@Tarzelf Tarzelf changed the title Benchmark native grasp tasks with complete contact evidence Add native grasp benchmarks, additional robot profiles and public simulation lab Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant