Skip to content

Publish paired arm benchmark and Skillspace v0.3 release - #11

Merged
Tarzelf merged 2 commits into
mainfrom
codex/arm-benchmark-evolution-20261007
Oct 7, 2026
Merged

Tarzelf merged 2 commits into
mainfrom
codex/arm-benchmark-evolution-20261007

Conversation

@Tarzelf

@Tarzelf Tarzelf commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Question

What are the present Skillspace arm capability, DOF/gripper input contract, and evidence needed for future downloadable skills? Physical grounding.

Evidence

The Skillspace arm lab now exposes object dimensions, mass, contact parameters and actuator limits, binds installation/execution to exact runtime sources, and reports joint/gripper feedback plus bounded recovery failures. Publish its v0.3 source and a frozen paired evaluation: development completion rises from 14/20 to 17/20, but held-out completion declines from 30/32 to 29/32 and observed CPU throughput declines from 12.28× to 11.39×. The report preserves this regression and the uncalibrated, simulation-only scope.

The proposed future Skillspace inspector separates media, datasets, grounded scenarios, controllers and evaluation bindings; drafts never claim simulation or hardware qualification. Existing research evidence remains unchanged, and individual contributor attribution remains empty.

Reproduction

Source archive and full receipts are included under topics/physical-grounding/runs/2026-10-07-arm-benchmark-v1/. Python 3.12.9, MuJoCo 3.15.0, NumPy 2.5.3, Apple M5 CPU, one native world. python -m unittest discover -s tests -q; python -m simlab.studio --offline --port 8240; individual frozen DEV reproduction documented in README. No learned-from-video policy, physical qualification, GPU measurements or air-gapped dependency installation claim.

Checks

  • 100 distinct Python tests covered: 99-test full run plus a targeted 26-test subset containing one additional test.

  • Native network-denied execution: 3/3 known layouts; extracted final archive: three visible development cases reproduced exactly, with all 85 manifest file hashes checked.

  • 32 held-out cases per controller, six negative controls per controller, unchanged contact/success rules, zero captured engine warnings.

  • Research build/link validation and all 23 publication tests pass; dedicated research Worker dry-run passes.

  • Primary documentation is cited; Grok opinions are separated from evidence.

  • Release scan and deterministic build checks exclude private session paths, credentials and personal footage.

  • Source LICENSE and pinned dependency list are included; no model weights or third-party dataset is redistributed.

  • npm run check and all 23 publication tests pass; dedicated research Worker dry-run passes.

@Tarzelf
Tarzelf merged commit f6bf76f into main Oct 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant