Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
# Review the viewer source; this reproducible bundle includes upstream GLSL text.
src/dvidia_training/reference_replay.bundle.js linguist-generated=true -diff
src/dvidia_training/workstation.bundle.js linguist-generated=true -diff
src/dvidia_training/workstation_worker.bundle.js linguist-generated=true -diff
src/dvidia_training/mujoco.wasm binary
71 changes: 71 additions & 0 deletions .github/workflows/workstation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Live workstation
on:
push:
pull_request:
workflow_dispatch:
permissions:
contents: read
concurrency:
group: workstation-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
native-browser:
runs-on: ubuntu-latest
timeout-minutes: 20
env:
OPENBLAS_NUM_THREADS: "1"
OMP_NUM_THREADS: "1"
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09
with:
persist-credentials: false
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1
with:
python-version: "3.12"
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "22"
- name: Reproduce pinned browser runtime
run: |
npm ci --ignore-scripts --prefix viewer
npm run --prefix viewer build:workstation
git diff --exit-code -- src/dvidia_training/workstation.bundle.js src/dvidia_training/workstation_worker.bundle.js src/dvidia_training/mujoco.wasm src/dvidia_training/workstation.third-party.txt
node --test tests/browser/workstation_control.mjs tests/browser/workstation_assets.mjs
- name: Install native mechanics and verify contracts
run: |
python -m pip install '.[arm]'
dvidia-robot-sim fetch piper --output runs/workstation-assets
DVIDIA_ROBOT_PROFILE_ASSETS=runs/workstation-assets python -m unittest tests.test_workstation tests.test_workstation_benchmark -v
dvidia-workstation --assets runs/workstation-assets --output runs/workstation
dvidia-workstation-benchmark --assets runs/workstation-assets --output runs/workstation-benchmark --stability-seconds 10
- name: Compare native and WASM state with loaded-contact counts
run: |
python scripts/workstation-parity-fixture.py --scene runs/workstation --output runs/native-parity.json
python - <<'PY'
import json, subprocess
result = subprocess.run(['node','tests/browser/workstation_parity.mjs','runs/workstation','runs/native-parity.json','runs/wasm-parity.json'])
report = json.load(open('runs/wasm-parity.json'))
assert result.returncode in (0,1)
assert report['valid'] and report['numerical_parity_passed'] and report['active_contact_count_parity_passed']
print('Raw contact count parity is retained separately:', report['strict_parity_passed'])
PY
- name: Exercise actual browser controls and live physics
run: |
viewer/node_modules/.bin/playwright install --with-deps chromium
python -m http.server 8295 --bind 127.0.0.1 --directory runs/workstation > /tmp/workstation-http.log 2>&1 &
server_pid=$!
trap 'kill "$server_pid"' EXIT
DVIDIA_PLAYWRIGHT_MODULE="$GITHUB_WORKSPACE/viewer/node_modules/playwright" DVIDIA_BENCH_SECONDS=20 node tests/browser/workstation_live.cjs http://127.0.0.1:8295/ runs/workstation-browser
node tests/browser/workstation_download.cjs http://127.0.0.1:8295/ runs/workstation-download-recovery
- name: Retain all results including failed attempts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: live-workstation-evidence
path: |
runs/workstation-benchmark
runs/workstation-browser
runs/workstation-download-recovery
runs/native-parity.json
runs/wasm-parity.json
if-no-files-found: ignore
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@

**Research alpha.** The recorded demo covers the original one-scene simulation pilot; the frozen benchmark is a separate experiment. Training runs locally. No human-video action bridge or hardware skill is established.

**[Open the live PiPER shoe workstation](https://dvidia-robot-simulation-lab.static.hf.space/workstation/v01/index.html).**
Drag the tool target, move the native gripper, nudge a free shoe and inspect the
resulting contacts. MuJoCo 3.15.0 runs locally in a browser worker; a warm 3D
workstation makes the native mechanics directly interactive. The shoe remains a
rigid synthetic asset and the controller is authored. Read the
[local setup and benchmark guide](docs/live-workstation.md) and
[research results](https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/).

The new [native grasp task benchmark](docs/native-task-benchmark.md) connects the
pinned six-axis arm and coupled gripper to free-object grasp, carry and release
attempts. It retains matched open-jaw/hold controls, frozen protocols and local
Expand Down
161 changes: 161 additions & 0 deletions docs/live-workstation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Live PiPER shoe workstation v0.1

[Open the workstation](https://dvidia-robot-simulation-lab.static.hf.space/workstation/v01/index.html)
or read the [research and complete benchmark evidence](https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/).

The six-axis PiPER and its native gripper respond to dragged or typed tool-position
targets. A free rigid trainer falls, settles on the worktop and moves under a
bounded physical nudge. Pause, reset, camera views, collision geometry and a local
session receipt make the mechanics inspectable. No account or inference service
is needed to run the hosted workstation.

## Run a complete copy locally

Use Python 3.12 or newer and the published source revision. Provision the package
and upstream robot assets once:

```sh
git clone --branch feat/live-piper-workstation-v01 https://github.com/Dvidia-Inference/dvidia-training.git
cd dvidia-training
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install '.[arm]'
dvidia-robot-sim fetch piper --output ./robot-assets
dvidia-workstation --assets ./robot-assets --output ./live-workstation
python -m http.server 8000 --bind 127.0.0.1 --directory ./live-workstation
```

Open http://localhost:8000. Keep the exported folder together. All meshes,
JavaScript, WASM, geometry, catalog and license notices are local. Later sessions
work offline. On Windows use `py -3.12 -m venv .venv` and the corresponding
executables under `.venv\Scripts`; activation is optional.

The browser fetches one gzip bundle for all 84 native robot meshes, concurrently
with the physics engine and scene. It verifies the bundle, bounds decompression,
and checks every original mesh hash before compiling the model. The original
individual meshes remain in the export for reproduction. This changes delivery
only; the scene, controller, contacts and engine binary are unchanged. First-load
time still depends on the connection and device; it is measured separately from
steady-state simulation speed.

The WASM engine downloads as `mujoco.wasm.gz` (2.54 MB instead of 10.37 MB),
then decompresses to the exact original engine bytes verified by SHA-256.
The original uncompressed engine remains in the export. Geometry, engine and
mesh downloads share a bounded downloader: at most three total attempts,
30 seconds without incoming data before an attempt is stopped, and a
120-second total deadline per resource. Only transient network/body-stream
failures and selected temporary HTTP errors retry; invalid sizes, checksums or
destinations fail immediately. Loading progress names retries; a terminal
failure exposes **Reload and retry**. Reload starts a new session.

The official Hugging Face Space downloads three known compressed assets from
content-addressed URLs on `research.dvidia.org`. The requested size and SHA-256
must match the published mirror inventory, and the received bytes are checked
before use. A temporary mirror failure falls back to Hugging Face within the
same retry and time budget; a checksum, size or redirect-policy failure stops.
This routing applies only to the exact official Space and asset paths. Local
exports and other deployments use their own included files.

On 2026-10-13 the earlier Hugging Face startup failure could not be reproduced
as a deterministic CORS, gzip or corrupt-asset defect. Fresh diagnostics verified
all three old binary assets. The exact historical network event was not captured;
the confirmed defect was the absence of bounded transfer recovery. The download
regression test deliberately interrupts each large download and verifies native
control after recovery, then exhausts retries and tests a manual reload:

```sh
node tests/browser/workstation_download.cjs http://localhost:8000/ ./download-recovery-evidence
```

The export includes an integrity inventory and hashed upstream assets. To inspect:

```sh
python -c "from dvidia_training.workstation import inspect_export; print(inspect_export('./live-workstation')['file_count'])"
```

## Controls and mechanics

- Drag a colored target axis or enter X/Y/Z in millimeters. Height and keyboard
controls provide alternatives to pointer dragging.
- The controller holds one declared tool orientation while solving six arm joint
targets. The allowed input box is a command bound, not a promise that every
point is reachable at that orientation. The measured target error stays visible.
- Finger travel spans 0–35 mm for one side of the coupled gripper. It is a native
position command, not a calibrated jaw gap or force.
- Nudge applies at most 0.12 N·s for one physics tick. It is an explicit user
disturbance, never a hidden lifting or attachment force.
- Reset restores the declared initial state. Hidden tabs pause; resume explicitly
after returning. Session receipts retain a bounded recent history and disclose
dropped samples and resets.

MuJoCo advances at a fixed 2 ms physics timestep. Differential IK runs every
20 ms. Joint-command rates and original servo limits remain enforced. Browser
state updates and rendering run separately, with bounded catch-up and disclosed
lag when a device cannot keep up. There is no shoe weld, object teleport during
interaction, neural policy or physical robot connection.

## Reproduce the benchmarks

```sh
dvidia-workstation-benchmark --assets ./robot-assets --output ./workstation-benchmark
```

The protocol records two resets for eight target positions, jaw close/open travel,
a shoe impulse, disabled-table gravity control and 600 simulated seconds of
continuous native holding. Each tracking attempt has a four-second deadline and
requires both 5 mm position and 3° orientation tolerances. Every attempt and raw
control-window duration remains in the output, including failures. Native timing
excludes browser rendering and network. This evaluates target tracking and
contact diagnostics; it does not score an autonomous shoe-organizing skill.

Browser acceptance and native/WASM parity are separate:

```sh
PYTHONPATH=src python scripts/workstation-parity-fixture.py --scene ./live-workstation --output ./native-parity.json
node tests/browser/workstation_parity.mjs ./live-workstation ./native-parity.json ./wasm-parity.json
```

The actual-browser tests use the same exported folder served over HTTP:

```sh
npm ci --prefix viewer
viewer/node_modules/.bin/playwright install chromium
node tests/browser/workstation_ui.cjs http://localhost:8000/ ./ui-evidence
DVIDIA_BENCH_SECONDS=600 DVIDIA_BENCH_RENDERER=device DVIDIA_BROWSER_CHANNEL=chromium node tests/browser/workstation_live.cjs http://localhost:8000/ ./browser-evidence
```

The device renderer is requested, then identified in the receipt. The live benchmark defaults to
SwiftShader software rendering. Neither render-call counts nor worker state
updates measure GPU presentation completion or hardware actuator latency.

The parity runner requires `npm ci --prefix viewer`. It retains numerical,
loaded-contact and strict raw contact-count comparisons separately. A strict
contact-count mismatch exits nonzero even when state tolerances agree; inspect
all result fields. Floating-point differences around zero-load contacts must not
be presented as exact contact parity.

## Hardware and licensing

Physics runs on a CPU. Browser rendering requires WebGL2; JavaScript,
WebAssembly and DecompressionStream must be available. No CUDA training GPU or
large model download is required. Minimum RAM and CPU specifications have not
been established across devices. Published measurements name their browser,
rendering backend and timing scope; software-rendered browser results are not
hardware GPU measurements.

DVIDIA code, original shoe shapes and procedural visual details are MIT. PiPER
assets retain their pinned upstream MIT notices. Three.js is MIT. MuJoCo is
Apache-2.0 with dependency notices retained in `workstation.third-party.txt`.
The header uses DVIDIA's existing mark and wordmark. The embedded IBM Plex Sans
wordmark font retains its SIL Open Font License 1.1 notice in the HTML; the
software license does not grant rights to DVIDIA trademarks.
Visual shoelaces and surface details are cosmetic; they do not simulate cloth or
change collision geometry. Shoe mass, friction and contact parameters remain
authored assumptions. Retained shoe lifting and real-world transfer are open.

To rebuild the browser runtime with pinned dependencies:

```sh
npm ci --prefix viewer
npm run --prefix viewer build:workstation
```
Loading
Loading