Product implementation and measured synthetic pilot
Skillspace footage training pipeline
Public CPU package · one native scene qualified · simulation-only
Abstract
The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.
In this paper
The public, standalone DVIDIA training repository provides a local CPU pipeline that turns actual Skillspace media into a fitted visual model and, when aligned numeric actions are supplied, a learned movement head. The useful advance is a reproducible path from source files through provenance checks, grouped splits, measured fitting and portable artifacts. Video prediction and robot movement remain separate claims. Ten synthetic six-second clips demonstrate visual fitting; simulator telemetry supplies the movement pilot's action labels. Its exported capsule installs and passes native qualification in one exact scene, while identical joint targets with open jaws fail. Neither experiment learns shoe organization or reconstructs robot actions from human footage. This report documents the implemented path and its limited measured evidence, rather than establishing a sufficient demonstration count or hardware competence. (Standalone source archive, Qualification receipt)
+Skip to contentProduct implementation and measured synthetic pilot
Skillspace footage training pipeline
Public CPU package · one native scene qualified · simulation-only
Abstract
The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.
In this paper
The public, standalone DVIDIA training repository provides a local CPU pipeline that turns actual Skillspace media into a fitted visual model and, when aligned numeric actions are supplied, a learned movement head. The useful advance is a reproducible path from source files through provenance checks, grouped splits, measured fitting and portable artifacts. Video prediction and robot movement remain separate claims. Ten synthetic six-second clips demonstrate visual fitting; simulator telemetry supplies the movement pilot's action labels. Its exported capsule installs and passes native qualification in one exact scene, while identical joint targets with open jaws fail. Neither experiment learns shoe organization or reconstructs robot actions from human footage. This report documents the implemented path and its limited measured evidence, rather than establishing a sufficient demonstration count or hardware competence. (Standalone source archive, Qualification receipt)
Published artifacts on Hugging Face
The DVIDIA organization on Hugging Face mirrors the measured pilot for inspection and local reproduction:
-
@@ -23,6 +23,12 @@
Executable
Its grouped split contains seven training, two development and one test recording, totaling 116.5 seconds of footage. A normalized radial-basis ridge model fits 4,029 training labels, chooses regularization on development data and reaches the joint-delta error shown above on 621 test samples. This is an offline prediction result from one held-out recording. It does not measure closed-loop success, independent scene generalization or improvement over the earlier controller. The learned head maps privileged simulator context and a requested goal to joint deltas; phases, waypoints, grasping and recovery remain authored. This scope follows the earlier skill-capsule prototype. (Movement measurements, Run receipt, Skill capsule acquisition)
An explicit supported task source permits export of a runtime-bound simulation capsule. The standalone run records 1.482 seconds on its CPU pipeline timer, excluding final ZIP creation and integrity verification. It exports a 14,843-byte candidate, passes inspection and installs it. This describes one small run, not a throughput comparison. An earlier filename-handoff failure remains recorded in the release ledger. (Run receipt, Release ledger)
Installation alone does not enable qualified execution. A separate native test in the exact scene of the held-out recording completes placement in 12.952 simulated seconds with 0.311 mm target error. Replaying the same joint targets with open jaws fails at the 18-second horizon. This supports the contact-dependent execution path in that scene; it adds no independent scene sample or hardware evidence. Artifact hashes establish which candidate and runtime were tested, not physical accuracy. (Qualification receipt, Native replay)
+8 October 2026: frozen sample-count follow-up
+A separate follow-up compares nested sets of two, four and seven whole connected training groups from the published native simulator recordings, with 1,135, 2,281 and 4,029 aligned action samples respectively. The same two development recordings, one test recording and twelve evaluation layouts remain fixed across conditions. The protocol freezes source, scene, training-code and runtime hashes before fitting; all three movement candidates are fitted, exported and installed before any evaluation scene runs. Each installed candidate uses the existing exact-scene qualification gate, with an identical-target open-jaw control. An authored teacher baseline runs once per scene. (Benchmark guide, Frozen protocol)
+| Installed candidate or baseline | Nominal completion | Actuator-stress completion |
|---|---|---|
| Two training groups | 8/8 | 3/4 |
| Four training groups | 7/8 | 3/4 |
| Seven training groups | 8/8 | 3/4 |
| Authored teacher | 8/8 | 3/4 |
Every successful candidate episode also passes its scene-specific qualification predicate. All 36 open-jaw controls fail, and the experiment records zero execution errors. The failure cases remain in each planned denominator: all three candidates and the teacher fail the same actuator-stress scene, while the four-group candidate also fails one nominal scene. The seven-group candidate's nominal target error is 0.267 mm mean and 0.483 mm maximum, measured across all eight nominal episodes, which all succeed. These results extend the execution evidence to the frozen layouts without changing the original one-scene pilot. (Final report and per-scene outcomes, Experiment summary)
+The benchmark takes 102.7268 seconds of CPU wall time on an Apple M5 host with 24 GiB RAM. Its timer includes dependency imports inside the benchmark function, input validation and copies, scene planning, fitting, installation, artifact output and 84 native episodes: twelve teacher episodes, 36 candidate episodes and 36 controls. Process startup, top-level module imports and final report writing are excluded. Native episode clocks retain their narrower scope; throughput in the report divides total simulated time by total episode wall time. This run establishes neither minimum hardware nor an end-to-end speedup. (Timing scope and measurements)
+This is a single composition-order pilot: increasing the count changes which training groups are included, and the nonmonotonic completion counts do not establish a learning curve or a sufficient demonstration count. All conditions reuse the same twelve distinct scenes; condition-by-scene runs are not independent scenes, and nominal and actuator-stress results remain separate. Layout exclusion checks only the exact object/target positions declared in hash-bound native telemetry, without independent capture attestation or an out-of-distribution claim. The movement head consumes privileged simulator state and aligned numerical actions; the RGB predictor does not control the arm, and phases, targets and gripper logic remain authored. Human-video action recovery and hardware qualification remain unestablished. (Frozen protocol and limitations)
The next experiment should measure the missing bridge
The current local path avoids API dependence once its tools and data are installed. Local media decoding restricts FFmpeg protocols, while optional remote intake is explicit. Separate continuous-integration checks passed visual fitting and studio tests on Linux, macOS and Windows with Python 3.12 and 3.13, the optional Linux arm pilot, and an actual Docker example, training run and inspection under --network none. The image build required network access; this does not establish installation on a new air-gapped machine or minimum hardware. No GPU scaling, throughput improvement over another implementation or hardware transfer follows from the pilots. The package makes experiments inspectable and repeatable; its scientific value depends on what demonstrations, labels and task environments enter it. (Passing software checks, Source and training guide)
For shoe organization, the next decisive step is a reviewed corpus of complete one-shoe-to-marked-slot attempts and a validated observation-to-action bridge for a defined arm, gripper and sensing arrangement. Sample-count comparisons should hold architecture, task definition and evaluation fixed, group related originals, and report independent closed-loop episodes alongside prediction error. Until those components exist, a shorter visual training run cannot answer how many human videos make a robot competent. The pipeline's contribution is to preserve those distinctions in executable product gates, making the missing evidence measurable before a candidate becomes a downloadable skill.
diff --git a/reports/Skillspace footage training pipeline.md b/reports/Skillspace footage training pipeline.md index 4bd972c..540c835 100644 --- a/reports/Skillspace footage training pipeline.md +++ b/reports/Skillspace footage training pipeline.md @@ -41,6 +41,23 @@ An explicit supported task source permits export of a runtime-bound simulation c Installation alone does not enable qualified execution. A separate native test in the exact scene of the held-out recording completes placement in **12.952 simulated seconds with 0.311 mm target error**. Replaying the same joint targets with open jaws fails at the 18-second horizon. This supports the contact-dependent execution path in that scene; it adds no independent scene sample or hardware evidence. Artifact hashes establish which candidate and runtime were tested, not physical accuracy. ([Qualification receipt](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-08-footage-pipeline-v1/qualification/qualification.json), [Native replay](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-08-footage-pipeline-v1/qualification/replay)) +## 8 October 2026: frozen sample-count follow-up + +A separate follow-up compares nested sets of **two, four and seven whole connected training groups** from the published native simulator recordings, with 1,135, 2,281 and 4,029 aligned action samples respectively. The same two development recordings, one test recording and twelve evaluation layouts remain fixed across conditions. The protocol freezes source, scene, training-code and runtime hashes before fitting; all three movement candidates are fitted, exported and installed before any evaluation scene runs. Each installed candidate uses the existing exact-scene qualification gate, with an identical-target open-jaw control. An authored teacher baseline runs once per scene. ([Benchmark guide](https://github.com/Dvidia-Inference/dvidia-training/blob/main/docs/benchmarks.md), [Frozen protocol](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/protocol.json)) + +| Installed candidate or baseline | Nominal completion | Actuator-stress completion | +|---|---:|---:| +| Two training groups | 8/8 | 3/4 | +| Four training groups | 7/8 | 3/4 | +| Seven training groups | 8/8 | 3/4 | +| Authored teacher | 8/8 | 3/4 | + +Every successful candidate episode also passes its scene-specific qualification predicate. All **36 open-jaw controls fail**, and the experiment records **zero execution errors**. The failure cases remain in each planned denominator: all three candidates and the teacher fail the same actuator-stress scene, while the four-group candidate also fails one nominal scene. The seven-group candidate's nominal target error is **0.267 mm mean and 0.483 mm maximum**, measured across all eight nominal episodes, which all succeed. These results extend the execution evidence to the frozen layouts without changing the original one-scene pilot. ([Final report and per-scene outcomes](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/report.json), [Experiment summary](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/README.md)) + +The benchmark takes **102.7268 seconds of CPU wall time** on an Apple M5 host with 24 GiB RAM. Its timer includes dependency imports inside the benchmark function, input validation and copies, scene planning, fitting, installation, artifact output and **84 native episodes**: twelve teacher episodes, 36 candidate episodes and 36 controls. Process startup, top-level module imports and final report writing are excluded. Native episode clocks retain their narrower scope; throughput in the report divides total simulated time by total episode wall time. This run establishes neither minimum hardware nor an end-to-end speedup. ([Timing scope and measurements](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/report.json)) + +This is a **single composition-order pilot**: increasing the count changes which training groups are included, and the nonmonotonic completion counts do not establish a learning curve or a sufficient demonstration count. All conditions reuse the same twelve distinct scenes; condition-by-scene runs are not independent scenes, and nominal and actuator-stress results remain separate. Layout exclusion checks only the exact object/target positions declared in hash-bound native telemetry, without independent capture attestation or an out-of-distribution claim. The movement head consumes privileged simulator state and aligned numerical actions; the RGB predictor does not control the arm, and phases, targets and gripper logic remain authored. Human-video action recovery and hardware qualification remain unestablished. ([Frozen protocol and limitations](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/protocol.json)) + ## The next experiment should measure the missing bridge The current local path avoids API dependence once its tools and data are installed. Local media decoding restricts FFmpeg protocols, while optional remote intake is explicit. Separate continuous-integration checks passed visual fitting and studio tests on Linux, macOS and Windows with Python 3.12 and 3.13, the optional Linux arm pilot, and an actual Docker example, training run and inspection under `--network none`. The image build required network access; this does not establish installation on a new air-gapped machine or minimum hardware. No GPU scaling, throughput improvement over another implementation or hardware transfer follows from the pilots. The package makes experiments inspectable and repeatable; its scientific value depends on what demonstrations, labels and task environments enter it. ([Passing software checks](https://github.com/Dvidia-Inference/dvidia-training/actions/runs/37740260434), [Source and training guide](https://research.dvidia.org/downloads/topics/physical-grounding/runs/2026-10-08-footage-pipeline-v1/release/dvidia-training-v0.1.zip))