From 0ddb9bffa6e6cbc8846f59c962ed39194ff501bd Mon Sep 17 00:00:00 2001 From: Tarzelf Date: Mon, 19 Oct 2026 11:09:15 -0400 Subject: [PATCH] Publish sensor corroboration v0.4 research note --- .../reports/Sensor corroboration v04.md | 62 +++++++++++++++ docs/feed.xml | 2 +- docs/index.html | 2 +- .../index.html | 2 +- docs/papers/dvidia-observation-lab/index.html | 2 +- .../index.html | 2 +- .../from-skillspace-to-robot-skill/index.html | 2 +- .../grok-robotics-research-on-x/index.html | 2 +- .../index.html | 2 +- .../native-grasp-task-benchmark/index.html | 2 +- .../index.html | 2 +- .../physical-skill-research-agenda/index.html | 2 +- .../piper-contact-history-v03/index.html | 2 +- .../piper-contact-recovery-v04/index.html | 2 +- .../index.html | 2 +- docs/papers/piper-reliability-v05/index.html | 2 +- .../robotics-research-pipeline/index.html | 2 +- .../sensor-corroboration-v04/index.html | 30 ++++++++ docs/papers/shoe-asset-library-v01/index.html | 2 +- .../skill-capsule-acquisition/index.html | 2 +- .../skillspace-arm-benchmarks/index.html | 2 +- .../skillspace-arm-installation/index.html | 2 +- .../skillspace-footage-training/index.html | 2 +- .../index.html | 2 +- docs/publications.json | 13 ++++ docs/sitemap.xml | 2 +- docs/topics/capture-quality/index.html | 2 +- docs/topics/evaluation-provenance/index.html | 2 +- docs/topics/physical-grounding/index.html | 2 +- docs/topics/skill-relations/index.html | 2 +- publications.json | 75 +++++++++++-------- reports/Sensor corroboration v04.md | 62 +++++++++++++++ 32 files changed, 238 insertions(+), 58 deletions(-) create mode 100644 docs/downloads/reports/Sensor corroboration v04.md create mode 100644 docs/papers/sensor-corroboration-v04/index.html create mode 100644 reports/Sensor corroboration v04.md diff --git a/docs/downloads/reports/Sensor corroboration v04.md b/docs/downloads/reports/Sensor corroboration v04.md new file mode 100644 index 0000000..6f98caf --- /dev/null +++ b/docs/downloads/reports/Sensor corroboration v04.md @@ -0,0 +1,62 @@ +# Independent camera and actuator corroboration v0.4 + +DVIDIA robotics research · 19 October 2026. + +The [local physical-goal verifier v0.3](https://research.dvidia.org/papers/local-goal-verifier-v03/) trusts its observation packet. This experiment adds separately processed RGB and PiPER drive channels, then asks whether those channels can contradict a primary “complete” reading. It is a simulator study of evidence fusion, not a hardware safety claim. + +The frozen benchmark contains **272 cases across eight layouts**: 96 native scenes, 80 primary-packet forgeries, 64 sensor faults and 32 cross-channel controls. All 96 native cases generated validly. The v0.3 baseline called 192 cases complete, including 112 physical false completions. The v0.4 gate retained 50 complete verdicts: seven physical positives and 43 false completions. It prevented 69/112 baseline false completions, but 43 remained. It also abstained on 73/80 baseline true completions; 64 had deliberately missing, stale, gapped or malformed auxiliary evidence, and nine were physically complete but visually unobservable. + +The result is useful because it shows both what a second signal catches and what two weak signals cannot establish. The camera detects large motion, position disagreement and many target-visibility failures. The drive path checks finger aperture, command, motor effort, settling and servo consistency. Neither authenticates its source. These channels still miss a 180° orientation reversal, a green lookalike, elevated support, motion that aliases between samples, and refreshed or fully replayed channels. + +## Paired outcome + +| Measure | v0.3 primary packet | v0.4 camera + drive gate | +| --- | ---: | ---: | +| Scheduled cases | 272 | 272 | +| Complete verdicts | 192 | 50 | +| Physically true completions recognized | 80/80 | 7/80 | +| False completions among physical negatives | 112/192 | 43/192 | +| Baseline false completions prevented | — | 69/112 | +| Baseline true completions lost | — | 73/80 | +| Unknown verdicts | 0/272 | 142/272 | +| Invalid native generation | 0 | 0 | + +The 64 sensor-fault controls remain physically complete; all 64 return `unknown` when their required camera or drive evidence is missing or invalid. In the native cohort, seven of eight visible settled scenes pass. Every explicitly occluded scene abstains, and one nominally visible shoe is obscured by the simulated arm. These are conservative refusals, not evidence that the physical task failed. + +Among the 112 baseline false completions, the gate prevents 69 and leaves 43. The residual cases are not one generic detector error: + +| Residual control | False completions that pass | +| --- | ---: | +| Instantaneous motion alias | 7/8 | +| 180° yaw flip | 8/8 | +| Visually identical green lookalike | 7/8 | +| Elevated object with hidden support surface | 7/8 | +| Refreshed camera replay | 7/8 | +| Coherent replay of all channels | 7/8 | + +The layout with an occluded camera path abstains, which is why several controls show seven rather than eight false completions. A coherent replay remains compatible with the channels' declared clock labels; there is no cryptographic sensor provenance in this simulator. + +## What the channels measure + +The camera observer receives only eleven 640×480 RGB frames at 20 ms intervals. A fixed orthographic projection maps a green elongated proxy to XY position and an undirected axis. It receives no object ID, simulator pose, segmentation mask or physical truth. Its color-and-shape rule is not a general shoe detector. Axial geometry cannot distinguish the toe from a 180°-reversed shoe, a same-color decoy is ambiguous, and RGB alone cannot measure height above the table. + +The drive observer receives one hundred PiPER joint/drive samples at 2 ms intervals: six arm channels, two signed finger positions and velocities, gripper command and simulated motor effort. It validates ranges, units, finger symmetry and a declared spring/servo consistency relation, then checks that the fingers are open, commanded open, settled and at low effort. Motor effort is not object contact force. Neither channel can verify table support or rule out contact from every arm link. + +Both channels originate in the same MuJoCo model as the primary packet. They use separate processing code, but are **not independent physical sensors**. Calibration bounds and IDs are synthetic declarations, not authenticated measurements. An unknown result means the verifier refuses to confirm; it does not mean the scene is physically incomplete. + +## Compute and limits + +On the recorded Apple Silicon host, the complete processed decision path had **22.1 ms median and 31.5 ms p95** across 816 repeated case evaluations. RGB extraction alone had 14.3 ms median and 20.1 ms p95; the baseline verifier had 0.42 ms median. The optional OpenCV CPU kernel was installed for this run. No GPU or hosted model call was used. + +The processed-path timer includes local JSON reads, PNG decoding, primary verification, RGB observation, drive observation and fusion. It excludes the 200 ms observation history, native rendering/acquisition, networking and robot actuation. The p95 is longer than a 20 ms camera frame interval, so this is not a demonstrated real-time control loop or hard-real-time bound. The observer uses an optional CPU package for speed and keeps a NumPy-only fallback; no minimum hardware specification has been established. + +The next useful measurement is a hardware-in-the-loop capture with timestamped camera, encoders, motor current and a second view that can see object height and support. Before that, the simulator needs stronger toe-direction and identity cues, explicit bounded-motion claims, and anti-replay provenance. Training a decision model on these labels would not remove those missing observables. + +## Reproduce and inspect + +- [Interactive case viewer and downloadable inputs](https://dvidia-robot-simulation-lab.static.hf.space/image-goals/v04/index.html) +- [Pinned implementation](https://github.com/Dvidia-Inference/dvidia-training/tree/90e84701ce7d5bc330ef41b02793e2ce037d702d) +- [Full run directory and scored results](https://github.com/Dvidia-Inference/dvidia-training/tree/main/runs/sensor-goal-verifier-v04-final) +- [Protocol and reproduction commands](https://github.com/Dvidia-Inference/dvidia-training/blob/main/docs/sensor-goal-verifier.md) + +Source commit `90e84701ce7d5bc330ef41b02793e2ce037d702d` freezes the final seed, eight layouts, 272-case inventory, and three timing repetitions. Every scheduled case and every sensor stream remains in the release. No provider, Jev API, physical robot or learned controller was used. Original DVIDIA code is MIT; the pinned simulator and robot assets retain their upstream terms. diff --git a/docs/feed.xml b/docs/feed.xml index fea03bc..89691e3 100644 --- a/docs/feed.xml +++ b/docs/feed.xml @@ -1,2 +1,2 @@ -DVIDIA Researchhttps://research.dvidia.org/Research footprints on demonstrations, calibrated simulation and robot skills.enA local verifier for physical completionhttps://research.dvidia.org/papers/local-goal-verifier-v03/https://research.dvidia.org/papers/local-goal-verifier-v03/Mon, 19 Oct 2026 12:00:00 GMTNative: 16/16 true complete, 0/48 false complete · forged readings: 8/8 false complete. An offline audit traces the seven missed Jev completions without claiming an internal model cause. A local deterministic verifier then checks timestamped pose, motion, support, release and uncertainty on 200 fresh cases. It recognizes all 16 native positives and rejects all 48 native negatives; 128 evidence faults yield 120 abstentions and 8 incomplete answers. All 8 consistent forged streams produce physical false completions, exposing the remaining sensor-trust problem. CPU-only p95 is 0.318 ms, excluding the required 200 ms history and sensing. Complete native inputs, every decision, independent review and explicit failure cohorts remain public; no new Jev calls or hardware qualification.Jev image-goal verification: the measured follow-uphttps://research.dvidia.org/papers/image-goal-jev-v02/https://research.dvidia.org/papers/image-goal-jev-v02/Mon, 19 Oct 2026 12:00:00 GMT64/64 valid responses · telemetry: 0/24 false complete, 1/8 true complete. The first measured run preserves all 64 requests from the image-goal v0.1 protocol. Jev returned valid structured responses for every trial and made no false-completion declaration, but recognized only 1/8 physically complete cases with privileged native telemetry and 0/8 with images alone. All four occluded complete scenes were called incomplete in both conditions. RGB abstention matched 8/16 ambiguous references. Median online latency was 1.776 seconds, p95 3.287 seconds. The release preserves every result, the independent review and the original unrun protocol. This small synthetic diagnostic does not qualify a robot completion gate, actuator loop or learned skill.Image-goal verification: native benchmark and Jev protocolhttps://research.dvidia.org/papers/image-goal-jev-v01/https://research.dvidia.org/papers/image-goal-jev-v01/Sun, 18 Oct 2026 12:00:00 GMT32 native fixtures · 4 identical-image pairs · 64 Jev trials not run. A frozen PiPER image-goal cohort crosses eight physical conditions with four layouts. Four moving/settled pairs have byte-identical images and different completion labels, exposing when one image cannot establish task completion. Native measurements verify 32 authored fixtures; a paired 64-request Jev protocol distinguishes RGB judgments from privileged simulator telemetry. The provider evaluation has not run because credentials were unavailable: accuracy, false-completion rates and service latency remain unmeasured. Every native state, image, frozen request hash and explicit unrun result is retained. No learned policy or hardware qualification is claimed.A visual goal still needs physical feedbackhttps://research.dvidia.org/papers/goal-images-physical-feedback/https://research.dvidia.org/papers/goal-images-physical-feedback/Sun, 18 Oct 2026 12:00:00 GMT14/14 exact passive replays · 249,502 native ticks · visual models unbenchmarked. A goal image specifies a destination; physical feedback verifies whether a robot gets there. We describe a bounded decision role for TypeSafe Jev, distinguish it from action-conditioned world-model planning, and propose an independently scored reaching experiment. A separate passive audit reproduces fourteen previously inspected rigid-box arm episodes exactly and records 249,502 native ticks. Millisecond contact gaps hidden between stored observations become visible, while task outcomes remain unchanged. Complete inputs, traces, source snapshots and a verified downloader preserve every case. This is measurement work, not a new policy benchmark, PiPER shoe result or hardware qualification.Give the gripper a fair testhttps://research.dvidia.org/papers/shoe-graspability-v03/https://research.dvidia.org/papers/shoe-graspability-v03/Wed, 14 Oct 2026 12:00:00 GMT24/24 valid · 30 g: 8/12 lifts · 300 g: 0/12 · no net gain from removing rake. A frozen 24-attempt PiPER diagnostic separates three rigid shapes, 30 g and 300 g loads, and mounted versus absent rake conditions. All attempts are valid; eight of twelve lightweight controls retain a direct lift while every 300 g attempt fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result, with no net completion gain. A verified settled reference, dense contact traces, unchanged drive fingerprint and independent scoring expose limits in the modeled grasp and force response. Recorded replays and runnable CPU code retain every failure. No autonomous tool removal, retrieval, learned skill or physical hardware qualification is claimed.From reaching a shoe to keeping holdhttps://research.dvidia.org/papers/continuous-shoe-grip-v02/https://research.dvidia.org/papers/continuous-shoe-grip-v02/Tue, 13 Oct 2026 12:00:00 GMT20/20 valid · 2/10 completed under both policies · 9 contact-loss alarms verified. A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20–52 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification.Skill prerequisites: access, reach and recoveryhttps://research.dvidia.org/papers/skill-prerequisites-v01/https://research.dvidia.org/papers/skill-prerequisites-v01/Tue, 13 Oct 2026 12:00:00 GMT16/16 valid · 2 lid contacts avoided · 3/4 retrieval checkpoints · 0/4 retained tool lifts. A frozen sixteen-attempt native MuJoCo study tests hinged shoe-box access and a mounted rake for an out-of-workspace shoe. Guarded access avoids two baseline lid collisions while task completion stays at 2/4; a transient withdrawal adds 0.70 simulated seconds without being necessary in its matched baseline. The mounted rake reaches an intermediate retrieval checkpoint in 3/4 cases, but all subsequent retained lifts fail and the short-tool case is deferred. All native traces and failures remain public. The browser replays recorded states; fresh physics runs locally. Privileged observations, synthetic contact and authored policies establish no learned skill or hardware qualification.Live PiPER shoe workstation v0.1https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/Tue, 13 Oct 2026 12:00:00 GMTLive browser physics · 14/16 targets · strict contact parity failed · shoe lifting unqualified. A six-axis PiPER and its coupled gripper respond to user targets while MuJoCo runs locally in a browser worker. The final native run retains 20 valid attempts: 14/16 target trials pass, both high-target repeats fail, and jaw, impulse, disabled-table and 600-second hold checks pass. A ten-minute headless Chromium run reports 51.389 render calls per second and a 0.9999795 simulation/wall-time ratio on the tested Apple M5 host; simulated movement response is 61.9 ms p95 across 61 commands. Render calls do not measure GPU presentation. Native/WASM numerical and loaded-contact-count checks pass, while strict raw contact counts differ in 13/1,500 states. All fifteen development shoe-lift attempts fail. Complete development, failure and timing records remain separate. This authored rigid-shoe environment establishes no learned skill, calibrated material model or physical robot qualification.Reusable shoe asset library v0.1https://research.dvidia.org/papers/shoe-asset-library-v01/https://research.dvidia.org/papers/shoe-asset-library-v01/Mon, 12 Oct 2026 12:00:00 GMTSix shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified. Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.PiPER placement reliability v0.5https://research.dvidia.org/papers/piper-reliability-v05/https://research.dvidia.org/papers/piper-reliability-v05/Mon, 12 Oct 2026 12:00:00 GMT23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified. Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.PiPER contact recovery v0.4https://research.dvidia.org/papers/piper-contact-recovery-v04/https://research.dvidia.org/papers/piper-contact-recovery-v04/Sun, 11 Oct 2026 12:00:00 GMTAll modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified. A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.PiPER contact history v0.3https://research.dvidia.org/papers/piper-contact-history-v03/https://research.dvidia.org/papers/piper-contact-history-v03/Sun, 11 Oct 2026 12:00:00 GMTHistory 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified. A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.PiPER grip validation and recoveryhttps://research.dvidia.org/papers/piper-grip-validation-and-recovery/https://research.dvidia.org/papers/piper-grip-validation-and-recovery/Sun, 11 Oct 2026 12:00:00 GMTModerate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified. A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.Native grasp task benchmarkhttps://research.dvidia.org/papers/native-grasp-task-benchmark/https://research.dvidia.org/papers/native-grasp-task-benchmark/Sun, 11 Oct 2026 12:00:00 GMT6/6 simulated placements · controls 0/12 · real-time and hardware unqualified. A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.Hardware matched robot simulationhttps://research.dvidia.org/papers/hardware-matched-robot-simulation/https://research.dvidia.org/papers/hardware-matched-robot-simulation/Sun, 11 Oct 2026 12:00:00 GMTPinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured. A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.DVIDIA Observation Lab pilothttps://research.dvidia.org/papers/dvidia-observation-lab/https://research.dvidia.org/papers/dvidia-observation-lab/Sun, 11 Oct 2026 12:00:00 GMTLocal review release · motion and actuator source pilots · hardware unqualified. DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.Robotics research pipelinehttps://research.dvidia.org/papers/robotics-research-pipeline/https://research.dvidia.org/papers/robotics-research-pipeline/Fri, 09 Oct 2026 12:00:00 GMTArchitecture proposal · current simulation alpha · hardware unqualified. An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.Skillspace footage training pipelinehttps://research.dvidia.org/papers/skillspace-footage-training/https://research.dvidia.org/papers/skillspace-footage-training/Thu, 08 Oct 2026 12:00:00 GMTPublic CPU package · one native scene qualified · simulation-only. The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.Skill capsule acquisition and optimizationhttps://research.dvidia.org/papers/skill-capsule-acquisition/https://research.dvidia.org/papers/skill-capsule-acquisition/Wed, 07 Oct 2026 12:00:00 GMTSimulation-only · teacher/student 7/8 · no overall speedup. A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.Skillspace arm benchmarks and evolutionhttps://research.dvidia.org/papers/skillspace-arm-benchmarks/https://research.dvidia.org/papers/skillspace-arm-benchmarks/Wed, 07 Oct 2026 12:00:00 GMTMixed held-out result · 30/32 baseline, 29/32 updated. Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.Installing a Skillspace on a simulated armhttps://research.dvidia.org/papers/skillspace-arm-installation/https://research.dvidia.org/papers/skillspace-arm-installation/Wed, 07 Oct 2026 12:00:00 GMTPublic skill link to authored adapter · 6/6 scene placements. The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.Offline cable training environmenthttps://research.dvidia.org/papers/offline-cable-training-environment/https://research.dvidia.org/papers/offline-cable-training-environment/Wed, 07 Oct 2026 12:00:00 GMTCPU simulation-only · Controller search and frozen evaluation executed. The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.Fast accurate robot training simulationhttps://research.dvidia.org/papers/fast-accurate-robot-training-simulation/https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/Wed, 07 Oct 2026 12:00:00 GMT54 CPU fixture trials · GPU and physical transfer untested. A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.Grok robotics research on Xhttps://research.dvidia.org/papers/grok-robotics-research-on-x/https://research.dvidia.org/papers/grok-robotics-research-on-x/Wed, 07 Oct 2026 12:00:00 GMTPrimary sources checked · Experiments proposed. Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.Spatially grounded downloadable robot skillshttps://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/Wed, 07 Oct 2026 12:00:00 GMTSource review · DVIDIA experiments not run. A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.Clef and grounded video observationshttps://research.dvidia.org/papers/clef-and-grounded-video-observations/https://research.dvidia.org/papers/clef-and-grounded-video-observations/Tue, 06 Oct 2026 12:00:00 GMTFive licensed excerpts · Accuracy evaluation outstanding. A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.From Skillspace to robot skillhttps://research.dvidia.org/papers/from-skillspace-to-robot-skill/https://research.dvidia.org/papers/from-skillspace-to-robot-skill/Tue, 06 Oct 2026 12:00:00 GMTGraduation and robot training not run. An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.Turn physical footage into testable evidencehttps://research.dvidia.org/papers/physical-skill-research-agenda/https://research.dvidia.org/papers/physical-skill-research-agenda/Tue, 06 Oct 2026 12:00:00 GMTFour baseline protocols proposed. A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct. +DVIDIA Researchhttps://research.dvidia.org/Research footprints on demonstrations, calibrated simulation and robot skills.enIndependent camera and actuator corroboration v0.4https://research.dvidia.org/papers/sensor-corroboration-v04/https://research.dvidia.org/papers/sensor-corroboration-v04/Mon, 19 Oct 2026 12:00:00 GMT272 cases · 69/112 false completions prevented · 43 remain · 73/80 true completions abstained. A fresh eight-layout PiPER simulation benchmark adds separately processed RGB and drive checks to a local physical-goal verifier. The checks prevent 69 of 112 false completion readings while leaving 43 replay, identity, orientation, support and sampling blind spots; the conservative gate abstains on 73 of 80 baseline true completions, including all 64 deliberate sensor-fault controls. The measured CPU path is 22.1 ms median and 31.5 ms p95, excluding 200 ms acquisition. All raw channels and results are preserved; this is synthetic evidence, not hardware validation.A local verifier for physical completionhttps://research.dvidia.org/papers/local-goal-verifier-v03/https://research.dvidia.org/papers/local-goal-verifier-v03/Mon, 19 Oct 2026 12:00:00 GMTNative: 16/16 true complete, 0/48 false complete · forged readings: 8/8 false complete. An offline audit traces the seven missed Jev completions without claiming an internal model cause. A local deterministic verifier then checks timestamped pose, motion, support, release and uncertainty on 200 fresh cases. It recognizes all 16 native positives and rejects all 48 native negatives; 128 evidence faults yield 120 abstentions and 8 incomplete answers. All 8 consistent forged streams produce physical false completions, exposing the remaining sensor-trust problem. CPU-only p95 is 0.318 ms, excluding the required 200 ms history and sensing. Complete native inputs, every decision, independent review and explicit failure cohorts remain public; no new Jev calls or hardware qualification.Jev image-goal verification: the measured follow-uphttps://research.dvidia.org/papers/image-goal-jev-v02/https://research.dvidia.org/papers/image-goal-jev-v02/Mon, 19 Oct 2026 12:00:00 GMT64/64 valid responses · telemetry: 0/24 false complete, 1/8 true complete. The first measured run preserves all 64 requests from the image-goal v0.1 protocol. Jev returned valid structured responses for every trial and made no false-completion declaration, but recognized only 1/8 physically complete cases with privileged native telemetry and 0/8 with images alone. All four occluded complete scenes were called incomplete in both conditions. RGB abstention matched 8/16 ambiguous references. Median online latency was 1.776 seconds, p95 3.287 seconds. The release preserves every result, the independent review and the original unrun protocol. This small synthetic diagnostic does not qualify a robot completion gate, actuator loop or learned skill.Image-goal verification: native benchmark and Jev protocolhttps://research.dvidia.org/papers/image-goal-jev-v01/https://research.dvidia.org/papers/image-goal-jev-v01/Sun, 18 Oct 2026 12:00:00 GMT32 native fixtures · 4 identical-image pairs · 64 Jev trials not run. A frozen PiPER image-goal cohort crosses eight physical conditions with four layouts. Four moving/settled pairs have byte-identical images and different completion labels, exposing when one image cannot establish task completion. Native measurements verify 32 authored fixtures; a paired 64-request Jev protocol distinguishes RGB judgments from privileged simulator telemetry. The provider evaluation has not run because credentials were unavailable: accuracy, false-completion rates and service latency remain unmeasured. Every native state, image, frozen request hash and explicit unrun result is retained. No learned policy or hardware qualification is claimed.A visual goal still needs physical feedbackhttps://research.dvidia.org/papers/goal-images-physical-feedback/https://research.dvidia.org/papers/goal-images-physical-feedback/Sun, 18 Oct 2026 12:00:00 GMT14/14 exact passive replays · 249,502 native ticks · visual models unbenchmarked. A goal image specifies a destination; physical feedback verifies whether a robot gets there. We describe a bounded decision role for TypeSafe Jev, distinguish it from action-conditioned world-model planning, and propose an independently scored reaching experiment. A separate passive audit reproduces fourteen previously inspected rigid-box arm episodes exactly and records 249,502 native ticks. Millisecond contact gaps hidden between stored observations become visible, while task outcomes remain unchanged. Complete inputs, traces, source snapshots and a verified downloader preserve every case. This is measurement work, not a new policy benchmark, PiPER shoe result or hardware qualification.Give the gripper a fair testhttps://research.dvidia.org/papers/shoe-graspability-v03/https://research.dvidia.org/papers/shoe-graspability-v03/Wed, 14 Oct 2026 12:00:00 GMT24/24 valid · 30 g: 8/12 lifts · 300 g: 0/12 · no net gain from removing rake. A frozen 24-attempt PiPER diagnostic separates three rigid shapes, 30 g and 300 g loads, and mounted versus absent rake conditions. All attempts are valid; eight of twelve lightweight controls retain a direct lift while every 300 g attempt fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result, with no net completion gain. A verified settled reference, dense contact traces, unchanged drive fingerprint and independent scoring expose limits in the modeled grasp and force response. Recorded replays and runnable CPU code retain every failure. No autonomous tool removal, retrieval, learned skill or physical hardware qualification is claimed.From reaching a shoe to keeping holdhttps://research.dvidia.org/papers/continuous-shoe-grip-v02/https://research.dvidia.org/papers/continuous-shoe-grip-v02/Tue, 13 Oct 2026 12:00:00 GMT20/20 valid · 2/10 completed under both policies · 9 contact-loss alarms verified. A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20–52 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification.Skill prerequisites: access, reach and recoveryhttps://research.dvidia.org/papers/skill-prerequisites-v01/https://research.dvidia.org/papers/skill-prerequisites-v01/Tue, 13 Oct 2026 12:00:00 GMT16/16 valid · 2 lid contacts avoided · 3/4 retrieval checkpoints · 0/4 retained tool lifts. A frozen sixteen-attempt native MuJoCo study tests hinged shoe-box access and a mounted rake for an out-of-workspace shoe. Guarded access avoids two baseline lid collisions while task completion stays at 2/4; a transient withdrawal adds 0.70 simulated seconds without being necessary in its matched baseline. The mounted rake reaches an intermediate retrieval checkpoint in 3/4 cases, but all subsequent retained lifts fail and the short-tool case is deferred. All native traces and failures remain public. The browser replays recorded states; fresh physics runs locally. Privileged observations, synthetic contact and authored policies establish no learned skill or hardware qualification.Live PiPER shoe workstation v0.1https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/Tue, 13 Oct 2026 12:00:00 GMTLive browser physics · 14/16 targets · strict contact parity failed · shoe lifting unqualified. A six-axis PiPER and its coupled gripper respond to user targets while MuJoCo runs locally in a browser worker. The final native run retains 20 valid attempts: 14/16 target trials pass, both high-target repeats fail, and jaw, impulse, disabled-table and 600-second hold checks pass. A ten-minute headless Chromium run reports 51.389 render calls per second and a 0.9999795 simulation/wall-time ratio on the tested Apple M5 host; simulated movement response is 61.9 ms p95 across 61 commands. Render calls do not measure GPU presentation. Native/WASM numerical and loaded-contact-count checks pass, while strict raw contact counts differ in 13/1,500 states. All fifteen development shoe-lift attempts fail. Complete development, failure and timing records remain separate. This authored rigid-shoe environment establishes no learned skill, calibrated material model or physical robot qualification.Reusable shoe asset library v0.1https://research.dvidia.org/papers/shoe-asset-library-v01/https://research.dvidia.org/papers/shoe-asset-library-v01/Mon, 12 Oct 2026 12:00:00 GMTSix shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified. Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.PiPER placement reliability v0.5https://research.dvidia.org/papers/piper-reliability-v05/https://research.dvidia.org/papers/piper-reliability-v05/Mon, 12 Oct 2026 12:00:00 GMT23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified. Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.PiPER contact recovery v0.4https://research.dvidia.org/papers/piper-contact-recovery-v04/https://research.dvidia.org/papers/piper-contact-recovery-v04/Sun, 11 Oct 2026 12:00:00 GMTAll modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified. A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.PiPER contact history v0.3https://research.dvidia.org/papers/piper-contact-history-v03/https://research.dvidia.org/papers/piper-contact-history-v03/Sun, 11 Oct 2026 12:00:00 GMTHistory 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified. A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.PiPER grip validation and recoveryhttps://research.dvidia.org/papers/piper-grip-validation-and-recovery/https://research.dvidia.org/papers/piper-grip-validation-and-recovery/Sun, 11 Oct 2026 12:00:00 GMTModerate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified. A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.Native grasp task benchmarkhttps://research.dvidia.org/papers/native-grasp-task-benchmark/https://research.dvidia.org/papers/native-grasp-task-benchmark/Sun, 11 Oct 2026 12:00:00 GMT6/6 simulated placements · controls 0/12 · real-time and hardware unqualified. A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.Hardware matched robot simulationhttps://research.dvidia.org/papers/hardware-matched-robot-simulation/https://research.dvidia.org/papers/hardware-matched-robot-simulation/Sun, 11 Oct 2026 12:00:00 GMTPinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured. A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.DVIDIA Observation Lab pilothttps://research.dvidia.org/papers/dvidia-observation-lab/https://research.dvidia.org/papers/dvidia-observation-lab/Sun, 11 Oct 2026 12:00:00 GMTLocal review release · motion and actuator source pilots · hardware unqualified. DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.Robotics research pipelinehttps://research.dvidia.org/papers/robotics-research-pipeline/https://research.dvidia.org/papers/robotics-research-pipeline/Fri, 09 Oct 2026 12:00:00 GMTArchitecture proposal · current simulation alpha · hardware unqualified. An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.Skillspace footage training pipelinehttps://research.dvidia.org/papers/skillspace-footage-training/https://research.dvidia.org/papers/skillspace-footage-training/Thu, 08 Oct 2026 12:00:00 GMTPublic CPU package · one native scene qualified · simulation-only. The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.Skill capsule acquisition and optimizationhttps://research.dvidia.org/papers/skill-capsule-acquisition/https://research.dvidia.org/papers/skill-capsule-acquisition/Wed, 07 Oct 2026 12:00:00 GMTSimulation-only · teacher/student 7/8 · no overall speedup. A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.Skillspace arm benchmarks and evolutionhttps://research.dvidia.org/papers/skillspace-arm-benchmarks/https://research.dvidia.org/papers/skillspace-arm-benchmarks/Wed, 07 Oct 2026 12:00:00 GMTMixed held-out result · 30/32 baseline, 29/32 updated. Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.Installing a Skillspace on a simulated armhttps://research.dvidia.org/papers/skillspace-arm-installation/https://research.dvidia.org/papers/skillspace-arm-installation/Wed, 07 Oct 2026 12:00:00 GMTPublic skill link to authored adapter · 6/6 scene placements. The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.Offline cable training environmenthttps://research.dvidia.org/papers/offline-cable-training-environment/https://research.dvidia.org/papers/offline-cable-training-environment/Wed, 07 Oct 2026 12:00:00 GMTCPU simulation-only · Controller search and frozen evaluation executed. The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.Fast accurate robot training simulationhttps://research.dvidia.org/papers/fast-accurate-robot-training-simulation/https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/Wed, 07 Oct 2026 12:00:00 GMT54 CPU fixture trials · GPU and physical transfer untested. A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.Grok robotics research on Xhttps://research.dvidia.org/papers/grok-robotics-research-on-x/https://research.dvidia.org/papers/grok-robotics-research-on-x/Wed, 07 Oct 2026 12:00:00 GMTPrimary sources checked · Experiments proposed. Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.Spatially grounded downloadable robot skillshttps://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/Wed, 07 Oct 2026 12:00:00 GMTSource review · DVIDIA experiments not run. A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.Clef and grounded video observationshttps://research.dvidia.org/papers/clef-and-grounded-video-observations/https://research.dvidia.org/papers/clef-and-grounded-video-observations/Tue, 06 Oct 2026 12:00:00 GMTFive licensed excerpts · Accuracy evaluation outstanding. A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.From Skillspace to robot skillhttps://research.dvidia.org/papers/from-skillspace-to-robot-skill/https://research.dvidia.org/papers/from-skillspace-to-robot-skill/Tue, 06 Oct 2026 12:00:00 GMTGraduation and robot training not run. An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.Turn physical footage into testable evidencehttps://research.dvidia.org/papers/physical-skill-research-agenda/https://research.dvidia.org/papers/physical-skill-research-agenda/Tue, 06 Oct 2026 12:00:00 GMTFour baseline protocols proposed. A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct. diff --git a/docs/index.html b/docs/index.html index bdee64d..1730930 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,4 +1,4 @@ Research journal · DVIDIA Research -

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

28 papers and working notes

· Local CPU verifier with native, evidence-fault and forgery diagnostics

A local verifier for physical completion

An offline audit traces the seven missed Jev completions without claiming an internal model cause. A local deterministic verifier then checks timestamped pose, motion, support, release and uncertainty on 200 fresh cases. It recognizes all 16 native positives and rejects all 48 native negatives; 128 evidence faults yield 120 abstentions and 8 incomplete answers. All 8 consistent forged streams produce physical false completions, exposing the remaining sensor-trust problem. CPU-only p95 is 0.318 ms, excluding the required 200 ms history and sensing. Complete native inputs, every decision, independent review and explicit failure cohorts remain public; no new Jev calls or hardware qualification.

Native: 16/16 true complete, 0/48 false complete · forged readings: 8/8 false complete

· Frozen hosted-model evaluation with native physical references

Jev image-goal verification: the measured follow-up

The first measured run preserves all 64 requests from the image-goal v0.1 protocol. Jev returned valid structured responses for every trial and made no false-completion declaration, but recognized only 1/8 physically complete cases with privileged native telemetry and 0/8 with images alone. All four occluded complete scenes were called incomplete in both conditions. RGB abstention matched 8/16 ambiguous references. Median online latency was 1.776 seconds, p95 3.287 seconds. The release preserves every result, the independent review and the original unrun protocol. This small synthetic diagnostic does not qualify a robot completion gate, actuator loop or learned skill.

64/64 valid responses · telemetry: 0/24 false complete, 1/8 true complete

· Native diagnostic benchmark and frozen hosted-model protocol

Image-goal verification: native benchmark and Jev protocol

A frozen PiPER image-goal cohort crosses eight physical conditions with four layouts. Four moving/settled pairs have byte-identical images and different completion labels, exposing when one image cannot establish task completion. Native measurements verify 32 authored fixtures; a paired 64-request Jev protocol distinguishes RGB judgments from privileged simulator telemetry. The provider evaluation has not run because credentials were unavailable: accuracy, false-completion rates and service latency remain unmeasured. Every native state, image, frozen request hash and explicit unrun result is retained. No learned policy or hardware qualification is claimed.

32 native fixtures · 4 identical-image pairs · 64 Jev trials not run

· Research direction and passive native contact audit

A visual goal still needs physical feedback

A goal image specifies a destination; physical feedback verifies whether a robot gets there. We describe a bounded decision role for TypeSafe Jev, distinguish it from action-conditioned world-model planning, and propose an independently scored reaching experiment. A separate passive audit reproduces fourteen previously inspected rigid-box arm episodes exactly and records 249,502 native ticks. Millisecond contact gaps hidden between stored observations become visible, while task outcomes remain unchanged. Complete inputs, traces, source snapshots and a verified downloader preserve every case. This is measurement work, not a new policy benchmark, PiPER shoe result or hardware qualification.

14/14 exact passive replays · 249,502 native ticks · visual models unbenchmarked

· Frozen shape, load and attached-tool diagnostic

Give the gripper a fair test

A frozen 24-attempt PiPER diagnostic separates three rigid shapes, 30 g and 300 g loads, and mounted versus absent rake conditions. All attempts are valid; eight of twelve lightweight controls retain a direct lift while every 300 g attempt fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result, with no net completion gain. A verified settled reference, dense contact traces, unchanged drive fingerprint and independent scoring expose limits in the modeled grasp and force response. Recorded replays and runnable CPU code retain every failure. No autonomous tool removal, retrieval, learned skill or physical hardware qualification is claimed.

24/24 valid · 30 g: 8/12 lifts · 300 g: 0/12 · no net gain from removing rake

· Frozen native comparison with independent contact audit

From reaching a shoe to keeping hold

A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20–52 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification.

20/20 valid · 2/10 completed under both policies · 9 contact-loss alarms verified

· Two native environments with authored policies and recorded benchmark evidence

Skill prerequisites: access, reach and recovery

A frozen sixteen-attempt native MuJoCo study tests hinged shoe-box access and a mounted rake for an out-of-workspace shoe. Guarded access avoids two baseline lid collisions while task completion stays at 2/4; a transient withdrawal adds 0.70 simulated seconds without being necessary in its matched baseline. The mounted rake reaches an intermediate retrieval checkpoint in 3/4 cases, but all subsequent retained lifts fail and the short-tool case is deferred. All native traces and failures remain public. The browser replays recorded states; fresh physics runs locally. Privileged observations, synthetic contact and authored policies establish no learned skill or hardware qualification.

16/16 valid · 2 lid contacts avoided · 3/4 retrieval checkpoints · 0/4 retained tool lifts

· Live browser-native simulation with authored control and separate benchmark evidence

Live PiPER shoe workstation v0.1

A six-axis PiPER and its coupled gripper respond to user targets while MuJoCo runs locally in a browser worker. The final native run retains 20 valid attempts: 14/16 target trials pass, both high-target repeats fail, and jaw, impulse, disabled-table and 600-second hold checks pass. A ten-minute headless Chromium run reports 51.389 render calls per second and a 0.9999795 simulation/wall-time ratio on the tested Apple M5 host; simulated movement response is 61.9 ms p95 across 61 commands. Render calls do not measure GPU presentation. Native/WASM numerical and loaded-contact-count checks pass, while strict raw contact counts differ in 13/1,500 states. All fifteen development shoe-lift attempts fail. Complete development, failure and timing records remain separate. This authored rigid-shoe environment establishes no learned skill, calibrated material model or physical robot qualification.

Live browser physics · 14/16 targets · strict contact parity failed · shoe lifting unqualified

· Versioned native scene assets and frozen passive-contact benchmark

Reusable shoe asset library v0.1

Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.

Six shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified

· Frozen paired reliability experiment with complete native evidence

PiPER placement reliability v0.5

Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.

23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified

· Paired carry-recovery and supported-acquisition experiment with complete confirmation receipt

PiPER contact recovery v0.4

A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.

All modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified

· Paired probe-only and contact-history experiment with complete confirmation receipt

PiPER contact history v0.3

A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.

History 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified

· Paired authored controller experiment and complete confirmation receipt

PiPER grip validation and recovery

A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.

Moderate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified

· Native contact task implementation and authored baseline

Native grasp task benchmark

A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.

6/6 simulated placements · controls 0/12 · real-time and hardware unqualified

· Pinned mechanics implementation and CPU numerical benchmark

Hardware matched robot simulation

A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.

Pinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured

· Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.

Local review release · motion and actuator source pilots · hardware unqualified

· Architecture proposal and staged validation roadmap

Robotics research pipeline

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

Architecture proposal · current simulation alpha · hardware unqualified

· Product implementation and measured synthetic pilot

Skillspace footage training pipeline

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

Public CPU package · one native scene qualified · simulation-only

· Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

Simulation-only · teacher/student 7/8 · no overall speedup

· Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

Mixed held-out result · 30/32 baseline, 29/32 updated

· Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

Public skill link to authored adapter · 6/6 scene placements

· Product prototype and local qualification

Offline cable training environment

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

CPU simulation-only · Controller search and frozen evaluation executed

· Decision report and local experiment

Fast accurate robot training simulation

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

54 CPU fixture trials · GPU and physical transfer untested

· Source review and thesis comparison

Grok robotics research on X

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

Primary sources checked · Experiments proposed

· Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

Source review · DVIDIA experiments not run

· Engineering feasibility pilot

Clef and grounded video observations

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

Five licensed excerpts · Accuracy evaluation outstanding

· Source review and implementation proposal

From Skillspace to robot skill

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

Graduation and robot training not run

· Research agenda

Turn physical footage into testable evidence

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

Four baseline protocols proposed

Research footprints

  1. Measure Jev against physical completion

    All 64 frozen requests returned valid responses. No false completion was observed, but images plus telemetry recognized only 1/8 completed states, and images alone recognized 0/8. Occluded finished scenes failed in both conditions. Median hosted latency was 1.776 seconds. The original protocol remains unchanged; all measured cases and independent review are public.

    Read the measured follow-up · Inspect every judgment
  2. Make image-goal uncertainty testable

    Thirty-two native PiPER fixtures include four moving/settled pairs with identical images and different physical completion labels. The frozen 64-request Jev protocol compares RGB with privileged simulator telemetry. Jev has not been run; all provider trials remain explicitly unrun, with accuracy and service latency unmeasured. Native states, images, source hashes and physical labels are public.

    Read the benchmark and protocol · Compare every current and goal image
  3. Connect visual goals to measured physical feedback

    Fourteen passive rigid-box arm replays reproduce every original sampled state and outcome, recording 249,502 native ticks. Millisecond contact gaps appear between stored observations; a successful comparator also has gaps. The separate architecture note evaluates Jev as a candidate decision layer and defines a goal-image reaching test. No visual model is integrated and no controller is promoted.

    Read the architecture and audit · Download verified evidence tool
  4. Separate object difficulty from the gripper and attached tool

    A frozen direct-grasp diagnostic retains 24 valid attempts. Lightweight controls complete 8/12 lifts; every 300 g condition fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result. Relative lift, settled references and dense contact traces make the modeled drive and geometry limits visible. These are diagnostic controls, not a shoe retrieval rate or hardware qualification.

    Read the graspability study · Inspect every recorded attempt
  5. Measure contact-loss response without claiming a better grasp

    Twenty valid fresh attempts complete the same 2/10 cases under both controllers. Nine trace-verified bilateral-contact-loss alarms arrive in 20–52 ms; twelve supported lowering responses produce no additional completion. Full trajectories and an independent audit expose failed acquisition and an unmonitored support-contamination gap. Hardware transfer remains open.

    Read the continuous-grip comparison · Inspect the recorded attempts
  6. Make skill prerequisites and failed recovery measurable

    Two native environments retain sixteen valid held-out attempts. A lid guard avoids two baseline contacts while completion remains 2/4. A mounted rake reaches three of four intermediate retrieval checkpoints, but no subsequent retained lift succeeds. Recorded replays expose every failure; fresh physics runs locally. Tool acquisition, learned control and hardware transfer remain open.

    Read the paired results · Explore both recorded experiments
  7. Move the PiPER arm in a live browser workstation

    Drag a target and nudge a free rigid trainer with live browser MuJoCo physics. The final native run keeps 20 valid attempts: 14/16 targets pass, both high-target repeats fail, and four mechanics checks pass. A ten-minute browser run reports 51.39 render calls per second on the tested host. Strict raw-contact parity fails in 13/1,500 sampled states despite numerical agreement; all fifteen development shoe-lift attempts fail. Learned shoe skills, calibrated contact and hardware transfer remain open.

    Read the workstation and evidence · Open the live workstation
  8. Load reusable shoes with three native robot arms

    Six original rigid shoe proxies now compose with UR5e/Robotiq, PiPER and SO-101. Versioned scene files, offline 3D previews and a frozen 54-attempt benchmark retain every trajectory: supported pairs 27/27, disabled-table support 0/27, all gravity checks pass. Native loading and passive contact are tested; shoe grasping, deformation and physical transfer remain unqualified.

    Read the asset library results · Browse shoes and arms
  9. Complete more placements with a bounded second lift and revised grasp

    Three fresh cohorts retain 252 valid attempts. The revised grasp completes 23/36 placements versus 15/36 for v0.4 and 18/36 with second lift alone; all 144 controls fail. Eight completions are gained and none lost, while shared successes average 0.772 simulated seconds slower. Twelve low-friction acquisitions and one retention failure remain. All 1,035 computation overruns are retained; physical qualification is open.

    Read the v0.5 reliability comparison · Inspect all three cohorts
  10. Separate faster recovery from an ineffective retry

    A fresh 84-attempt comparison completes 4/12 placements in every mode; all 48 controls fail. Two resumed tasks finish 2.440–2.640 simulated seconds sooner after unchanged settling holds, while four failed grips are refused 2.88–2.90 seconds sooner. Eight failures remain, including two held blocks below the resume-height gate. Every native trajectory and raw computation duration is retained; physical qualification is open.

    Read the v0.4 recovery comparison · Inspect every paired attempt
  11. Compare probe-only motion with contact history

    A frozen 84-attempt paired cohort completes 6/12 history placements versus 4/12 original and probe-only placements; all 48 controls fail. Four tasks complete after five closed holds/resumes, but all six supported acquisition retries fail. Only four of five resume windows meet the independent low absolute tool-speed check. Actual motion and 75 missed computation windows remain visible; physical robot qualification is open.

    Read the contact history comparison · Inspect all 84 attempts
  12. Test the grip and publish a completion regression

    A slow probe and one supported retry retain 30 valid attempts. At moderate friction baseline completes 2/2 while guarded completes 0/2; normal cases both pass and low-friction cases both fail. No successful recovery or overall improvement is demonstrated. Four brief following windows do not establish detector accuracy; 53 computation windows miss 20 ms. Every full attempt remains public.

    Read the paired results and actual stop motion · Inspect all 30 attempts
  13. Measure PiPER contact and a friction failure

    A frozen authored rigid-block pilot retains nine valid attempts: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3. One of 9,000 computation windows exceeds 20 ms. Complete task receipts remain distinct from unloaded profiles; learned skills, shoe physics and physical qualification are open.

    Read the PiPER contact pilot · Open the Robot Simulation Lab
  14. Run more arms on a local CPU

    PiPER and SO-101 join the library with pinned models, explicit arm/gripper units, preserved licenses and complete numerical receipts. Each completes six unloaded motion checks offline. Their learned-policy and physical qualification remain open. The PiPER contact pilot and UR5e grasp demonstration have separate task receipts.

    Read the profile measurements · Open the Robot Simulation Lab
  15. Give the reference arm a contact task

    A free rigid block must be grasped, lifted, carried, released and left settled at its destination. Frozen scenarios compare the authored controller with identical arm commands and open jaws, and with a hold control. Full traces and 3D replay preserve the evidence for the next compact action-learning experiment. Shoe physics, a learned task policy and hardware transfer remain open.

    Read the native task benchmark · Replay grasp and placement
  16. Build a hardware reference we can measure

    A pinned six-axis arm and coupled gripper expose fourteen coordinates and seven commands. All 18 unloaded numerical runs are valid; 1 ms CPU physics measures 40.80–47.29× elapsed-time throughput with sensing, rendering and policy excluded. A paired trace scorer makes physical mismatch measurable. Five-finger references and LeRobot collection have separate contracts; hardware fidelity and shoe competence remain unmeasured.

    Read the mechanical contract and upcoming arms · Open the 3D replay
  17. Learn a compact command-to-motion predictor

    A 6,438-byte CPU model predicts the next 20 ms of joint/jaw movement from native state and issued commands. Twelve seeded groups retain 36 recordings; 4,553 test transitions are not independent trials. Warm computation measures 0.207 ms p95, but jaw mean absolute/worst errors regress against persistence and the first opening is overpredicted. Shoe collection, hardware sensing and validated shoe physics remain open.

    Read the measured pilot and shoe collection plan
  18. Observe image motion and forecast its path

    A manually seeded CPU source alpha records pixel speed and causal forecasts. Six authored cases process 720 frames in 1.454 seconds; occlusion and low-texture losses remain visible, and a 500 ms reversal forecast misses by nearly 90 pixels. The v0.1 release/demo remain separate; real-scene accuracy, metric speed and live robot control are unqualified.

    Read the source alpha and measured limitations
  19. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  20. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  21. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  22. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  23. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  24. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  25. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

  • Capture quality — Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.
  • Skill relations — Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.
  • Evaluation and provenance — Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.
  • Physical grounding — Measure whether state/contact evidence and controlled synthetic augmentation improve a defined physical-skill task.

Follow new publications through RSS or inspect the source and revisions on GitHub.

+

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

29 papers and working notes

· Paired synthetic benchmark for camera and drive evidence

Independent camera and actuator corroboration v0.4

A fresh eight-layout PiPER simulation benchmark adds separately processed RGB and drive checks to a local physical-goal verifier. The checks prevent 69 of 112 false completion readings while leaving 43 replay, identity, orientation, support and sampling blind spots; the conservative gate abstains on 73 of 80 baseline true completions, including all 64 deliberate sensor-fault controls. The measured CPU path is 22.1 ms median and 31.5 ms p95, excluding 200 ms acquisition. All raw channels and results are preserved; this is synthetic evidence, not hardware validation.

272 cases · 69/112 false completions prevented · 43 remain · 73/80 true completions abstained

· Local CPU verifier with native, evidence-fault and forgery diagnostics

A local verifier for physical completion

An offline audit traces the seven missed Jev completions without claiming an internal model cause. A local deterministic verifier then checks timestamped pose, motion, support, release and uncertainty on 200 fresh cases. It recognizes all 16 native positives and rejects all 48 native negatives; 128 evidence faults yield 120 abstentions and 8 incomplete answers. All 8 consistent forged streams produce physical false completions, exposing the remaining sensor-trust problem. CPU-only p95 is 0.318 ms, excluding the required 200 ms history and sensing. Complete native inputs, every decision, independent review and explicit failure cohorts remain public; no new Jev calls or hardware qualification.

Native: 16/16 true complete, 0/48 false complete · forged readings: 8/8 false complete

· Frozen hosted-model evaluation with native physical references

Jev image-goal verification: the measured follow-up

The first measured run preserves all 64 requests from the image-goal v0.1 protocol. Jev returned valid structured responses for every trial and made no false-completion declaration, but recognized only 1/8 physically complete cases with privileged native telemetry and 0/8 with images alone. All four occluded complete scenes were called incomplete in both conditions. RGB abstention matched 8/16 ambiguous references. Median online latency was 1.776 seconds, p95 3.287 seconds. The release preserves every result, the independent review and the original unrun protocol. This small synthetic diagnostic does not qualify a robot completion gate, actuator loop or learned skill.

64/64 valid responses · telemetry: 0/24 false complete, 1/8 true complete

· Native diagnostic benchmark and frozen hosted-model protocol

Image-goal verification: native benchmark and Jev protocol

A frozen PiPER image-goal cohort crosses eight physical conditions with four layouts. Four moving/settled pairs have byte-identical images and different completion labels, exposing when one image cannot establish task completion. Native measurements verify 32 authored fixtures; a paired 64-request Jev protocol distinguishes RGB judgments from privileged simulator telemetry. The provider evaluation has not run because credentials were unavailable: accuracy, false-completion rates and service latency remain unmeasured. Every native state, image, frozen request hash and explicit unrun result is retained. No learned policy or hardware qualification is claimed.

32 native fixtures · 4 identical-image pairs · 64 Jev trials not run

· Research direction and passive native contact audit

A visual goal still needs physical feedback

A goal image specifies a destination; physical feedback verifies whether a robot gets there. We describe a bounded decision role for TypeSafe Jev, distinguish it from action-conditioned world-model planning, and propose an independently scored reaching experiment. A separate passive audit reproduces fourteen previously inspected rigid-box arm episodes exactly and records 249,502 native ticks. Millisecond contact gaps hidden between stored observations become visible, while task outcomes remain unchanged. Complete inputs, traces, source snapshots and a verified downloader preserve every case. This is measurement work, not a new policy benchmark, PiPER shoe result or hardware qualification.

14/14 exact passive replays · 249,502 native ticks · visual models unbenchmarked

· Frozen shape, load and attached-tool diagnostic

Give the gripper a fair test

A frozen 24-attempt PiPER diagnostic separates three rigid shapes, 30 g and 300 g loads, and mounted versus absent rake conditions. All attempts are valid; eight of twelve lightweight controls retain a direct lift while every 300 g attempt fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result, with no net completion gain. A verified settled reference, dense contact traces, unchanged drive fingerprint and independent scoring expose limits in the modeled grasp and force response. Recorded replays and runnable CPU code retain every failure. No autonomous tool removal, retrieval, learned skill or physical hardware qualification is claimed.

24/24 valid · 30 g: 8/12 lifts · 300 g: 0/12 · no net gain from removing rake

· Frozen native comparison with independent contact audit

From reaching a shoe to keeping hold

A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20–52 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification.

20/20 valid · 2/10 completed under both policies · 9 contact-loss alarms verified

· Two native environments with authored policies and recorded benchmark evidence

Skill prerequisites: access, reach and recovery

A frozen sixteen-attempt native MuJoCo study tests hinged shoe-box access and a mounted rake for an out-of-workspace shoe. Guarded access avoids two baseline lid collisions while task completion stays at 2/4; a transient withdrawal adds 0.70 simulated seconds without being necessary in its matched baseline. The mounted rake reaches an intermediate retrieval checkpoint in 3/4 cases, but all subsequent retained lifts fail and the short-tool case is deferred. All native traces and failures remain public. The browser replays recorded states; fresh physics runs locally. Privileged observations, synthetic contact and authored policies establish no learned skill or hardware qualification.

16/16 valid · 2 lid contacts avoided · 3/4 retrieval checkpoints · 0/4 retained tool lifts

· Live browser-native simulation with authored control and separate benchmark evidence

Live PiPER shoe workstation v0.1

A six-axis PiPER and its coupled gripper respond to user targets while MuJoCo runs locally in a browser worker. The final native run retains 20 valid attempts: 14/16 target trials pass, both high-target repeats fail, and jaw, impulse, disabled-table and 600-second hold checks pass. A ten-minute headless Chromium run reports 51.389 render calls per second and a 0.9999795 simulation/wall-time ratio on the tested Apple M5 host; simulated movement response is 61.9 ms p95 across 61 commands. Render calls do not measure GPU presentation. Native/WASM numerical and loaded-contact-count checks pass, while strict raw contact counts differ in 13/1,500 states. All fifteen development shoe-lift attempts fail. Complete development, failure and timing records remain separate. This authored rigid-shoe environment establishes no learned skill, calibrated material model or physical robot qualification.

Live browser physics · 14/16 targets · strict contact parity failed · shoe lifting unqualified

· Versioned native scene assets and frozen passive-contact benchmark

Reusable shoe asset library v0.1

Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.

Six shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified

· Frozen paired reliability experiment with complete native evidence

PiPER placement reliability v0.5

Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.

23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified

· Paired carry-recovery and supported-acquisition experiment with complete confirmation receipt

PiPER contact recovery v0.4

A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.

All modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified

· Paired probe-only and contact-history experiment with complete confirmation receipt

PiPER contact history v0.3

A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.

History 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified

· Paired authored controller experiment and complete confirmation receipt

PiPER grip validation and recovery

A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.

Moderate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified

· Native contact task implementation and authored baseline

Native grasp task benchmark

A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.

6/6 simulated placements · controls 0/12 · real-time and hardware unqualified

· Pinned mechanics implementation and CPU numerical benchmark

Hardware matched robot simulation

A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.

Pinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured

· Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.

Local review release · motion and actuator source pilots · hardware unqualified

· Architecture proposal and staged validation roadmap

Robotics research pipeline

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

Architecture proposal · current simulation alpha · hardware unqualified

· Product implementation and measured synthetic pilot

Skillspace footage training pipeline

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

Public CPU package · one native scene qualified · simulation-only

· Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

Simulation-only · teacher/student 7/8 · no overall speedup

· Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

Mixed held-out result · 30/32 baseline, 29/32 updated

· Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

Public skill link to authored adapter · 6/6 scene placements

· Product prototype and local qualification

Offline cable training environment

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

CPU simulation-only · Controller search and frozen evaluation executed

· Decision report and local experiment

Fast accurate robot training simulation

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

54 CPU fixture trials · GPU and physical transfer untested

· Source review and thesis comparison

Grok robotics research on X

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

Primary sources checked · Experiments proposed

· Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

Source review · DVIDIA experiments not run

· Engineering feasibility pilot

Clef and grounded video observations

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

Five licensed excerpts · Accuracy evaluation outstanding

· Source review and implementation proposal

From Skillspace to robot skill

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

Graduation and robot training not run

· Research agenda

Turn physical footage into testable evidence

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

Four baseline protocols proposed

Research footprints

  1. Measure Jev against physical completion

    All 64 frozen requests returned valid responses. No false completion was observed, but images plus telemetry recognized only 1/8 completed states, and images alone recognized 0/8. Occluded finished scenes failed in both conditions. Median hosted latency was 1.776 seconds. The original protocol remains unchanged; all measured cases and independent review are public.

    Read the measured follow-up · Inspect every judgment
  2. Make image-goal uncertainty testable

    Thirty-two native PiPER fixtures include four moving/settled pairs with identical images and different physical completion labels. The frozen 64-request Jev protocol compares RGB with privileged simulator telemetry. Jev has not been run; all provider trials remain explicitly unrun, with accuracy and service latency unmeasured. Native states, images, source hashes and physical labels are public.

    Read the benchmark and protocol · Compare every current and goal image
  3. Connect visual goals to measured physical feedback

    Fourteen passive rigid-box arm replays reproduce every original sampled state and outcome, recording 249,502 native ticks. Millisecond contact gaps appear between stored observations; a successful comparator also has gaps. The separate architecture note evaluates Jev as a candidate decision layer and defines a goal-image reaching test. No visual model is integrated and no controller is promoted.

    Read the architecture and audit · Download verified evidence tool
  4. Separate object difficulty from the gripper and attached tool

    A frozen direct-grasp diagnostic retains 24 valid attempts. Lightweight controls complete 8/12 lifts; every 300 g condition fails, including the small block. Removing the rake gains one narrow-shoe result and loses one block result. Relative lift, settled references and dense contact traces make the modeled drive and geometry limits visible. These are diagnostic controls, not a shoe retrieval rate or hardware qualification.

    Read the graspability study · Inspect every recorded attempt
  5. Measure contact-loss response without claiming a better grasp

    Twenty valid fresh attempts complete the same 2/10 cases under both controllers. Nine trace-verified bilateral-contact-loss alarms arrive in 20–52 ms; twelve supported lowering responses produce no additional completion. Full trajectories and an independent audit expose failed acquisition and an unmonitored support-contamination gap. Hardware transfer remains open.

    Read the continuous-grip comparison · Inspect the recorded attempts
  6. Make skill prerequisites and failed recovery measurable

    Two native environments retain sixteen valid held-out attempts. A lid guard avoids two baseline contacts while completion remains 2/4. A mounted rake reaches three of four intermediate retrieval checkpoints, but no subsequent retained lift succeeds. Recorded replays expose every failure; fresh physics runs locally. Tool acquisition, learned control and hardware transfer remain open.

    Read the paired results · Explore both recorded experiments
  7. Move the PiPER arm in a live browser workstation

    Drag a target and nudge a free rigid trainer with live browser MuJoCo physics. The final native run keeps 20 valid attempts: 14/16 targets pass, both high-target repeats fail, and four mechanics checks pass. A ten-minute browser run reports 51.39 render calls per second on the tested host. Strict raw-contact parity fails in 13/1,500 sampled states despite numerical agreement; all fifteen development shoe-lift attempts fail. Learned shoe skills, calibrated contact and hardware transfer remain open.

    Read the workstation and evidence · Open the live workstation
  8. Load reusable shoes with three native robot arms

    Six original rigid shoe proxies now compose with UR5e/Robotiq, PiPER and SO-101. Versioned scene files, offline 3D previews and a frozen 54-attempt benchmark retain every trajectory: supported pairs 27/27, disabled-table support 0/27, all gravity checks pass. Native loading and passive contact are tested; shoe grasping, deformation and physical transfer remain unqualified.

    Read the asset library results · Browse shoes and arms
  9. Complete more placements with a bounded second lift and revised grasp

    Three fresh cohorts retain 252 valid attempts. The revised grasp completes 23/36 placements versus 15/36 for v0.4 and 18/36 with second lift alone; all 144 controls fail. Eight completions are gained and none lost, while shared successes average 0.772 simulated seconds slower. Twelve low-friction acquisitions and one retention failure remain. All 1,035 computation overruns are retained; physical qualification is open.

    Read the v0.5 reliability comparison · Inspect all three cohorts
  10. Separate faster recovery from an ineffective retry

    A fresh 84-attempt comparison completes 4/12 placements in every mode; all 48 controls fail. Two resumed tasks finish 2.440–2.640 simulated seconds sooner after unchanged settling holds, while four failed grips are refused 2.88–2.90 seconds sooner. Eight failures remain, including two held blocks below the resume-height gate. Every native trajectory and raw computation duration is retained; physical qualification is open.

    Read the v0.4 recovery comparison · Inspect every paired attempt
  11. Compare probe-only motion with contact history

    A frozen 84-attempt paired cohort completes 6/12 history placements versus 4/12 original and probe-only placements; all 48 controls fail. Four tasks complete after five closed holds/resumes, but all six supported acquisition retries fail. Only four of five resume windows meet the independent low absolute tool-speed check. Actual motion and 75 missed computation windows remain visible; physical robot qualification is open.

    Read the contact history comparison · Inspect all 84 attempts
  12. Test the grip and publish a completion regression

    A slow probe and one supported retry retain 30 valid attempts. At moderate friction baseline completes 2/2 while guarded completes 0/2; normal cases both pass and low-friction cases both fail. No successful recovery or overall improvement is demonstrated. Four brief following windows do not establish detector accuracy; 53 computation windows miss 20 ms. Every full attempt remains public.

    Read the paired results and actual stop motion · Inspect all 30 attempts
  13. Measure PiPER contact and a friction failure

    A frozen authored rigid-block pilot retains nine valid attempts: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3. One of 9,000 computation windows exceeds 20 ms. Complete task receipts remain distinct from unloaded profiles; learned skills, shoe physics and physical qualification are open.

    Read the PiPER contact pilot · Open the Robot Simulation Lab
  14. Run more arms on a local CPU

    PiPER and SO-101 join the library with pinned models, explicit arm/gripper units, preserved licenses and complete numerical receipts. Each completes six unloaded motion checks offline. Their learned-policy and physical qualification remain open. The PiPER contact pilot and UR5e grasp demonstration have separate task receipts.

    Read the profile measurements · Open the Robot Simulation Lab
  15. Give the reference arm a contact task

    A free rigid block must be grasped, lifted, carried, released and left settled at its destination. Frozen scenarios compare the authored controller with identical arm commands and open jaws, and with a hold control. Full traces and 3D replay preserve the evidence for the next compact action-learning experiment. Shoe physics, a learned task policy and hardware transfer remain open.

    Read the native task benchmark · Replay grasp and placement
  16. Build a hardware reference we can measure

    A pinned six-axis arm and coupled gripper expose fourteen coordinates and seven commands. All 18 unloaded numerical runs are valid; 1 ms CPU physics measures 40.80–47.29× elapsed-time throughput with sensing, rendering and policy excluded. A paired trace scorer makes physical mismatch measurable. Five-finger references and LeRobot collection have separate contracts; hardware fidelity and shoe competence remain unmeasured.

    Read the mechanical contract and upcoming arms · Open the 3D replay
  17. Learn a compact command-to-motion predictor

    A 6,438-byte CPU model predicts the next 20 ms of joint/jaw movement from native state and issued commands. Twelve seeded groups retain 36 recordings; 4,553 test transitions are not independent trials. Warm computation measures 0.207 ms p95, but jaw mean absolute/worst errors regress against persistence and the first opening is overpredicted. Shoe collection, hardware sensing and validated shoe physics remain open.

    Read the measured pilot and shoe collection plan
  18. Observe image motion and forecast its path

    A manually seeded CPU source alpha records pixel speed and causal forecasts. Six authored cases process 720 frames in 1.454 seconds; occlusion and low-texture losses remain visible, and a 500 ms reversal forecast misses by nearly 90 pixels. The v0.1 release/demo remain separate; real-scene accuracy, metric speed and live robot control are unqualified.

    Read the source alpha and measured limitations
  19. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  20. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  21. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  22. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  23. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  24. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  25. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

  • Capture quality — Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.
  • Skill relations — Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.
  • Evaluation and provenance — Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.
  • Physical grounding — Measure whether state/contact evidence and controlled synthetic augmentation improve a defined physical-skill task.

Follow new publications through RSS or inspect the source and revisions on GitHub.

diff --git a/docs/papers/clef-and-grounded-video-observations/index.html b/docs/papers/clef-and-grounded-video-observations/index.html index c4f7417..d85a961 100644 --- a/docs/papers/clef-and-grounded-video-observations/index.html +++ b/docs/papers/clef-and-grounded-video-observations/index.html @@ -1,7 +1,7 @@ Clef and grounded video observations · DVIDIA Research -

Engineering feasibility pilot

Clef and grounded video observations

· DVIDIA

Five licensed excerpts · Accuracy evaluation outstanding

Source & history · Download Markdown

Abstract

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

In this paper

Engineering pilot executed · 6 October 2026 · Accuracy evaluation outstanding

+

Engineering feasibility pilot

Clef and grounded video observations

· DVIDIA

Five licensed excerpts · Accuracy evaluation outstanding

Source & history · Download Markdown

Abstract

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

In this paper

Engineering pilot executed · 6 October 2026 · Accuracy evaluation outstanding

We executed a small, preregistered feasibility run: five existing public, licensed three-second POV excerpts, four original-timestamped frames per excerpt, one Gemma caption call and one Clef-flash check per excerpt. All five completed. The estimated inference cost was $0.00140634. This establishes that the image APIs and bounded data contract work; it does not establish that the annotations are accurate.

The useful finding was a failure: a caption called a pink plate a cutting board, and the second model agreed. Several descriptions listed objects or hand positions while missing visible folding or transfer. Model agreement must not become a “verified” badge. The observations below come from AI-assisted inspection of the exact input frames after outputs were frozen, not independent human annotations.

Preregistration · Complete sanitized results and source/frame hashes · Prompts and request recipes · Next evaluation protocol

diff --git a/docs/papers/dvidia-observation-lab/index.html b/docs/papers/dvidia-observation-lab/index.html index 0ea7ff7..02ef88a 100644 --- a/docs/papers/dvidia-observation-lab/index.html +++ b/docs/papers/dvidia-observation-lab/index.html @@ -1,7 +1,7 @@ DVIDIA Observation Lab pilot · DVIDIA Research -

Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

· DVIDIA

Local review release · motion and actuator source pilots · hardware unqualified

Source & history · Download Markdown

Abstract

DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.

In this paper

Engineering integration passed on six public excerpts; annotation accuracy remains unmeasured. On 10 October 2026, we ran a local observation pipeline that decodes video, proposes sampled object and palm boxes, associates nearby detections and opens an editable review timeline. Review changes persist as revisions and export with their source identity. This establishes a working research tool, not reliable automatic task labeling, a cup-placement benchmark or a trained robot skill. The published implementation is anchored to DVIDIA Training commit d355a4a.

+

Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

· DVIDIA

Local review release · motion and actuator source pilots · hardware unqualified

Source & history · Download Markdown

Abstract

DVIDIA Observation Lab supplies local object/palm proposals, reviewed timeline revisions and separate causal image-motion forecasts; real annotation and motion accuracy remain unmeasured. A new CPU pilot learns next-20-ms joint/jaw deltas from current native state and issued commands. Twelve seeded groups retain 36 recordings with 8/2/2 train/development/test grouping; 4,553 test transitions are not independent robot trials. The 6,438-byte action model computes at 0.207 ms p95 on one host and improves RMSE, while jaw MAE/worst error regress against persistence and an initial opening is substantially overpredicted. These are privileged simulator measurements and model-only timings. A proposed shoe capture plan has no footage or executed shoe benchmark. The v0.1 release and hosted synthetic review demo remain separate; no shoe policy, physical adapter, validated shoe physics or hardware qualification is established.

In this paper

Engineering integration passed on six public excerpts; annotation accuracy remains unmeasured. On 10 October 2026, we ran a local observation pipeline that decodes video, proposes sampled object and palm boxes, associates nearby detections and opens an editable review timeline. Review changes persist as revisions and export with their source identity. This establishes a working research tool, not reliable automatic task labeling, a cup-placement benchmark or a trained robot skill. The published implementation is anchored to DVIDIA Training commit d355a4a.

Install the Observation Lab release · GitHub release and downloads · Try the synthetic review demo · Frozen engineering receipts

The pilot produces reviewable 2D evidence

The local adapter uses the pinned OpenCV Zoo YOLOX COCO object detector and MediaPipe palm detector. It samples at two frames per second, up to 120 frames and a 640-pixel maximum image side per clip. Proposals retain decoded presentation timestamps, frame and source hashes, raw labels and scores. Same-class image overlap associates detections; palm/object overlap proposes 2D proximity. Neither association establishes physical identity nor overlap proves touching. Palm rectangles are not full-hand segmentation. No optical flow, depth reconstruction, natural-language action captioner or language model participates in this pilot. (Observation contract, frozen model and recipe)

diff --git a/docs/papers/fast-accurate-robot-training-simulation/index.html b/docs/papers/fast-accurate-robot-training-simulation/index.html index fd52f01..e404109 100644 --- a/docs/papers/fast-accurate-robot-training-simulation/index.html +++ b/docs/papers/fast-accurate-robot-training-simulation/index.html @@ -1,7 +1,7 @@ Fast accurate robot training simulation · DVIDIA Research -

Decision report and local experiment

Fast accurate robot training simulation

· DVIDIA

54 CPU fixture trials · GPU and physical transfer untested

Source & history · Download Markdown

Abstract

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

In this paper

Decision report · 7 October 2026 · Grok 4.7 research plus primary-source verification · 54 local CPU fixture trials; no GPU training or physical robot tests

+

Decision report and local experiment

Fast accurate robot training simulation

· DVIDIA

54 CPU fixture trials · GPU and physical transfer untested

Source & history · Download Markdown

Abstract

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

In this paper

Decision report · 7 October 2026 · Grok 4.7 research plus primary-source verification · 54 local CPU fixture trials; no GPU training or physical robot tests

Build an offline, self-hosted environment that converts demonstrations into calibrated practice and qualified robot skills. Start with existing physics engines and concentrate the product on task construction, uncertainty, failure/recovery coverage, efficient scheduling and qualification. Use native MuJoCo for local numerical checks, test MuJoCo Warp for batched rigid contact, and qualify Newton rods or another deformable backend against the actual rope task before selecting it. The microscopic intuition explains why objects resist and deform; the useful simulation interface is the resulting force, motion, friction and sensor response at robot scale. The first product vertical should be bounded rope grasping, pulling, threading and regrasping, with bow tying a later capability. One owned GPU is the starting design target; minimum useful VRAM and an “ultrafast” training rate remain unmeasured. Adding GPUs or servers should scale independent practice worlds, while a released skill retains a declared hardware profile and physical evidence.

Reproduce the response that changes the robot's decision

When a fingertip presses a table, resistance ultimately comes from electromagnetic interactions and quantum constraints on matter. “Electrons reject each other” is a useful intuition but incomplete: atoms also attract and bond, and Pauli exclusion restricts allowed electron states rather than adding a separate classical repulsive force. Molecular structure and collective interactions produce elasticity. We do not need to track electrons to calculate whether a gripper holds a lace; we need the effective mechanical behavior at the force, length and time scales the task uses. (Feynman on forces and elasticity, exclusion and matter)

diff --git a/docs/papers/from-skillspace-to-robot-skill/index.html b/docs/papers/from-skillspace-to-robot-skill/index.html index db3abfd..5ba10d5 100644 --- a/docs/papers/from-skillspace-to-robot-skill/index.html +++ b/docs/papers/from-skillspace-to-robot-skill/index.html @@ -1,7 +1,7 @@ From Skillspace to robot skill · DVIDIA Research -

Source review and implementation proposal

From Skillspace to robot skill

· DVIDIA

Graduation and robot training not run

Source & history · Download Markdown

Abstract

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

In this paper

Source review and proposal · 6 October 2026 · Graduation experiment not run

+

Source review and implementation proposal

From Skillspace to robot skill

· DVIDIA

Graduation and robot training not run

Source & history · Download Markdown

Abstract

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

In this paper

Source review and proposal · 6 October 2026 · Graduation experiment not run

Follow-up: the Clef image feasibility pilot has now run on five public excerpts. It does not complete the 40-episode accuracy protocol below.

A Skillspace can become the home for a skill's learning history. More uploaded videos do not, by themselves, establish a better policy. We need evidence that new demonstrations add useful coverage, survive quality checks, and improve held-out performance on a specified robot.

This review follows Varun Nair's pipeline post, its linked Argus article, selected upstream source files, and an audit of the DVIDIA website. The design below is our proposal, not an implemented training service. No user footage was sent to any model, no training was run, and no robot was operated for this review.

diff --git a/docs/papers/grok-robotics-research-on-x/index.html b/docs/papers/grok-robotics-research-on-x/index.html index ef2f3ff..bbb0283 100644 --- a/docs/papers/grok-robotics-research-on-x/index.html +++ b/docs/papers/grok-robotics-research-on-x/index.html @@ -1,7 +1,7 @@ Grok robotics research on X · DVIDIA Research -

Source review and thesis comparison

Grok robotics research on X

· DVIDIA

Primary sources checked · Experiments proposed

Source & history · Download Markdown

Abstract

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

In this paper

Research snapshot: 7 October 2026 · Grok 4.7 used through Pi CLI · No training, simulation or hardware experiment was run

+

Source review and thesis comparison

Grok robotics research on X

· DVIDIA

Primary sources checked · Experiments proposed

Source & history · Download Markdown

Abstract

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

In this paper

Research snapshot: 7 October 2026 · Grok 4.7 used through Pi CLI · No training, simulation or hardware experiment was run

Your initial thesis has substantial support when expressed as observation-derived goals and task state, language-based retrieval, contextual spatial attention, and simulation-assisted learning of a feedback skill. The evidence does not establish that a fixed quantity of video covers every variant, that XYZ determines what to do, or that a simulated policy downloads unchanged onto arbitrary robots. The strongest interpretation of your small-sample idea is adaptation: existing models supply reusable manipulation knowledge, while new demonstrations specify a task, object or operating region. Recent work makes that interpretation more credible, and also exposes its dependencies. Robots can learn through trial and error and retain useful repairs; execution still needs observable outcomes and compatible actions. A fast virtual training environment is consequently a plausible product hypothesis, but the research first needs to establish which observed information makes training more efficient and which simulated improvements survive physical transfer.

The observation-to-skill thesis matches the evidence when each skill has a tested operating region

The following mapping preserves the sequence you described. “Supported” means a component has demonstrated precedents, not that the complete proposed architecture has been demonstrated.

diff --git a/docs/papers/hardware-matched-robot-simulation/index.html b/docs/papers/hardware-matched-robot-simulation/index.html index b4b18e1..4ffb9d3 100644 --- a/docs/papers/hardware-matched-robot-simulation/index.html +++ b/docs/papers/hardware-matched-robot-simulation/index.html @@ -1,7 +1,7 @@ Hardware matched robot simulation · DVIDIA Research -

Pinned mechanics implementation and CPU numerical benchmark

Hardware matched robot simulation

· DVIDIA

Pinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured

Source & history · Download Markdown

Abstract

A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.

In this paper

DVIDIA research · 11 October 2026 · Compiled reference assembly and unloaded CPU diagnostics; physical transfer unmeasured

+

Pinned mechanics implementation and CPU numerical benchmark

Hardware matched robot simulation

· DVIDIA

Pinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured

Source & history · Download Markdown

Abstract

A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming.

In this paper

DVIDIA research · 11 October 2026 · Compiled reference assembly and unloaded CPU diagnostics; physical transfer unmeasured

Build the first simulator around a specific six-axis arm and gripper, then measure the physical response that the learner will encounter. DVIDIA's vision remains a Skillspace that supplies demonstrations, local training and an installable skill. Rendered images support visual learning, but matching a room's appearance and the arm's number of joints does not establish that a learned action will produce the same motion, grip or outcome on hardware. We have compiled a pinned UR5e and Robotiq 2F-85 reference and run eighteen unloaded CPU diagnostics; this does not select the user's eventual purchase or reproduce that robot's controller. Later, two arms with five-finger hands need a different command, sensing and contact contract. Imported structure, numerical characterization and held-out physical agreement remain separate evidence states. Useful speed is the cost of acquiring a qualified capability, measured alongside errors and failures.

Six arm joints do not specify the whole robot

DVIDIA's existing dvidia-authored-6dof-parallel-jaw-v0 environment is an authored rigid-box contact experiment. It has six arm hinges, two simulated sliding jaws and eight native position actuators, exposed through six joint targets and one total-gap target. Arm links, base and palm have collision masks disabled; the experiment therefore does not evaluate whole-arm avoidance or self-collision. Exact object state, contact counts and pad forces are native simulator observations, not measured camera or fingertip signals. Preserve this profile and its frozen results while adding a separately named hardware reference. (Current arm source)

diff --git a/docs/papers/native-grasp-task-benchmark/index.html b/docs/papers/native-grasp-task-benchmark/index.html index 9b19976..a1acd43 100644 --- a/docs/papers/native-grasp-task-benchmark/index.html +++ b/docs/papers/native-grasp-task-benchmark/index.html @@ -1,7 +1,7 @@ Native grasp task benchmark · DVIDIA Research -

Native contact task implementation and authored baseline

Native grasp task benchmark

· DVIDIA

6/6 simulated placements · controls 0/12 · real-time and hardware unqualified

Source & history · Download Markdown

Abstract

A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.

In this paper

11 October 2026 · Native simulation benchmark and next learning experiment

+

Native contact task implementation and authored baseline

Native grasp task benchmark

· DVIDIA

6/6 simulated placements · controls 0/12 · real-time and hardware unqualified

Source & history · Download Markdown

Abstract

A pinned six-axis arm and coupled gripper complete 6/6 frozen synthetic rigid-block placements, while matched recorded-arm/open-jaw and hold controls complete 0/12. Native contact must produce grasp, sustained lift, retained carry, release and settled placement with tool retreat. All 18 attempts and timing traces remain in the receipt; one placement takes 29.51 wall seconds for 20 simulated seconds and misses 251 of 1,000 computation windows, so real-time operation is not established. An offline 3D mesh replay exposes the recorded motion. The authored privileged-state baseline provides a benchmark for a future compact DVIDIA action policy, not a learned footage-to-action model, validated shoe simulation or physical robot qualification.

In this paper

11 October 2026 · Native simulation benchmark and next learning experiment

DVIDIA's native contact benchmark completed 6/6 grasp, carry and release placements across six frozen synthetic layouts; matched open-jaw replay and hold controls completed 0/12. All 18 attempts were numerically valid. The six-axis reference arm, coupled parallel-jaw gripper and freely moving rigid block provide an executable task for a future learned action policy. The first baseline is authored and uses privileged simulator state; no policy has learned this task from footage. CPU execution also showed substantial timing variation, including a 29.51-second attempt for 20 seconds of simulated motion, so the result establishes task behavior under these settings rather than real-time control. Complete receipts, unsuccessful controls and source identities accompany the replay. Shoe organization and physical robot qualification require their own evidence. (Frozen confirmation receipt)

Native contact makes the task measurable

The reference combines six UR5e joint targets and one Robotiq 2F-85 actuator command. Its eight gripper linkage joints move through the upstream coupled mechanism; the action interface is one native jaw target in 0–255, rather than eight independent finger commands or a calibrated force request. The task adds a free-joint block and a support table while preserving the pinned arm/gripper geometry, inertias, active collision proxies and linkage constraints. An authored position controller and gravity feedforward act through the bounded native servos. During the rollout, the object moves through dynamics and contact rather than a weld or runtime pose assignment. The openly licensed simulation assets do not imply open-source UR5e or Robotiq physical hardware. (Frozen task source, hardware reference)

diff --git a/docs/papers/offline-cable-training-environment/index.html b/docs/papers/offline-cable-training-environment/index.html index 3432cc8..19605ce 100644 --- a/docs/papers/offline-cable-training-environment/index.html +++ b/docs/papers/offline-cable-training-environment/index.html @@ -1,7 +1,7 @@ Offline cable training environment · DVIDIA Research -

Product prototype and local qualification

Offline cable training environment

· DVIDIA

CPU simulation-only · Controller search and frozen evaluation executed

Source & history · Download Markdown

Abstract

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

In this paper

Product prototype and local qualification · 7 October 2026 · Native CPU simulation only

+

Product prototype and local qualification

Offline cable training environment

· DVIDIA

CPU simulation-only · Controller search and frozen evaluation executed

Source & history · Download Markdown

Abstract

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

In this paper

Product prototype and local qualification · 7 October 2026 · Native CPU simulation only

The first runnable product loop now connects an authored environment → local practice → a frozen controller → evaluation → an inspectable skill pack. CableEndReach-v0 bends an anchored cable using bounded forces at its terminal site. The selected controller completed 10/10 held-out simulated episodes, but a fixed feedback baseline also completed 10/10 and finished faster. The result establishes an executable research and packaging foundation, with explicit failure controls. It does not establish physical rope accuracy, shoe tying, realistic grasping or a performance advantage over existing simulators.

Runnable source and instructions · Open the recorded replay · Download the skill pack · Complete qualification

The task has an explicit operating envelope

diff --git a/docs/papers/physical-skill-research-agenda/index.html b/docs/papers/physical-skill-research-agenda/index.html index b4932d2..d484f42 100644 --- a/docs/papers/physical-skill-research-agenda/index.html +++ b/docs/papers/physical-skill-research-agenda/index.html @@ -1,7 +1,7 @@ Turn physical footage into testable evidence · DVIDIA Research -

Research agenda

Turn physical footage into testable evidence

· DVIDIA

Four baseline protocols proposed

Source & history · Download Markdown

Abstract

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

In this paper

DVIDIA should test the usefulness of its evidence before expanding collection volume. The immediate opportunity is to make demonstrations easier to inspect, compare and reproduce, then measure whether added structure or physical signals help a defined task. Sam Padilla's 5 October essay provides a strategic prompt; the technical sources support narrower, testable questions. The four baseline protocols proposed in this agenda have not run, and neither customer demand nor robot transfer has been established. Four contributor-sized studies cover capture quality, relations over time, provenance and physical grounding.

+

Research agenda

Turn physical footage into testable evidence

· DVIDIA

Four baseline protocols proposed

Source & history · Download Markdown

Abstract

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

In this paper

DVIDIA should test the usefulness of its evidence before expanding collection volume. The immediate opportunity is to make demonstrations easier to inspect, compare and reproduce, then measure whether added structure or physical signals help a defined task. Sam Padilla's 5 October essay provides a strategic prompt; the technical sources support narrower, testable questions. The four baseline protocols proposed in this agenda have not run, and neither customer demand nor robot transfer has been established. Four contributor-sized studies cover capture quality, relations over time, provenance and physical grounding.

The essay motivates a hypothesis, not a market forecast

In Robotics Data: Bet Hard Or Get Out, Sam Padilla, 5 October 2026, argues for exceptional collection scale, distinctive physical information or synthetic generation. He warns that buyers can internalize embodiment-specific collection and points to opportunities beyond raw data supply. His conversations and volume anecdote are not independently verified demand evidence. (Original essay).

Our inference is a practical test: does an attributable demonstration with observable state changes improve a specific evaluation more than a clip with coarse tags? This would establish a useful property of a research artifact. It would not establish willingness to pay, a durable business model or an executable robot skill. The first deliverable should therefore be a small benchmark with a clear failure condition, rather than a volume target.

diff --git a/docs/papers/piper-contact-history-v03/index.html b/docs/papers/piper-contact-history-v03/index.html index 641dcd2..21fda30 100644 --- a/docs/papers/piper-contact-history-v03/index.html +++ b/docs/papers/piper-contact-history-v03/index.html @@ -1,7 +1,7 @@ PiPER contact history v0.3 · DVIDIA Research -

Paired probe-only and contact-history experiment with complete confirmation receipt

PiPER contact history v0.3

· DVIDIA

History 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified

Source & history · Download Markdown

Abstract

A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.

In this paper

DVIDIA research · 11 October 2026 · Contact history v0.3

+

Paired probe-only and contact-history experiment with complete confirmation receipt

PiPER contact history v0.3

· DVIDIA

History 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified

Source & history · Download Markdown

Abstract

A frozen paired PiPER experiment retains all 84 numerically valid attempts across twelve synthetic rigid-block scenes. History completes 6/12 placements versus 4/12 for both original motion and the identical acquisition probe without carry monitoring; all 48 separately matched open-jaw and hold controls fail. The two added completions occur at nominal friction with 60 g blocks. Four tasks complete after five closed holds/resumes, but only four of five resume windows meet the independent low absolute TCP-speed criterion. All six supported acquisition retries fail. Native gains, forces and outcome rules remain unchanged; requested movement reduction is distinct from actual speed, and two slowdown windows include early holds. Seventy-five of 84,000 computation windows exceed 20 ms. Complete sealed trajectories, original model notices, source-snapshotted development and pinned offline inspection preserve both success and failure. Privileged authored simulation establishes no learned skill, true-slip detector accuracy, shoe competence or physical qualification.

In this paper

DVIDIA research · 11 October 2026 · Contact history v0.3

History completes 6/12 placements, versus 4/12 for both original motion and probe-only motion. The two additional completions are the 60 g, nominal-friction cases; all three modes complete both 40 g moderate-friction cases and fail the 60 g moderate-friction and all low-friction cases. All 84 planned attempts remain numerically valid, including 48 unsuccessful negative controls. Four history tasks resume after a closed hold and complete, but all six supported acquisition retries still fail. This is a descriptive result on synthetic rigid blocks with privileged state, not detector accuracy, a learned skill or physical robot qualification. (Complete original report)

The question left by the previous guard

The earlier grip experiment diff --git a/docs/papers/piper-contact-recovery-v04/index.html b/docs/papers/piper-contact-recovery-v04/index.html index acc3a28..465a87b 100644 --- a/docs/papers/piper-contact-recovery-v04/index.html +++ b/docs/papers/piper-contact-recovery-v04/index.html @@ -1,7 +1,7 @@ PiPER contact recovery v0.4 · DVIDIA Research -

Paired carry-recovery and supported-acquisition experiment with complete confirmation receipt

PiPER contact recovery v0.4

· DVIDIA

All modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified

Source & history · Download Markdown

Abstract

A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.

In this paper

DVIDIA research · 11 October 2026 · Contact recovery v0.4

+

Paired carry-recovery and supported-acquisition experiment with complete confirmation receipt

PiPER contact recovery v0.4

· DVIDIA

All modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified

Source & history · Download Markdown

Abstract

A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification.

In this paper

DVIDIA research · 11 October 2026 · Contact recovery v0.4

v0.4 completes the same 4/12 placements as v0.3 on fresh layouts, while two completed recoveries finish 2.440 and 2.640 simulated seconds sooner. The carry-only and full versions have the same completion count; none is gained or lost. Four low-friction failures reach refusal 2.88–2.90 seconds sooner. All 84 planned attempts are numerically valid, including 48 failed negative controls. Eight placements still fail in each mode. Two nominal-friction 60 g cases that succeeded in the older cohort now remain suspended below the unchanged resume-height gate and time out in every mode. This establishes faster progress after settling on two tasks, with continuing acquisition and resume limitations. It is not a learned skill, true-slip accuracy or physical robot qualification. (Complete frozen report)

The six failed placements in contact history v0.3 all fail before acquiring a load-bearing grasp. A faster carry response cannot repair a grip that never lifts its object. Version 0.4 therefore separates two changes: restoring requested movement after a settled carry hold, and avoiding an unchanged supported retry when current observations indicate that it is not helping. Completed placement, successful recovery and faster refusal remain separate results.

Diagnose the six failures first

diff --git a/docs/papers/piper-grip-validation-and-recovery/index.html b/docs/papers/piper-grip-validation-and-recovery/index.html index 56b0b8a..a6b6bfa 100644 --- a/docs/papers/piper-grip-validation-and-recovery/index.html +++ b/docs/papers/piper-grip-validation-and-recovery/index.html @@ -1,7 +1,7 @@ PiPER grip validation and recovery · DVIDIA Research -

Paired authored controller experiment and complete confirmation receipt

PiPER grip validation and recovery

· DVIDIA

Moderate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified

Source & history · Download Markdown

Abstract

A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.

In this paper

DVIDIA research · 11 October 2026 · Authored native simulation experiment

+

Paired authored controller experiment and complete confirmation receipt

PiPER grip validation and recovery

· DVIDIA

Moderate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified

Source & history · Download Markdown

Abstract

A slow test lift and bounded supported retry add recorded acquisition decisions to the pinned PiPER contact pilot, but completion regresses: at moderate friction baseline completes 2/2 and guarded completes 0/2. Both complete 2/2 normal cases; neither completes the two low-friction cases. All 30 attempts remain numerically valid, including 18 failed controls and full-duration aborts. Four qualification decisions are followed by a brief independent following window, yet two tasks still fail; no successful recovery or overall improvement is demonstrated. Actual motion continues after hold requests, and 53/30,000 computation windows exceed 20 ms. Complete compressed receipts, original model notices and pinned offline inspection preserve the evidence. Authored privileged-state simulation establishes no detector accuracy, learned shoe skill, calibrated contact or physical robot qualification.

In this paper

DVIDIA research · 11 October 2026 · Authored native simulation experiment

The guarded controller regresses completion at moderate friction: baseline 2/2, guarded 0/2. Both complete 2/2 normal-friction placements and neither completes the two low-friction cases. Across the six paired scenes, placement completion falls from baseline 4/6 to guarded 2/6; all 18 open-jaw/hold controls fail and all 30 attempts remain numerically valid. The new controller exposes acquisition decisions, alarms, one supported retry and refusal, but demonstrates neither a successful recovery nor an overall completion improvement. These are synthetic rigid-block results, not a learned skill or physical robot qualification. (Complete original report)

A pinch is not yet a load-bearing grasp

The earlier PiPER contact pilot completed both nominal placements but failed to lift its slippery block. Both fingers registered contact while the tool moved away from the table and the block stayed supported. The missing decision was whether the object followed the gripper under load. This experiment adds a bounded test lift before the full carry. It preserves the pinned model, native servo settings, force limits, free-body object and independent task scorer. (Earlier pilot)

diff --git a/docs/papers/piper-reliability-v05/index.html b/docs/papers/piper-reliability-v05/index.html index d7b0cba..6106c7e 100644 --- a/docs/papers/piper-reliability-v05/index.html +++ b/docs/papers/piper-reliability-v05/index.html @@ -1,7 +1,7 @@ PiPER placement reliability v0.5 · DVIDIA Research -

Frozen paired reliability experiment with complete native evidence

PiPER placement reliability v0.5

· DVIDIA

23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified

Source & history · Download Markdown

Abstract

Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.

In this paper

DVIDIA research · 12 October 2026 · Placement reliability v0.5

+

Frozen paired reliability experiment with complete native evidence

PiPER placement reliability v0.5

· DVIDIA

23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified

Source & history · Download Markdown

Abstract

Three frozen fresh PiPER cohorts retain 252 numerically valid attempts and 144 failed negative controls. A fixed diagonal grasp with bounded pose planning completes 23/36 placements versus 15/36 for frozen v0.4 and 18/36 with second-lift recovery alone: eight gained completions and none lost. The fifteen shared successes are 0.772 simulated seconds slower on average. Three lift-only second probes qualify and finish their tasks; the full grasp candidate needs none. Twelve low-friction acquisitions and one later retention failure remain. Every trajectory, native task outcome and raw computation duration is retained, including 1,035 windows exceeding 20 ms and a 3.082-second outlier on a loaded host. Native mechanics and scoring remain unchanged. Separate development keeps 200 earlier attempts, including discarded regressions. Authored privileged simulation establishes no learned skill, real-time guarantee or physical qualification.

In this paper

DVIDIA research · 12 October 2026 · Placement reliability v0.5

The revised grasp completes 23/36 placements, compared with 15/36 for frozen v0.4 and 18/36 with the second lift alone. Across three fresh cohorts, it gains eight completions and loses none, reducing failed placements from 21 to 13 in this narrow synthetic comparison. All 252 attempts are numerically valid; all 144 negative controls fail, with no solver warnings or prohibited native contact ticks. The fifteen shared successes take 0.772 simulated seconds longer on average with the revised grasp. Twelve low-friction acquisitions and one carry-retention failure remain. This is a measured completion improvement with a task-speed tradeoff, not learned competence or physical qualification. (Complete aggregate evidence)

The previous v0.4 recovery experiment made two recovered tasks finish sooner but did not complete more placements. Its failures exposed two different problems: a held object could settle below the original resume-height gate, or a supported object could never follow the initial pinch. Faster refusal did not solve either placement. Version 0.5 tests a separate response to each mechanism and measures the entire task again.

Two changes, one unchanged physical task

diff --git a/docs/papers/robotics-research-pipeline/index.html b/docs/papers/robotics-research-pipeline/index.html index 8fd8a54..b5e5e8d 100644 --- a/docs/papers/robotics-research-pipeline/index.html +++ b/docs/papers/robotics-research-pipeline/index.html @@ -1,7 +1,7 @@ Robotics research pipeline · DVIDIA Research -

Architecture proposal and staged validation roadmap

Robotics research pipeline

· DVIDIA

Architecture proposal · current simulation alpha · hardware unqualified

Source & history · Download Markdown

Abstract

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

In this paper

DVIDIA's product goal is a local training environment that converts Skillspace evidence into a versioned skill a compatible robot can install, test and execute with feedback. The next architectural step couples reviewed task descriptions to measured robot state, allowing a skill to decide whether a grasp worked, whether progress stopped and whether to retry or abstain. This paper specifies that pipeline and its validation gates. It adds no trained supervisor, human-video action bridge or hardware qualification. The implemented foundation remains a CPU research alpha with bounded simulation evidence. (Public training package, Published measurements)

+

Architecture proposal and staged validation roadmap

Robotics research pipeline

· DVIDIA

Architecture proposal · current simulation alpha · hardware unqualified

Source & history · Download Markdown

Abstract

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

In this paper

DVIDIA's product goal is a local training environment that converts Skillspace evidence into a versioned skill a compatible robot can install, test and execute with feedback. The next architectural step couples reviewed task descriptions to measured robot state, allowing a skill to decide whether a grasp worked, whether progress stopped and whether to retry or abstain. This paper specifies that pipeline and its validation gates. It adds no trained supervisor, human-video action bridge or hardware qualification. The implemented foundation remains a CPU research alpha with bounded simulation evidence. (Public training package, Published measurements)

Pipeline at a glance

DVIDIA robotics pipeline: source evidence branches into reviewed label proposals and synchronized numerical telemetry; grouped datasets feed separate visual and motor learners, a proposed state supervisor, frozen simulation evaluation and compatible skill installation. Runtime observations return to bounded decisions and future reviewed recordings.
DVIDIA robotics pipeline: source evidence branches into reviewed label proposals and synchronized numerical telemetry; grouped datasets feed separate visual and motor learners, a proposed state supervisor, frozen simulation evaluation and compatible skill installation. Runtime observations return to bounded decisions and future reviewed recordings.

Figure 1. Solid elements represent implemented components within the bounded simulation pipeline; dashed elements represent proposed integrations. Physical robot execution remains unqualified. A visual model, reviewed label and executable motor policy have different evidence requirements. The diagram describes a target architecture, not a newly completed end-to-end system.

diff --git a/docs/papers/sensor-corroboration-v04/index.html b/docs/papers/sensor-corroboration-v04/index.html new file mode 100644 index 0000000..ed54450 --- /dev/null +++ b/docs/papers/sensor-corroboration-v04/index.html @@ -0,0 +1,30 @@ + + +Independent camera and actuator corroboration v0.4 · DVIDIA Research +

Paired synthetic benchmark for camera and drive evidence

Independent camera and actuator corroboration v0.4

· DVIDIA

272 cases · 69/112 false completions prevented · 43 remain · 73/80 true completions abstained

Source & history · Download Markdown

Abstract

A fresh eight-layout PiPER simulation benchmark adds separately processed RGB and drive checks to a local physical-goal verifier. The checks prevent 69 of 112 false completion readings while leaving 43 replay, identity, orientation, support and sampling blind spots; the conservative gate abstains on 73 of 80 baseline true completions, including all 64 deliberate sensor-fault controls. The measured CPU path is 22.1 ms median and 31.5 ms p95, excluding 200 ms acquisition. All raw channels and results are preserved; this is synthetic evidence, not hardware validation.

In this paper

DVIDIA robotics research · 19 October 2026.

+

The local physical-goal verifier v0.3 trusts its observation packet. This experiment adds separately processed RGB and PiPER drive channels, then asks whether those channels can contradict a primary “complete” reading. It is a simulator study of evidence fusion, not a hardware safety claim.

+

The frozen benchmark contains 272 cases across eight layouts: 96 native scenes, 80 primary-packet forgeries, 64 sensor faults and 32 cross-channel controls. All 96 native cases generated validly. The v0.3 baseline called 192 cases complete, including 112 physical false completions. The v0.4 gate retained 50 complete verdicts: seven physical positives and 43 false completions. It prevented 69/112 baseline false completions, but 43 remained. It also abstained on 73/80 baseline true completions; 64 had deliberately missing, stale, gapped or malformed auxiliary evidence, and nine were physically complete but visually unobservable.

+

The result is useful because it shows both what a second signal catches and what two weak signals cannot establish. The camera detects large motion, position disagreement and many target-visibility failures. The drive path checks finger aperture, command, motor effort, settling and servo consistency. Neither authenticates its source. These channels still miss a 180° orientation reversal, a green lookalike, elevated support, motion that aliases between samples, and refreshed or fully replayed channels.

+

Paired outcome

+
Measurev0.3 primary packetv0.4 camera + drive gate
Scheduled cases272272
Complete verdicts19250
Physically true completions recognized80/807/80
False completions among physical negatives112/19243/192
Baseline false completions prevented—69/112
Baseline true completions lost—73/80
Unknown verdicts0/272142/272
Invalid native generation00
+

The 64 sensor-fault controls remain physically complete; all 64 return unknown when their required camera or drive evidence is missing or invalid. In the native cohort, seven of eight visible settled scenes pass. Every explicitly occluded scene abstains, and one nominally visible shoe is obscured by the simulated arm. These are conservative refusals, not evidence that the physical task failed.

+

Among the 112 baseline false completions, the gate prevents 69 and leaves 43. The residual cases are not one generic detector error:

+
Residual controlFalse completions that pass
Instantaneous motion alias7/8
180° yaw flip8/8
Visually identical green lookalike7/8
Elevated object with hidden support surface7/8
Refreshed camera replay7/8
Coherent replay of all channels7/8
+

The layout with an occluded camera path abstains, which is why several controls show seven rather than eight false completions. A coherent replay remains compatible with the channels' declared clock labels; there is no cryptographic sensor provenance in this simulator.

+

What the channels measure

+

The camera observer receives only eleven 640×480 RGB frames at 20 ms intervals. A fixed orthographic projection maps a green elongated proxy to XY position and an undirected axis. It receives no object ID, simulator pose, segmentation mask or physical truth. Its color-and-shape rule is not a general shoe detector. Axial geometry cannot distinguish the toe from a 180°-reversed shoe, a same-color decoy is ambiguous, and RGB alone cannot measure height above the table.

+

The drive observer receives one hundred PiPER joint/drive samples at 2 ms intervals: six arm channels, two signed finger positions and velocities, gripper command and simulated motor effort. It validates ranges, units, finger symmetry and a declared spring/servo consistency relation, then checks that the fingers are open, commanded open, settled and at low effort. Motor effort is not object contact force. Neither channel can verify table support or rule out contact from every arm link.

+

Both channels originate in the same MuJoCo model as the primary packet. They use separate processing code, but are not independent physical sensors. Calibration bounds and IDs are synthetic declarations, not authenticated measurements. An unknown result means the verifier refuses to confirm; it does not mean the scene is physically incomplete.

+

Compute and limits

+

On the recorded Apple Silicon host, the complete processed decision path had 22.1 ms median and 31.5 ms p95 across 816 repeated case evaluations. RGB extraction alone had 14.3 ms median and 20.1 ms p95; the baseline verifier had 0.42 ms median. The optional OpenCV CPU kernel was installed for this run. No GPU or hosted model call was used.

+

The processed-path timer includes local JSON reads, PNG decoding, primary verification, RGB observation, drive observation and fusion. It excludes the 200 ms observation history, native rendering/acquisition, networking and robot actuation. The p95 is longer than a 20 ms camera frame interval, so this is not a demonstrated real-time control loop or hard-real-time bound. The observer uses an optional CPU package for speed and keeps a NumPy-only fallback; no minimum hardware specification has been established.

+

The next useful measurement is a hardware-in-the-loop capture with timestamped camera, encoders, motor current and a second view that can see object height and support. Before that, the simulator needs stronger toe-direction and identity cues, explicit bounded-motion claims, and anti-replay provenance. Training a decision model on these labels would not remove those missing observables.

+

Reproduce and inspect

+ +

Source commit 90e84701ce7d5bc330ef41b02793e2ce037d702d freezes the final seed, eight layouts, 272-case inventory, and three timing repetitions. Every scheduled case and every sensor stream remains in the release. No provider, Jev API, physical robot or learned controller was used. Original DVIDIA code is MIT; the pinned simulator and robot assets retain their upstream terms.

+
diff --git a/docs/papers/shoe-asset-library-v01/index.html b/docs/papers/shoe-asset-library-v01/index.html index d12d618..9909ea6 100644 --- a/docs/papers/shoe-asset-library-v01/index.html +++ b/docs/papers/shoe-asset-library-v01/index.html @@ -1,7 +1,7 @@ Reusable shoe asset library v0.1 · DVIDIA Research -

Versioned native scene assets and frozen passive-contact benchmark

Reusable shoe asset library v0.1

· DVIDIA

Six shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified

Source & history · Download Markdown

Abstract

Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.

In this paper

DVIDIA research · 12 October 2026 · Asset library v0.1

+

Versioned native scene assets and frozen passive-contact benchmark

Reusable shoe asset library v0.1

· DVIDIA

Six shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified

Source & history · Download Markdown

Abstract

Six original MIT rigid shoe proxies compose with the pinned UR5e/Robotiq, PiPER and SO-101 models through versioned scene descriptors and offline native 3D previews. A frozen 54-attempt test records 27/27 supported pairs and 0/27 support successes when table collisions are disabled; all 54 gravity checks pass without numerical warnings. No loaded non-table shoe contacts appear in the 20 ms samples, which cannot rule out shorter transients. All sampled trajectories and 27,000 native-step durations remain available. On one Apple M5 host, 54 simulated seconds take 12.3616 wall seconds including fresh model construction; the slowest attempt takes 6.7047 seconds. Geometry and physical properties are authored assumptions. Loading, gravity and passive support do not establish shoe grasping, calibrated deformation, a learned policy or hardware transfer.

In this paper

DVIDIA research · 12 October 2026 · Asset library v0.1

The first DVIDIA asset library loads six original rigid shoe proxies into native simulation with UR5e/Robotiq, PiPER or SO-101. A versioned scene file records the exact catalog, robot, shoe identities and timestep. Users can browse the native geometry in 3D, download a selected pair's scene descriptor, and reproduce it locally using dvidia-assets. This release supplies reusable scene components and passive contact tests. A shoe-organizing policy is a subsequent task.

Explore the asset library · Hugging Face lab · Every benchmark attempt

What a shoe asset contains

diff --git a/docs/papers/skill-capsule-acquisition/index.html b/docs/papers/skill-capsule-acquisition/index.html index 5f3d819..a977df4 100644 --- a/docs/papers/skill-capsule-acquisition/index.html +++ b/docs/papers/skill-capsule-acquisition/index.html @@ -1,7 +1,7 @@ Skill capsule acquisition and optimization · DVIDIA Research -

Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

· DVIDIA

Simulation-only · teacher/student 7/8 · no overall speedup

Source & history · Download Markdown

Abstract

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

In this paper

Measured prototype and proposed next steps · 7 October 2026 · Simulation-only

+

Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

· DVIDIA

Simulation-only · teacher/student 7/8 · no overall speedup

Source & history · Download Markdown

Abstract

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

In this paper

Measured prototype and proposed next steps · 7 October 2026 · Simulation-only

DVIDIA now has a compact learned movement artifact that can be packaged for a compatible simulated arm. A normalized RBF head replaces the teacher's joint-target computation; task phases, waypoints, gripper/contact supervision, and bounded recovery remain authored. The frozen head contains 34,171 canonical JSON bytes, and the compiled candidate capsule contains 39,845 bytes. In a new eight-scene evaluation, teacher and student both completed 7/8, without teacher fallback, while total episode wall time increased from 6.273 to 6.360 seconds. This establishes a bounded teacher-to-student acquisition step, with neither reliability improvement nor overall acceleration demonstrated. The useful product path is import → compatibility check → actual local execution and negative control → scene-specific qualification → execution. Human-video learning, calibrated perception/contact, and physical-arm competence remain separate future capabilities. (Training receipt, profiling, candidate capsule).

The acquired behavior is movement inside an authored task

The experiment declares 16 training scenes, four separate development-selection scenes, and eight evaluation scenes with disjoint scene digests. Historical v0.3 held-out cases were excluded and their earlier results remain unchanged. The teacher completed 16/16 collection scenes, generating 6,344 supervised rows: 3,172 sampled native teacher calls and 3,172 extra teacher queries with perturbed Cartesian goals at the same physical states. Augmentation adds movement labels, not another 3,172 physical attempts. The full canonical dataset occupies 3,814,171 bytes. (Protocol, collection).

diff --git a/docs/papers/skillspace-arm-benchmarks/index.html b/docs/papers/skillspace-arm-benchmarks/index.html index e1357f5..b0bba84 100644 --- a/docs/papers/skillspace-arm-benchmarks/index.html +++ b/docs/papers/skillspace-arm-benchmarks/index.html @@ -1,7 +1,7 @@ Skillspace arm benchmarks and evolution · DVIDIA Research -

Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

· DVIDIA

Mixed held-out result · 30/32 baseline, 29/32 updated

Source & history · Download Markdown

Abstract

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

In this paper

Research and implementation assessment · 7 October 2026 · Simulation-only evidence

+

Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

· DVIDIA

Mixed held-out result · 30/32 baseline, 29/32 updated

Source & history · Download Markdown

Abstract

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

In this paper

Research and implementation assessment · 7 October 2026 · Simulation-only evidence

Skillspace → install → simulated arm makes sense as a grounded, compatible execution workflow. The present implementation imports task metadata, binds an explicitly authored adapter, supplies a metric scene, executes native contact dynamics, and checks completion. Two engineering rounds increased development success from 14/20 to 16/20 to 17/20, but the frozen held-out comparison declined from 30/32 to 29/32: both controllers passed all 24 nominal cases, while actuator-stress success fell from 6/8 to 5/8. There were zero held-out improvements and one regression. Observed instrumented CPU throughput also declined from 12.283× to 11.388× simulated seconds per wall second. The result establishes an auditable simulation workflow and useful failure diagnostics; it establishes neither improved held-out reliability nor calibrated physical accuracy. Skillspaces should evolve toward immutable releases that bind task, embodiment, observations, controller, and independently evaluated evidence. (Frozen assessment).

Installation supplies an explicit execution bridge

The public place_cup envelope contains a task label, robot identifier, recording format, and demonstration count. It does not contain learned weights, actions, object dimensions, metric target coordinates, or camera calibration. The local bridge supplies a declared rigid-box proxy and contact-pick-place-box-v0 adapter for dvidia-authored-6dof-parallel-jaw-v0. Thus installation selects existing authored behavior; it does not infer cup manipulation from the referenced demonstrations. This distinction preserves the useful architecture: language identifies the goal, grounding supplies the state and physical assumptions, a compatible controller acts, and a verifier judges the result. (Public metadata, execution guide).

diff --git a/docs/papers/skillspace-arm-installation/index.html b/docs/papers/skillspace-arm-installation/index.html index b07e947..92c01ce 100644 --- a/docs/papers/skillspace-arm-installation/index.html +++ b/docs/papers/skillspace-arm-installation/index.html @@ -1,7 +1,7 @@ Installing a Skillspace on a simulated arm · DVIDIA Research -

Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

· DVIDIA

Public skill link to authored adapter · 6/6 scene placements

Source & history · Download Markdown

Abstract

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

In this paper

Research note, 7 October 2026. The current bridge makes a narrow part of the proposed skill workflow executable: import a DVIDIA skill envelope, install an explicitly authored local simulation adapter, create a scene, and run an articulated arm. It provides a concrete place to investigate observation-to-simulation-to-skill packaging. It does not yet infer an executable policy, object geometry, material properties, or robot calibration from human video.

+

Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

· DVIDIA

Public skill link to authored adapter · 6/6 scene placements

Source & history · Download Markdown

Abstract

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

In this paper

Research note, 7 October 2026. The current bridge makes a narrow part of the proposed skill workflow executable: import a DVIDIA skill envelope, install an explicitly authored local simulation adapter, create a scene, and run an articulated arm. It provides a concrete place to investigate observation-to-simulation-to-skill packaging. It does not yet infer an executable policy, object geometry, material properties, or robot calibration from human video.

The public example is DVIDIA place_cup, whose metadata is served at place_cup/skill.json. The recorded 360-byte source is vdia:0, version 0.3.0, with task “Set a cup on the mark.” Its exact-byte SHA-256 is 16b4e1ef0f3799345c1d18ce9f1da83b072863ab00ce30a83424080fb467d4ec. The envelope declares a demonstration count, recording format and robot identifier; these declarations do not supply trajectories, observations, trained weights, or an evaluated controller. The installation receipt preserves the source bytes, URL, source hash, separate grounding hash, adapter identity and physical_robot_ready: false. A later download may differ; its receipt must identify its own bytes.

In the local studio, one click installs the compatible public envelope and binds contact-pick-place-box-v0 to dvidia-authored-6dof-parallel-jaw-v0. A bundled envelope supports installation without fetching the public site. JSON upload/drop accepts the supported DVIDIA formats. Missing grounding is supplied through a visible, locally authored adapter; inline grounding remains separately validated. Installation downloads metadata and executes the already installed local controller. Object and target coordinates belong to the scene and can change between executions. The browser displays recorded native simulation poses rather than inventing an object trajectory. Execution and replay are designed to work with installed dependencies and local assets; the broader product's air-gapped deployment remains a separate validation task.

The embodiment is an original six-hinge arm with two actuated sliding jaws. Native MuJoCo 3.15.0 simulates a free rigid box, gravity and jaw/object/table contact. The default proxy is a 40 mm cube weighing 40 g on a table at 0.29 m. No weld, mocap attachment, or object-state update carries the object during execution. Arm controls use bounded position servos, model-based bias compensation and current-state damped Jacobian control. The declared limits are 40 N·m per arm actuator and 15 N per jaw. Physics uses a 1 ms timestep, implicitfast integration, Newton solving and an elliptic Coulomb friction cone. These choices describe an authored fixture, not measured hardware or calibrated material fidelity.

diff --git a/docs/papers/skillspace-footage-training/index.html b/docs/papers/skillspace-footage-training/index.html index afd4aaf..2296a02 100644 --- a/docs/papers/skillspace-footage-training/index.html +++ b/docs/papers/skillspace-footage-training/index.html @@ -1,7 +1,7 @@ Skillspace footage training pipeline · DVIDIA Research -

Product implementation and measured synthetic pilot

Skillspace footage training pipeline

· DVIDIA

Public CPU package · one native scene qualified · simulation-only

Source & history · Download Markdown

Abstract

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

In this paper

Research pipeline evolution — 9 October 2026. The Robotics research pipeline adds the proposed annotation-review and measured-state task-supervisor layers around this implemented training path. It includes an architecture diagram, data contracts and separate validation gates. Those additions are proposals; the recorded pilots and frozen sample-count results below retain their original scope.

+

Product implementation and measured synthetic pilot

Skillspace footage training pipeline

· DVIDIA

Public CPU package · one native scene qualified · simulation-only

Source & history · Download Markdown

Abstract

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

In this paper

Research pipeline evolution — 9 October 2026. The Robotics research pipeline adds the proposed annotation-review and measured-state task-supervisor layers around this implemented training path. It includes an architecture diagram, data contracts and separate validation gates. Those additions are proposals; the recorded pilots and frozen sample-count results below retain their original scope.

The public, standalone DVIDIA training repository provides a local CPU pipeline that turns actual Skillspace media into a fitted visual model and, when aligned numeric actions are supplied, a learned movement head. The useful advance is a reproducible path from source files through provenance checks, grouped splits, measured fitting and portable artifacts. Video prediction and robot movement remain separate claims. Ten synthetic six-second clips demonstrate visual fitting; simulator telemetry supplies the movement pilot's action labels. Its exported capsule installs and passes native qualification in one exact scene, while identical joint targets with open jaws fail. Neither experiment learns shoe organization or reconstructs robot actions from human footage. This report documents the implemented path and its limited measured evidence, rather than establishing a sufficient demonstration count or hardware competence. (Standalone source archive, Qualification receipt)

Published artifacts on Hugging Face

The DVIDIA organization on Hugging Face mirrors the measured pilot for inspection and local reproduction:

diff --git a/docs/papers/spatially-grounded-downloadable-robot-skills/index.html b/docs/papers/spatially-grounded-downloadable-robot-skills/index.html index 9470c20..a461fe3 100644 --- a/docs/papers/spatially-grounded-downloadable-robot-skills/index.html +++ b/docs/papers/spatially-grounded-downloadable-robot-skills/index.html @@ -1,7 +1,7 @@ Spatially grounded downloadable robot skills · DVIDIA Research -

Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

· DVIDIA

Source review · DVIDIA experiments not run

Source & history · Download Markdown

Abstract

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

In this paper

Research synthesis and proposed experiments · 7 October 2026 · No DVIDIA experiment in this report has run

+

Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

· DVIDIA

Source review · DVIDIA experiments not run

Source & history · Download Markdown

Abstract

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

In this paper

Research synthesis and proposed experiments · 7 October 2026 · No DVIDIA experiment in this report has run

My judgment is that the thesis identifies a useful architecture, but its strongest version needs a measurable operating boundary. Human demonstrations can teach what a task means, which state changes matter, and where to inspect; turning those observations into robot control needs an action and embodiment bridge. A command-conditioned spatial prior could accelerate target discovery, provided observation corrects it when a shoe is on a table or the person changes posture. Simulation can expand grounded experience, but generated volume cannot establish coverage of every material, contact, damaged object, or recovery. Fifty examples could adapt a substantial pretrained physical skill model; they are not an established threshold for acquiring all of shoe tying. The falsifiable research claim is: for a fixed pretrained model and robot, task-state annotations, spatially directed inspection, and targeted real/simulated recovery data improve held-out physical success per unit of collection cost, while preserving performance on unusual placements. A downloadable skill is the resulting tested controller, its compatibility and task contract, and evidence of where it works.

This synthesis combines three parallel primary-source research tracks with the existing DVIDIA research context. Grok 4.7 was inaccessible and was not used: the inspected browser was signed out, the exact model could not be verified, and no prompt or response exists. Recent 2026 results below include author-reported preprints; none was independently reproduced here. Architecture, experiments, budgets, and release criteria are proposals.

Demonstrations teach meaning before they supply motor control

diff --git a/docs/publications.json b/docs/publications.json index 41d0ee5..d7adc71 100644 --- a/docs/publications.json +++ b/docs/publications.json @@ -5,6 +5,19 @@ "follow": "https://x.com/iammrriver", "contributor": null, "publications": [ + { + "slug": "sensor-corroboration-v04", + "title": "Independent camera and actuator corroboration v0.4", + "source": "reports/Sensor corroboration v04.md", + "date": "2026-10-19", + "type": "Paired synthetic benchmark for camera and drive evidence", + "status": "272 cases · 69/112 false completions prevented · 43 remain · 73/80 true completions abstained", + "topics": [ + "physical-grounding", + "evaluation-provenance" + ], + "abstract": "A fresh eight-layout PiPER simulation benchmark adds separately processed RGB and drive checks to a local physical-goal verifier. The checks prevent 69 of 112 false completion readings while leaving 43 replay, identity, orientation, support and sampling blind spots; the conservative gate abstains on 73 of 80 baseline true completions, including all 64 deliberate sensor-fault controls. The measured CPU path is 22.1 ms median and 31.5 ms p95, excluding 200 ms acquisition. All raw channels and results are preserved; this is synthetic evidence, not hardware validation." + }, { "slug": "local-goal-verifier-v03", "title": "A local verifier for physical completion", diff --git a/docs/sitemap.xml b/docs/sitemap.xml index fe5be11..84f227f 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -1,2 +1,2 @@ -https://research.dvidia.org/2026-10-19https://research.dvidia.org/papers/local-goal-verifier-v03/2026-10-19https://research.dvidia.org/papers/image-goal-jev-v02/2026-10-19https://research.dvidia.org/papers/image-goal-jev-v01/2026-10-18https://research.dvidia.org/papers/goal-images-physical-feedback/2026-10-18https://research.dvidia.org/papers/shoe-graspability-v03/2026-10-14https://research.dvidia.org/papers/continuous-shoe-grip-v02/2026-10-13https://research.dvidia.org/papers/skill-prerequisites-v01/2026-10-13https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/2026-10-13https://research.dvidia.org/papers/shoe-asset-library-v01/2026-10-12https://research.dvidia.org/papers/piper-reliability-v05/2026-10-12https://research.dvidia.org/papers/piper-contact-recovery-v04/2026-10-11https://research.dvidia.org/papers/piper-contact-history-v03/2026-10-11https://research.dvidia.org/papers/piper-grip-validation-and-recovery/2026-10-11https://research.dvidia.org/papers/native-grasp-task-benchmark/2026-10-11https://research.dvidia.org/papers/hardware-matched-robot-simulation/2026-10-11https://research.dvidia.org/papers/dvidia-observation-lab/2026-10-11https://research.dvidia.org/papers/robotics-research-pipeline/2026-10-09https://research.dvidia.org/papers/skillspace-footage-training/2026-10-08https://research.dvidia.org/papers/skill-capsule-acquisition/2026-10-07https://research.dvidia.org/papers/skillspace-arm-benchmarks/2026-10-07https://research.dvidia.org/papers/skillspace-arm-installation/2026-10-07https://research.dvidia.org/papers/offline-cable-training-environment/2026-10-07https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/2026-10-07https://research.dvidia.org/papers/grok-robotics-research-on-x/2026-10-07https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/2026-10-07https://research.dvidia.org/papers/clef-and-grounded-video-observations/2026-10-06https://research.dvidia.org/papers/from-skillspace-to-robot-skill/2026-10-06https://research.dvidia.org/papers/physical-skill-research-agenda/2026-10-06https://research.dvidia.org/topics/capture-quality/2026-10-11https://research.dvidia.org/topics/skill-relations/2026-10-19https://research.dvidia.org/topics/evaluation-provenance/2026-10-19https://research.dvidia.org/topics/physical-grounding/2026-10-19 +https://research.dvidia.org/2026-10-19https://research.dvidia.org/papers/sensor-corroboration-v04/2026-10-19https://research.dvidia.org/papers/local-goal-verifier-v03/2026-10-19https://research.dvidia.org/papers/image-goal-jev-v02/2026-10-19https://research.dvidia.org/papers/image-goal-jev-v01/2026-10-18https://research.dvidia.org/papers/goal-images-physical-feedback/2026-10-18https://research.dvidia.org/papers/shoe-graspability-v03/2026-10-14https://research.dvidia.org/papers/continuous-shoe-grip-v02/2026-10-13https://research.dvidia.org/papers/skill-prerequisites-v01/2026-10-13https://research.dvidia.org/papers/live-piper-shoe-workstation-v01/2026-10-13https://research.dvidia.org/papers/shoe-asset-library-v01/2026-10-12https://research.dvidia.org/papers/piper-reliability-v05/2026-10-12https://research.dvidia.org/papers/piper-contact-recovery-v04/2026-10-11https://research.dvidia.org/papers/piper-contact-history-v03/2026-10-11https://research.dvidia.org/papers/piper-grip-validation-and-recovery/2026-10-11https://research.dvidia.org/papers/native-grasp-task-benchmark/2026-10-11https://research.dvidia.org/papers/hardware-matched-robot-simulation/2026-10-11https://research.dvidia.org/papers/dvidia-observation-lab/2026-10-11https://research.dvidia.org/papers/robotics-research-pipeline/2026-10-09https://research.dvidia.org/papers/skillspace-footage-training/2026-10-08https://research.dvidia.org/papers/skill-capsule-acquisition/2026-10-07https://research.dvidia.org/papers/skillspace-arm-benchmarks/2026-10-07https://research.dvidia.org/papers/skillspace-arm-installation/2026-10-07https://research.dvidia.org/papers/offline-cable-training-environment/2026-10-07https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/2026-10-07https://research.dvidia.org/papers/grok-robotics-research-on-x/2026-10-07https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/2026-10-07https://research.dvidia.org/papers/clef-and-grounded-video-observations/2026-10-06https://research.dvidia.org/papers/from-skillspace-to-robot-skill/2026-10-06https://research.dvidia.org/papers/physical-skill-research-agenda/2026-10-06https://research.dvidia.org/topics/capture-quality/2026-10-11https://research.dvidia.org/topics/skill-relations/2026-10-19https://research.dvidia.org/topics/evaluation-provenance/2026-10-19https://research.dvidia.org/topics/physical-grounding/2026-10-19 diff --git a/docs/topics/capture-quality/index.html b/docs/topics/capture-quality/index.html index fd5e092..64d691b 100644 --- a/docs/topics/capture-quality/index.html +++ b/docs/topics/capture-quality/index.html @@ -1,7 +1,7 @@ Capture quality · DVIDIA Research -

Research question

Capture quality

DVIDIA · Visible evidence

Protocols and progress are scoped below

Question

Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.

Question: Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.

+

Research question

Capture quality

DVIDIA · Visible evidence

Protocols and progress are scoped below

Question

Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.

Question: Test whether hands, relevant objects and state changes are actually observable in a POV demonstration.

Status: scoping · Updated 6 October 2026 · Results: not run

Open a research question · Baseline protocol · Bibliography

Primary sources

diff --git a/docs/topics/evaluation-provenance/index.html b/docs/topics/evaluation-provenance/index.html index 0ea9f21..2195cff 100644 --- a/docs/topics/evaluation-provenance/index.html +++ b/docs/topics/evaluation-provenance/index.html @@ -1,7 +1,7 @@ Evaluation and provenance · DVIDIA Research -

Research question

Evaluation and provenance

DVIDIA · Reproducible evidence

Protocols and progress are scoped below

Question

Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.

Question: Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.

+

Research question

Evaluation and provenance

DVIDIA · Reproducible evidence

Protocols and progress are scoped below

Question

Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.

Question: Test episode readback, original-to-derivative lineage and split integrity using small, auditable fixtures.

Status: evaluating · Updated 6 October 2026 · Results: image feasibility run complete; reference accuracy and provenance baselines not run

Open a research question · Baseline protocol · Bibliography

Primary sources

diff --git a/docs/topics/physical-grounding/index.html b/docs/topics/physical-grounding/index.html index 8ccea47..8d56d75 100644 --- a/docs/topics/physical-grounding/index.html +++ b/docs/topics/physical-grounding/index.html @@ -1,7 +1,7 @@ Physical grounding · DVIDIA Research -

Research question

Physical grounding

DVIDIA · Signals beyond appearance

Protocols and progress are scoped below

Question

Measure whether state/contact evidence and controlled synthetic augmentation improve a defined physical-skill task.

Live PiPER shoe workstation — 13 October 2026

+

Research question

Physical grounding

DVIDIA · Signals beyond appearance

Protocols and progress are scoped below

Question

Measure whether state/contact evidence and controlled synthetic augmentation improve a defined physical-skill task.

Live PiPER shoe workstation — 13 October 2026

Live PiPER shoe workstation v0.1 adds browser-local MuJoCo physics, a fixed-orientation target controller, native jaw commands and a free rigid trainer on a physical worktop. Users can move a diff --git a/docs/topics/skill-relations/index.html b/docs/topics/skill-relations/index.html index de2ca7e..f281b96 100644 --- a/docs/topics/skill-relations/index.html +++ b/docs/topics/skill-relations/index.html @@ -1,7 +1,7 @@ Skill relations · DVIDIA Research -

Research question

Skill relations

DVIDIA · Events with evidence

Protocols and progress are scoped below

Question

Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.

Question: Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.

+

Research question

Skill relations

DVIDIA · Events with evidence

Protocols and progress are scoped below

Question

Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.

Question: Calibrate hand–object relations, then test temporal event graphs whose claims point back to video intervals.

Status: scoping · Updated 6 October 2026 · Results: not run

Open a research question · Baseline protocol · Bibliography

Primary sources

diff --git a/publications.json b/publications.json index 91ae587..d7adc71 100644 --- a/publications.json +++ b/publications.json @@ -5,13 +5,26 @@ "follow": "https://x.com/iammrriver", "contributor": null, "publications": [ + { + "slug": "sensor-corroboration-v04", + "title": "Independent camera and actuator corroboration v0.4", + "source": "reports/Sensor corroboration v04.md", + "date": "2026-10-19", + "type": "Paired synthetic benchmark for camera and drive evidence", + "status": "272 cases · 69/112 false completions prevented · 43 remain · 73/80 true completions abstained", + "topics": [ + "physical-grounding", + "evaluation-provenance" + ], + "abstract": "A fresh eight-layout PiPER simulation benchmark adds separately processed RGB and drive checks to a local physical-goal verifier. The checks prevent 69 of 112 false completion readings while leaving 43 replay, identity, orientation, support and sampling blind spots; the conservative gate abstains on 73 of 80 baseline true completions, including all 64 deliberate sensor-fault controls. The measured CPU path is 22.1 ms median and 31.5 ms p95, excluding 200 ms acquisition. All raw channels and results are preserved; this is synthetic evidence, not hardware validation." + }, { "slug": "local-goal-verifier-v03", "title": "A local verifier for physical completion", "source": "reports/Local goal verifier v03.md", "date": "2026-10-19", "type": "Local CPU verifier with native, evidence-fault and forgery diagnostics", - "status": "Native: 16/16 true complete, 0/48 false complete \u00b7 forged readings: 8/8 false complete", + "status": "Native: 16/16 true complete, 0/48 false complete · forged readings: 8/8 false complete", "topics": [ "physical-grounding", "evaluation-provenance", @@ -25,7 +38,7 @@ "source": "reports/Image goal Jev measured v02.md", "date": "2026-10-19", "type": "Frozen hosted-model evaluation with native physical references", - "status": "64/64 valid responses \u00b7 telemetry: 0/24 false complete, 1/8 true complete", + "status": "64/64 valid responses · telemetry: 0/24 false complete, 1/8 true complete", "topics": [ "physical-grounding", "evaluation-provenance", @@ -39,7 +52,7 @@ "source": "reports/Image goal Jev v01.md", "date": "2026-10-18", "type": "Native diagnostic benchmark and frozen hosted-model protocol", - "status": "32 native fixtures \u00b7 4 identical-image pairs \u00b7 64 Jev trials not run", + "status": "32 native fixtures · 4 identical-image pairs · 64 Jev trials not run", "topics": [ "physical-grounding", "evaluation-provenance", @@ -53,7 +66,7 @@ "source": "reports/Goal images and physical feedback.md", "date": "2026-10-18", "type": "Research direction and passive native contact audit", - "status": "14/14 exact passive replays \u00b7 249,502 native ticks \u00b7 visual models unbenchmarked", + "status": "14/14 exact passive replays · 249,502 native ticks · visual models unbenchmarked", "topics": [ "physical-grounding", "evaluation-provenance", @@ -67,7 +80,7 @@ "source": "reports/Shoe graspability v03.md", "date": "2026-10-14", "type": "Frozen shape, load and attached-tool diagnostic", - "status": "24/24 valid \u00b7 30 g: 8/12 lifts \u00b7 300 g: 0/12 \u00b7 no net gain from removing rake", + "status": "24/24 valid · 30 g: 8/12 lifts · 300 g: 0/12 · no net gain from removing rake", "topics": [ "physical-grounding", "evaluation-provenance", @@ -81,13 +94,13 @@ "source": "reports/Continuous shoe grip v02.md", "date": "2026-10-13", "type": "Frozen native comparison with independent contact audit", - "status": "20/20 valid \u00b7 2/10 completed under both policies \u00b7 9 contact-loss alarms verified", + "status": "20/20 valid · 2/10 completed under both policies · 9 contact-loss alarms verified", "topics": [ "physical-grounding", "evaluation-provenance", "skill-relations" ], - "abstract": "A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20\u201352 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification." + "abstract": "A frozen twenty-attempt native MuJoCo comparison adds continuous acquisition checks and bounded lowering to the mounted-rake PiPER controller. Both policies complete the same 2/10 fresh cases: no gains or losses. Nine bilateral-contact-loss alarms follow a trace-verified native origin by 20–52 ms, median 30 ms; three acquisition failures are separate. Twelve supported lowering responses produce no additional completion. A non-finger-support gap remains unmonitored. Complete recorded trajectories, independent audit and development failures are public. Authored policies, synthetic contact and privileged observations establish no learned skill or physical robot qualification." }, { "slug": "skill-prerequisites-v01", @@ -95,7 +108,7 @@ "source": "reports/Skill prerequisites v01.md", "date": "2026-10-13", "type": "Two native environments with authored policies and recorded benchmark evidence", - "status": "16/16 valid \u00b7 2 lid contacts avoided \u00b7 3/4 retrieval checkpoints \u00b7 0/4 retained tool lifts", + "status": "16/16 valid · 2 lid contacts avoided · 3/4 retrieval checkpoints · 0/4 retained tool lifts", "topics": [ "physical-grounding", "evaluation-provenance", @@ -109,7 +122,7 @@ "source": "reports/Live PiPER shoe workstation v01.md", "date": "2026-10-13", "type": "Live browser-native simulation with authored control and separate benchmark evidence", - "status": "Live browser physics \u00b7 14/16 targets \u00b7 strict contact parity failed \u00b7 shoe lifting unqualified", + "status": "Live browser physics · 14/16 targets · strict contact parity failed · shoe lifting unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -123,7 +136,7 @@ "source": "reports/Shoe asset library v01.md", "date": "2026-10-12", "type": "Versioned native scene assets and frozen passive-contact benchmark", - "status": "Six shoe proxies \u00b7 three arms \u00b7 support 27/27, disabled table 0/27 \u00b7 manipulation unqualified", + "status": "Six shoe proxies · three arms · support 27/27, disabled table 0/27 · manipulation unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -137,7 +150,7 @@ "source": "reports/PiPER reliability v05.md", "date": "2026-10-12", "type": "Frozen paired reliability experiment with complete native evidence", - "status": "23/36 revised grasp \u00b7 15/36 v0.4 \u00b7 8 gained, none lost \u00b7 hardware unqualified", + "status": "23/36 revised grasp · 15/36 v0.4 · 8 gained, none lost · hardware unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -151,13 +164,13 @@ "source": "reports/PiPER contact recovery v04.md", "date": "2026-10-11", "type": "Paired carry-recovery and supported-acquisition experiment with complete confirmation receipt", - "status": "All modes 4/12 \u00b7 two resumed tasks faster \u00b7 controls 0/48 \u00b7 hardware unqualified", + "status": "All modes 4/12 · two resumed tasks faster · controls 0/48 · hardware unqualified", "topics": [ "physical-grounding", "evaluation-provenance", "skill-relations" ], - "abstract": "A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88\u20132.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification." + "abstract": "A fresh frozen 84-attempt PiPER comparison completes 4/12 placements for v0.3 history, v0.4 carry-only and v0.4 full; all 48 matched open-jaw and hold controls fail. Two completed post-hold placements finish 2.440 and 2.640 simulated seconds sooner with unchanged 220/240 ms settling holds. Four low-friction failures reach refusal 2.88–2.90 seconds sooner without becoming successes. Fresh nominal-friction 60 g layouts expose two held-but-too-low resume failures in every mode; neither reaches speed restoration. Six earlier acquisition failures are traced to insufficient support under the pinned position servo and side pinch, not the actuator force cap. All native mechanics and outcome criteria remain unchanged. Complete trajectories and 84,000 raw computation windows are retained; none exceeds 20 ms on this host. Source-snapshotted development remains separate. Privileged authored simulation establishes no learned skill, true-slip accuracy or physical qualification." }, { "slug": "piper-contact-history-v03", @@ -165,7 +178,7 @@ "source": "reports/PiPER contact history v03.md", "date": "2026-10-11", "type": "Paired probe-only and contact-history experiment with complete confirmation receipt", - "status": "History 6/12 \u00b7 baseline and probe-only 4/12 \u00b7 controls 0/48 \u00b7 hardware unqualified", + "status": "History 6/12 · baseline and probe-only 4/12 · controls 0/48 · hardware unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -179,7 +192,7 @@ "source": "reports/PiPER grip validation and recovery.md", "date": "2026-10-11", "type": "Paired authored controller experiment and complete confirmation receipt", - "status": "Moderate-friction regression \u00b7 baseline 4/6, guarded 2/6 \u00b7 hardware unqualified", + "status": "Moderate-friction regression · baseline 4/6, guarded 2/6 · hardware unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -193,7 +206,7 @@ "source": "reports/Native grasp task benchmark.md", "date": "2026-10-11", "type": "Native contact task implementation and authored baseline", - "status": "6/6 simulated placements \u00b7 controls 0/12 \u00b7 real-time and hardware unqualified", + "status": "6/6 simulated placements · controls 0/12 · real-time and hardware unqualified", "topics": [ "physical-grounding", "evaluation-provenance", @@ -207,13 +220,13 @@ "source": "reports/Hardware matched robot simulation.md", "date": "2026-10-11", "type": "Pinned mechanics implementation and CPU numerical benchmark", - "status": "Pinned CPU mechanics \u00b7 PiPER contact pilot \u00b7 physical fidelity unmeasured", + "status": "Pinned CPU mechanics · PiPER contact pilot · physical fidelity unmeasured", "topics": [ "physical-grounding", "evaluation-provenance", "skill-relations" ], - "abstract": "A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80\u201347.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming." + "abstract": "A separate UR5e/Robotiq reference preserves pinned geometry, inertias, moving-link collision proxies and coupled gripper structure: fourteen modeled coordinates and seven commands. Eighteen deterministic unloaded motion/closure runs are numerically valid; at a 1 ms step, CPU physics throughput measures 40.80–47.29 simulated seconds per measured stepping wall second on one host, excluding loading, sensing, rendering, policy and real actuation. Adjacent timestep comparisons and a matched static overlap expose numerical sensitivity and collision participation. A pure-Python paired trace scorer measures joint, tool and jaw mismatch without automatic qualification. Left/right five-finger references are separately audited; LeRobot collection with an original DVIDIA policy is proposed. Serial calibration, physical response, shoe task learning and bimanual execution remain unmeasured. An offline 3D mesh replay now exposes the recorded arm/gripper poses without rerunning physics. PiPER and SO-101 now have pinned CPU profiles with 6/6 valid unloaded numerical attempts each and explicit native command units. PiPER also has a frozen authored rigid-block contact pilot: nominal placement 2/2, low-friction placement 0/1, open-jaw 0/3 and hold 0/3, with all nine attempts numerically valid. One of 9,000 computation windows exceeds 20 ms; hardware real-time response is unqualified. SO-101 object tasks, learned policies and physical qualification remain open. YAM, OpenArm, ARX X5 and PAROL6 remain upcoming." }, { "slug": "dvidia-observation-lab", @@ -221,7 +234,7 @@ "source": "reports/DVIDIA Observation Lab pilot.md", "date": "2026-10-11", "type": "Local observation workflow and engineering feasibility pilot", - "status": "Local review release \u00b7 motion and actuator source pilots \u00b7 hardware unqualified", + "status": "Local review release · motion and actuator source pilots · hardware unqualified", "topics": [ "capture-quality", "evaluation-provenance", @@ -235,7 +248,7 @@ "source": "reports/Robotics research pipeline.md", "date": "2026-10-09", "type": "Architecture proposal and staged validation roadmap", - "status": "Architecture proposal \u00b7 current simulation alpha \u00b7 hardware unqualified", + "status": "Architecture proposal · current simulation alpha · hardware unqualified", "topics": [ "capture-quality", "skill-relations", @@ -250,7 +263,7 @@ "source": "reports/Skillspace footage training pipeline.md", "date": "2026-10-08", "type": "Product implementation and measured synthetic pilot", - "status": "Public CPU package \u00b7 one native scene qualified \u00b7 simulation-only", + "status": "Public CPU package · one native scene qualified · simulation-only", "topics": [ "capture-quality", "physical-grounding", @@ -264,7 +277,7 @@ "source": "reports/Skill capsule acquisition and optimization.md", "date": "2026-10-07", "type": "Learned movement artifact and installation prototype", - "status": "Simulation-only \u00b7 teacher/student 7/8 \u00b7 no overall speedup", + "status": "Simulation-only · teacher/student 7/8 · no overall speedup", "topics": [ "physical-grounding", "skill-relations", @@ -278,7 +291,7 @@ "source": "reports/Skillspace arm benchmarks and evolution.md", "date": "2026-10-07", "type": "Paired simulation benchmark and product assessment", - "status": "Mixed held-out result \u00b7 30/32 baseline, 29/32 updated", + "status": "Mixed held-out result · 30/32 baseline, 29/32 updated", "topics": [ "physical-grounding", "skill-relations", @@ -292,7 +305,7 @@ "source": "reports/Installing a Skillspace on a simulated arm.md", "date": "2026-10-07", "type": "Product prototype and native contact qualification", - "status": "Public skill link to authored adapter \u00b7 6/6 scene placements", + "status": "Public skill link to authored adapter · 6/6 scene placements", "topics": [ "physical-grounding", "skill-relations", @@ -306,7 +319,7 @@ "source": "reports/Offline cable training environment.md", "date": "2026-10-07", "type": "Product prototype and local qualification", - "status": "CPU simulation-only \u00b7 Controller search and frozen evaluation executed", + "status": "CPU simulation-only · Controller search and frozen evaluation executed", "topics": [ "physical-grounding", "evaluation-provenance" @@ -319,12 +332,12 @@ "source": "reports/Fast accurate robot training simulation.md", "date": "2026-10-07", "type": "Decision report and local experiment", - "status": "54 CPU fixture trials \u00b7 GPU and physical transfer untested", + "status": "54 CPU fixture trials · GPU and physical transfer untested", "topics": [ "physical-grounding", "evaluation-provenance" ], - "abstract": "A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance\u2013accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured." + "abstract": "A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured." }, { "slug": "grok-robotics-research-on-x", @@ -332,7 +345,7 @@ "source": "reports/Grok robotics research on X.md", "date": "2026-10-07", "type": "Source review and thesis comparison", - "status": "Primary sources checked \u00b7 Experiments proposed", + "status": "Primary sources checked · Experiments proposed", "topics": [ "skill-relations", "physical-grounding" @@ -345,7 +358,7 @@ "source": "reports/Spatially grounded downloadable robot skills.md", "date": "2026-10-07", "type": "Research synthesis and proposed experiments", - "status": "Source review \u00b7 DVIDIA experiments not run", + "status": "Source review · DVIDIA experiments not run", "topics": [ "skill-relations", "physical-grounding", @@ -359,7 +372,7 @@ "source": "reports/Clef and grounded video observations.md", "date": "2026-10-06", "type": "Engineering feasibility pilot", - "status": "Five licensed excerpts \u00b7 Accuracy evaluation outstanding", + "status": "Five licensed excerpts · Accuracy evaluation outstanding", "topics": [ "capture-quality", "evaluation-provenance", @@ -378,7 +391,7 @@ "evaluation-provenance", "skill-relations" ], - "abstract": "An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot\u2019s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun." + "abstract": "An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun." }, { "slug": "physical-skill-research-agenda", diff --git a/reports/Sensor corroboration v04.md b/reports/Sensor corroboration v04.md new file mode 100644 index 0000000..d0a10ba --- /dev/null +++ b/reports/Sensor corroboration v04.md @@ -0,0 +1,62 @@ +# Independent camera and actuator corroboration v0.4 + +DVIDIA robotics research · 19 October 2026. + +The [local physical-goal verifier v0.3](../reports/Local%20goal%20verifier%20v03.md) trusts its observation packet. This experiment adds separately processed RGB and PiPER drive channels, then asks whether those channels can contradict a primary “complete” reading. It is a simulator study of evidence fusion, not a hardware safety claim. + +The frozen benchmark contains **272 cases across eight layouts**: 96 native scenes, 80 primary-packet forgeries, 64 sensor faults and 32 cross-channel controls. All 96 native cases generated validly. The v0.3 baseline called 192 cases complete, including 112 physical false completions. The v0.4 gate retained 50 complete verdicts: seven physical positives and 43 false completions. It prevented 69/112 baseline false completions, but 43 remained. It also abstained on 73/80 baseline true completions; 64 had deliberately missing, stale, gapped or malformed auxiliary evidence, and nine were physically complete but visually unobservable. + +The result is useful because it shows both what a second signal catches and what two weak signals cannot establish. The camera detects large motion, position disagreement and many target-visibility failures. The drive path checks finger aperture, command, motor effort, settling and servo consistency. Neither authenticates its source. These channels still miss a 180° orientation reversal, a green lookalike, elevated support, motion that aliases between samples, and refreshed or fully replayed channels. + +## Paired outcome + +| Measure | v0.3 primary packet | v0.4 camera + drive gate | +| --- | ---: | ---: | +| Scheduled cases | 272 | 272 | +| Complete verdicts | 192 | 50 | +| Physically true completions recognized | 80/80 | 7/80 | +| False completions among physical negatives | 112/192 | 43/192 | +| Baseline false completions prevented | — | 69/112 | +| Baseline true completions lost | — | 73/80 | +| Unknown verdicts | 0/272 | 142/272 | +| Invalid native generation | 0 | 0 | + +The 64 sensor-fault controls remain physically complete; all 64 return `unknown` when their required camera or drive evidence is missing or invalid. In the native cohort, seven of eight visible settled scenes pass. Every explicitly occluded scene abstains, and one nominally visible shoe is obscured by the simulated arm. These are conservative refusals, not evidence that the physical task failed. + +Among the 112 baseline false completions, the gate prevents 69 and leaves 43. The residual cases are not one generic detector error: + +| Residual control | False completions that pass | +| --- | ---: | +| Instantaneous motion alias | 7/8 | +| 180° yaw flip | 8/8 | +| Visually identical green lookalike | 7/8 | +| Elevated object with hidden support surface | 7/8 | +| Refreshed camera replay | 7/8 | +| Coherent replay of all channels | 7/8 | + +The layout with an occluded camera path abstains, which is why several controls show seven rather than eight false completions. A coherent replay remains compatible with the channels' declared clock labels; there is no cryptographic sensor provenance in this simulator. + +## What the channels measure + +The camera observer receives only eleven 640×480 RGB frames at 20 ms intervals. A fixed orthographic projection maps a green elongated proxy to XY position and an undirected axis. It receives no object ID, simulator pose, segmentation mask or physical truth. Its color-and-shape rule is not a general shoe detector. Axial geometry cannot distinguish the toe from a 180°-reversed shoe, a same-color decoy is ambiguous, and RGB alone cannot measure height above the table. + +The drive observer receives one hundred PiPER joint/drive samples at 2 ms intervals: six arm channels, two signed finger positions and velocities, gripper command and simulated motor effort. It validates ranges, units, finger symmetry and a declared spring/servo consistency relation, then checks that the fingers are open, commanded open, settled and at low effort. Motor effort is not object contact force. Neither channel can verify table support or rule out contact from every arm link. + +Both channels originate in the same MuJoCo model as the primary packet. They use separate processing code, but are **not independent physical sensors**. Calibration bounds and IDs are synthetic declarations, not authenticated measurements. An unknown result means the verifier refuses to confirm; it does not mean the scene is physically incomplete. + +## Compute and limits + +On the recorded Apple Silicon host, the complete processed decision path had **22.1 ms median and 31.5 ms p95** across 816 repeated case evaluations. RGB extraction alone had 14.3 ms median and 20.1 ms p95; the baseline verifier had 0.42 ms median. The optional OpenCV CPU kernel was installed for this run. No GPU or hosted model call was used. + +The processed-path timer includes local JSON reads, PNG decoding, primary verification, RGB observation, drive observation and fusion. It excludes the 200 ms observation history, native rendering/acquisition, networking and robot actuation. The p95 is longer than a 20 ms camera frame interval, so this is not a demonstrated real-time control loop or hard-real-time bound. The observer uses an optional CPU package for speed and keeps a NumPy-only fallback; no minimum hardware specification has been established. + +The next useful measurement is a hardware-in-the-loop capture with timestamped camera, encoders, motor current and a second view that can see object height and support. Before that, the simulator needs stronger toe-direction and identity cues, explicit bounded-motion claims, and anti-replay provenance. Training a decision model on these labels would not remove those missing observables. + +## Reproduce and inspect + +- [Interactive case viewer and downloadable inputs](https://dvidia-robot-simulation-lab.static.hf.space/image-goals/v04/index.html) +- [Pinned implementation](https://github.com/Dvidia-Inference/dvidia-training/tree/90e84701ce7d5bc330ef41b02793e2ce037d702d) +- [Full run directory and scored results](https://github.com/Dvidia-Inference/dvidia-training/tree/main/runs/sensor-goal-verifier-v04-final) +- [Protocol and reproduction commands](https://github.com/Dvidia-Inference/dvidia-training/blob/main/docs/sensor-goal-verifier.md) + +Source commit `90e84701ce7d5bc330ef41b02793e2ce037d702d` freezes the final seed, eight layouts, 272-case inventory, and three timing repetitions. Every scheduled case and every sensor stream remains in the release. No provider, Jev API, physical robot or learned controller was used. Original DVIDIA code is MIT; the pinned simulator and robot assets retain their upstream terms.