Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ All four covers are **AI-generated editorial illustrations**, not footage, measu

[DVIDIA Observation Lab pilot](reports/DVIDIA%20Observation%20Lab%20pilot.md) publishes our local video detection and review workflow: sampled object/palm proposals, editable timeline revisions and JSON export. Each recorded engineering run completed six public excerpts (19 seconds and 38 sampled frames); independent annotation accuracy remains unmeasured. [Read the paper](https://research.dvidia.org/papers/dvidia-observation-lab/), [install the versioned open-source release](https://github.com/Dvidia-Inference/dvidia-training/releases/tag/observation-lab-v0.1.0), or [try the synthetic browser review demo](https://huggingface.co/spaces/Dvidia/observation-lab). The hosted demo uses authored clips and boxes; real detection runs locally with separately provisioned upstream models. No action captions, validated depth or robot skill are established.

The [image-motion source alpha](https://github.com/Dvidia-Inference/dvidia-training/pull/7) adds manually seeded causal image tracking, pixel speed, short-horizon forecasts and local replay. Its six authored cases cover 720 frames and process in 1.454 seconds for 24 source seconds on one CPU run; occlusion/low-texture losses and an 89.975 px worst 500 ms reversal forecast remain explicit. [Exact source, benchmark and installation](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md). The v0.1 release/Space remain separate; real-scene accuracy, metric speed, live robot latency and the full v0.2 are not established.

[Robotics research pipeline](reports/Robotics%20research%20pipeline.md) maps the path from Skillspace originals and synchronized robot measurements to reviewed episodes, local training, frozen simulation evaluation and compatible skill installation. The diagram distinguishes implemented simulation components from proposed automatic labeling and a learned task supervisor. A new annotation-labor pilot and feedback-supervisor experiment are specified; neither has been run, and physical robot qualification remains open. [Read the public architecture](https://research.dvidia.org/papers/robotics-research-pipeline/).

[Skillspace footage training pipeline](reports/Skillspace%20footage%20training%20pipeline.md) introduces the standalone [DVIDIA Training repository](https://github.com/Dvidia-Inference/dvidia-training): audit actual footage, freeze connected source groups, fit visual weights locally and export inspectable results. Its optional movement branch requires aligned numerical actions and a supported simulation adapter. The separate [frozen sample-count benchmark](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/README.md) compares two, four and seven training groups on the same twelve layouts: nominal completion is 8/8, 7/8 and 8/8; actuator-stress completion is 3/4 for each, and all 36 matched open-jaw controls fail. This single composition-order pilot establishes no sufficient video count, human-video action bridge or hardware skill. [Inspect the protocol, outcomes and recorded evidence release](https://github.com/Dvidia-Inference/dvidia-training/releases/tag/benchmark-2026-10-08).
Expand Down
20 changes: 20 additions & 0 deletions docs/downloads/reports/DVIDIA Observation Lab pilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,23 @@ This lab is upstream of robot learning. Image rectangles supply no calibrated XY
The public [Hugging Face Observation Lab Space](https://huggingface.co/spaces/Dvidia/observation-lab) reuses our review interface with two authored geometric videos and authored image boxes. It is a synthetic workflow demonstration: no detector, VLM or robot policy runs in the hosted page. Revisions stay in each visitor's browser, with a visible temporary-memory fallback if browser storage or cross-tab locking is unavailable. Visitors can edit, save and export their own review without server writes. The actual detector pipeline is installed and run locally on permitted footage. All sixteen hosted files were read anonymously at [Space commit 8547b78](https://huggingface.co/spaces/Dvidia/observation-lab/tree/8547b78f312e39d9a0b299ebe055c401f8135848) and compared with the staged hashes. The public runtime was observed running with its synthetic scope labels visible.

Publication preparation reran 311 repository tests and separately checked 22 publisher tests, six synthetic preparation/privacy/media tests and browser-local revision/conflict behavior. Both pinned ONNX detectors loaded and ran on a neutral image in the local CPU runtime; that smoke test measures neither detection accuracy nor task understanding. [Release source](https://github.com/Dvidia-Inference/dvidia-training/tree/3cc4a3b03ca6d61479c9f6a86cebf7ccc77fd117) preserves the original pilot receipts and clarifies the requested-versus-reported thread discrepancy. Original code uses MIT and the detector adapter retains its Apache-2.0 notices. Production uploads, accounts, storage and application analysis remain separate from this research release.

## Image-motion source alpha estimates speed and future paths

On 10 October 2026, the next source alpha adds causal image motion and passive forecasting to our local research tools. [Candidate pull request 7](https://github.com/Dvidia-Inference/dvidia-training/pull/7) and [exact source commit ade2b54](https://github.com/Dvidia-Inference/dvidia-training/tree/ade2b541d8ca4a785bbd1effdfd33eb2a306f150) identify this addition. The published `observation-lab-v0.1.0` release and Hugging Face review demo remain separate; neither includes these motion capabilities. The complete Observation Lab v0.2 remains planned. ([Installation and motion contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md), [next-version plan](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/observation-v0.2-plan.md))

The user supplies one first-frame image region and label. Sparse CPU optical flow translates its initial center using robust feature displacement; two contiguous observations estimate velocity, and constant-velocity forecasts project that image anchor 100, 250 and 500 ms ahead. Integer input presentation timestamps multiplied by the declared rational time base preserve the source clock. Initial labels and regions are human-authored, not detector outputs; no new weights are trained or downloaded. Position and speed are analysis-image pixels and pixels/second. The anchor is not a physical centroid, and this fixed-camera baseline has no calibrated metric speed, contact model or robot command authority. A lost track remains lost without interpolation or reacquisition. ([Exact implementation contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md))

The frozen authored benchmark includes stationary motion, constant velocity, stop/start, direction change, brief occlusion and low texture. Each generated clip is four seconds at 640 × 360 and 30 fps; the batch contains **six cases, 720 attempted frames and 24 source seconds**. Future authored reference trajectories enter only the scorer, after analysis; they do not enter earlier tracking or forecasts. These six cases share one generator and texture rather than independent real-world task attempts. ([Frozen protocol](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/protocol.json), [aggregate scores and denominators](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/result.json))

| Recorded result | Scope |
| --- | --- |
| Six of six pipeline cases completed | Engineering processing, not six successful physical tasks |
| 1.454 seconds processing for 24 source seconds | One offline CPU batch; excludes 1.514 seconds of fixture generation and provisioning |
| Occlusion: 45/100 visible reference frames tracked | Loss remains visible after reappearance; no hidden reacquisition |
| Low texture: 0/120 visible reference frames tracked | All rows abstain; missing predictions are not zero-error answers |
| Direction change: 89.975 px worst endpoint error at 500 ms | A failure tail despite small anchor error and a lower whole-case forecast mean |

The batch ran once on macOS ARM64 with Python 3.12.9, OpenCV 5.0.0 and NumPy 2.5.3. Processing covers intake, probing, streaming decode, tracking/forecast, writes and scoring; the offline processing/source-duration ratio is 0.0606. One OpenCV thread was requested while its runtime reported ten; actual concurrency and an enforced thread limit remain unmeasured. Fast replay does not establish live capture-to-command delay or a reliable 30 Hz robot loop. The 250 ms forecast falls between source frames and uses a disclosed nearest-frame reference, so it is not an exact-time accuracy result. Heuristic uncertainty envelopes remain uncalibrated. ([Measured run and limitations](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/README.md))

Local validation recorded **378 repository tests and six browser workflow checks** for this candidate. These are software/interface checks; no CI completion count, independent real-motion accuracy, live sensing or physical control result follows from them. Original DVIDIA motion, fixture and scoring code uses MIT, while existing upstream detector notices retain their separate terms. Next evidence is independently reviewed permitted motion footage, measured observation-only live streaming and calibrated geometry, followed by synchronized robot actions/state for interaction-conditioned prediction. ([Reviewable candidate and validation](https://github.com/Dvidia-Inference/dvidia-training/pull/7), [motion and interaction gates](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-prediction-plan.md))
Loading
Loading