diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0210f7f..8e151b5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -63,9 +63,52 @@ jobs:
dvidia-episode-tape --help
dvidia-reaction --help
dvidia-motor-memory --help
+ dvidia-observe --help
+ dvidia-observation-studio --help
dvidia-training-doctor --json
- name: Test footage, visual training and loopback studio
- run: python -m unittest tests.test_footage_data tests.test_footage_train tests.test_footage_pipeline tests.test_footage_benchmark tests.test_training_studio tests.test_doctor tests.test_coverage tests.test_episode_tape tests.test_reaction tests.test_reaction_metrics tests.test_reaction_convergence tests.test_reaction_benchmark tests.test_motor_memory tests.test_motor_metrics tests.test_motor_benchmark -v
+ run: python -m unittest tests.test_footage_data tests.test_footage_train tests.test_footage_pipeline tests.test_footage_benchmark tests.test_training_studio tests.test_doctor tests.test_coverage tests.test_episode_tape tests.test_reaction tests.test_reaction_metrics tests.test_reaction_convergence tests.test_reaction_benchmark tests.test_motor_memory tests.test_motor_metrics tests.test_motor_benchmark tests.test_observations tests.test_observation_models -v
+
+ observations:
+ name: Observation detectors / Linux / Python 3.12
+ runs-on: ubuntu-latest
+ timeout-minutes: 15
+ steps:
+ - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
+ with:
+ persist-credentials: false
+ - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
+ with:
+ python-version: "3.12"
+ - name: Install observation runtime
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y ffmpeg
+ python -m pip install ".[observations]"
+ python -m pip check
+ - name: Verify review contracts and explicitly provision pinned detectors
+ run: |
+ python -m unittest tests.test_observations tests.test_observation_models -v
+ python -m unittest discover -s publishing/huggingface -p 'test_publish_observation.py' -v
+ python -m unittest discover -s publishing/huggingface/observation-lab -p 'test_*.py' -v
+ node publishing/huggingface/observation-lab/test_adapter.cjs
+ python scripts/download_observation_models.py --download --output runs/observation-models-ci
+ - name: Smoke test actual ONNX loading and inference
+ run: |
+ python - <<'PY'
+ import json, tempfile
+ from pathlib import Path
+ import cv2, numpy as np
+ from dvidia_training.observation_models import OpenCVDetector
+ detector = OpenCVDetector('runs/observation-models-ci')
+ with tempfile.TemporaryDirectory() as directory:
+ frame = Path(directory) / 'neutral.png'
+ assert cv2.imwrite(str(frame), np.full((240, 320, 3), 240, np.uint8))
+ rows = detector.detect(frame)
+ assert isinstance(rows, list)
+ print(json.dumps({'scope': 'neutral-image model-loading smoke test; no accuracy measurement',
+ 'proposals': len(rows), 'runtime': detector.metadata}))
+ PY
arm:
name: Optional arm / Linux / Python 3.12
diff --git a/README.md b/README.md
index 7e2635f..3472d63 100644
--- a/README.md
+++ b/README.md
@@ -40,6 +40,24 @@ Open [the collection planner](http://127.0.0.1:8270/coverage) in the same local
The [episode tape recorder](docs/episode-tape.md) preserves the existing simulated arm's measured state, requested/applied commands, contacts and sensor ages. Recording needs the optional arm extra; validation and inspection need only Python's standard library. These tools are available from the current checkout and do not train a new policy or change the older Hugging Face pilot.
+## Review what is visible in a video
+
+The new [local observation lab](docs/observations.md) uses optional CPU object and
+palm detectors to propose timestamped image observations. Open the original video,
+inspect its boxes and timeline, correct labels, and save a separate review revision.
+Raw evidence stays unchanged. No account, cloud inference, or GPU is needed.
+
+[Observation Lab release](https://github.com/Dvidia-Inference/dvidia-training/releases/tag/observation-lab-v0.1.0) · [Research paper](https://research.dvidia.org/papers/dvidia-observation-lab/) · [Synthetic review demo](https://huggingface.co/spaces/Dvidia/observation-lab)
+
+Use the `observation-lab-v0.1.0` tag for this feature. The hosted demo uses
+authored synthetic clips and keeps edits in your browser; actual detection runs
+locally with the optional observation runtime and explicitly provisioned models.
+
+The [first feasibility run](benchmarks/observation-20261010/README.md) completed
+six public excerpts. This establishes the integration, not accuracy: labels can
+be wrong, outcomes remain unknown, and there is no 3D reconstruction, automatic
+action understanding, or human-video-to-robot control bridge.
+
## Try the pipeline
Generate an explicitly synthetic example, then train and verify its output:
diff --git a/benchmarks/observation-20261010/README.md b/benchmarks/observation-20261010/README.md
new file mode 100644
index 0000000..864df1b
--- /dev/null
+++ b/benchmarks/observation-20261010/README.md
@@ -0,0 +1,103 @@
+# Local observation feasibility — 10 October 2026
+
+**Engineering integration passed on six public excerpts; accuracy not assessed.**
+This is not a cup-placement benchmark, 3D reconstruction or trained robot skill.
+The [plan](../../docs/observation-pilot-plan.md) was written before the model run.
+
+## Inputs and execution
+
+The fixed order was shirt, stir, pack, onion, melon, jeans, selected from existing
+local public examples before examining model output. Five are attributed
+EgoAnnotate excerpts; stir is a CoMind excerpt. Licenses and public source links
+are preserved in both protocol files. Exact excerpts were hash-verified. The
+full upstream originals were not downloaded or hash-verified for this pilot.
+No raw media, sampled frames or private user metadata are included in this report.
+
+Six excerpts total 19 seconds, 2,753,287 bytes and 38 sampled frames. They are not
+six independent complete attempts; unknown session grouping remains one
+conservative shared group. No train/test split or learning run took place.
+Publisher titles were never fed to the detector.
+
+| Attempt | Complete / attempted | Processing time | Failures |
+| --- | --- | ---: | ---: |
+| Initial recipe v1 | 6 / 6 | 4.835 seconds | 0 |
+| Hardened recipe v1.1 | 6 / 6 | 4.635 seconds | 0 |
+
+Timing covers intake, copy/hash, frame decoding, detection, association and
+episode writes. It excludes model provisioning/loading, final receipt writing,
+UI startup and human review. Single runs, warm machine; this is neither a p95
+latency measurement nor a production throughput guarantee. Peak RAM, energy and
+dollar cost were not measured. No paid inference API was called.
+
+Runtime: Python 3.12.9, OpenCV 5.0.0.93, NumPy 2.5.3, macOS ARM64 and one
+inference worker. Four OpenCV threads were requested, but the saved model
+metadata records `cv2.getNumThreads()` as ten after model loading. Actual peak
+thread concurrency was not measured; a four-thread cap is not established.
+Both pinned ONNX models total approximately 40 MB. OpenCV
+5 printed that target selection is unsupported by its new graph engine; this
+build uses the OpenCV CPU backend and no GPU path was requested or measured.
+
+## What changed after the first run
+
+Review identified a possible browser-clock shift for videos with nonzero start
+timestamps. Recipe v1.1 rejects nonzero container/video/decoded starts rather
+than moving boxes relative to the unchanged original. The initial artifacts are
+preserved alongside the subsequent run, which processed the same six inputs.
+All six inputs have zero starts and were accepted in both runs.
+
+The review server was also hardened against same-size media replacement,
+malformed review IDs and oversized run metadata. These are workflow fixes,
+not improvements in detector quality.
+
+## Observed limitations
+
+- The three-frame preliminary adapter check included a saucepan incorrectly
+ labeled cup and storage jars labeled cups. Hands were missed in the stir
+ sample. These are qualitative findings, not scored accuracy estimates.
+- COCO labels do not cover every task object; palm rectangles do not cover full
+ hands or prove contact. Two frames/second can miss short actions.
+- Association uses same-class image overlap. It is not optical flow and can
+ fragment or switch identities. Overlap proposals do not establish grasping.
+- Outcomes remain unknown. Model scores are uncalibrated, and the pilot creates
+ no automatic natural-language action captions or robot commands.
+- The intended permission-cleared cup-placement cohort and independent reference
+ annotations have not been collected. Accuracy, false-completion rate and
+ correction time remain unmeasured. No model ranking is established.
+
+## Evidence
+
+- `initial-protocol.json`, `initial-run.json`: original frozen inputs and receipt.
+- `hardened-protocol.json`, `hardened-run.json`: later timing policy and receipt.
+- `episode-summary.json`: durations, proposal counts, timing and excerpt hashes.
+- Local ignored run directories retain media, frames and raw episode annotations.
+ Run and source hashes bind the receipts to those local artifacts; this report
+ alone cannot reproduce a visual inspection without obtaining permitted clips.
+
+Browser interaction testing uses a separate copied run. Test review edits do not
+count as independent human annotations and do not alter the original run's
+unreviewed status.
+
+## Implementation validation
+
+The repository suite passed 307 tests, followed by 24 focused observation tests
+including four additional detector-admission/suppression tests. These exercise
+real FFmpeg decoding, timestamps, source tampering, failed attempts, malformed
+reviews, revision conflicts, local-origin enforcement and video byte ranges.
+
+Fifteen browser checks passed on an isolated review copy: playback, timestamped
+overlays, correction save/reload, preservation of raw labels, manual events,
+export, unsaved-edit protection, completion validation and 390/320px layouts.
+There were no console/page errors. See `ui-validation.json`. Headless browser
+checks are not physical-phone or Safari qualification.
+
+The wheel built successfully; its final HTML, command entry points and both
+upstream license files were verified. Raw footage/model files are Git-ignored.
+Production uploads, accounts, storage and website deployment were not modified.
+
+## Next decision
+
+Collect 30–50 complete cup-placement attempts with permission and explicit
+session/person grouping, including failures, occlusion and recovery. Establish
+blinded reference labels first. Then compare object-only, temporal caption-only,
+combined proposals and manual annotation at matched quality and total cost.
+Only after that should an app integration claim reliable task labeling.
diff --git a/benchmarks/observation-20261010/episode-summary.json b/benchmarks/observation-20261010/episode-summary.json
new file mode 100644
index 0000000..a6cf4a6
--- /dev/null
+++ b/benchmarks/observation-20261010/episode-summary.json
@@ -0,0 +1,80 @@
+[
+ {
+ "id": "shirt",
+ "duration_seconds": 3.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 6,
+ "elapsed_seconds": 0.6693273328710347,
+ "frames": 6,
+ "inference_seconds": 0.46699687489308417
+ },
+ "proposals": 7,
+ "source_sha256": "8392c62f4e9590a15745311746114c2b6355e24c60fa87fef68e834f20762113"
+ },
+ {
+ "id": "stir",
+ "duration_seconds": 4.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 8,
+ "elapsed_seconds": 1.1173061658628285,
+ "frames": 8,
+ "inference_seconds": 0.6056890829931945
+ },
+ "proposals": 25,
+ "source_sha256": "4ffe78850d4e6475546a521e0eb425f7a34c632679183cf112aacb70ec1b0861"
+ },
+ {
+ "id": "pack",
+ "duration_seconds": 3.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 6,
+ "elapsed_seconds": 0.6751292911358178,
+ "frames": 6,
+ "inference_seconds": 0.4420628750231117
+ },
+ "proposals": 7,
+ "source_sha256": "87a67e5b10f124a65df7d669086d4a9b2cb7ae25a5d758594b9b3aa7b0938f4c"
+ },
+ {
+ "id": "onion",
+ "duration_seconds": 3.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 6,
+ "elapsed_seconds": 0.7398202081676573,
+ "frames": 6,
+ "inference_seconds": 0.43661770783364773
+ },
+ "proposals": 10,
+ "source_sha256": "100e25698ee99e99dc648c41d9864144e312f0fce7c07dc9a7b06b487b9a2a5f"
+ },
+ {
+ "id": "melon",
+ "duration_seconds": 3.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 6,
+ "elapsed_seconds": 0.7281540830153972,
+ "frames": 6,
+ "inference_seconds": 0.4237409171182662
+ },
+ "proposals": 14,
+ "source_sha256": "f32cf75d28602d169b4f302830ba633a351ac890596887c54c8e928efcf2650e"
+ },
+ {
+ "id": "jeans",
+ "duration_seconds": 3.0,
+ "metrics": {
+ "accuracy_status": "not_assessed",
+ "detected_frames": 6,
+ "elapsed_seconds": 0.6997090000659227,
+ "frames": 6,
+ "inference_seconds": 0.4299595837946981
+ },
+ "proposals": 9,
+ "source_sha256": "95845362951b3f78b7b731691df6d472bf176ced4b07ecef4d99d0b8a21da3e2"
+ }
+]
diff --git a/benchmarks/observation-20261010/hardened-protocol.json b/benchmarks/observation-20261010/hardened-protocol.json
new file mode 100644
index 0000000..f424e21
--- /dev/null
+++ b/benchmarks/observation-20261010/hardened-protocol.json
@@ -0,0 +1 @@
+{"created_at":"2026-10-10T11:22:03.270961+00:00","execution":"local_offline","hardware":{"machine":"arm64","platform":"Darwin"},"kind":"dvidia.observation-protocol.v1","limits":{"cpu_threads":4,"workers":1},"manifest_sha256":"6e4e3a0b1b12eeb32cafcba0668d2614c80b022a91eafc7740018be880eddb29","model":{"backend":"opencv","coordinates":"normalized_image_xyxy","cpu_threads":10,"device":"cpu","hand_box_semantics":"Palm detector rectangle; not full hand segmentation or observed contact.","hand_detector_enabled":true,"hand_threshold":0.5,"implementation_sha256":"e1c6939779ab39a32497e00c35892a98eef59c3dd512135f1ac6cd4f897d1a22","implementation_version":1,"kind":"opencv-zoo-yolox-palm","limitations":["COCO labels can be wrong or absent, especially for small and occluded objects.","Palm boxes do not establish finger contact, grasp, force, depth, task success or robot actions.","Object identity and temporal relationships require a separate tracking stage."],"max_detections":30,"max_nms_candidates":1000,"models":[{"bytes":35858002,"directory":"object_detection_yolox","download_url":"https://media.githubusercontent.com/media/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/object_detection_yolox_2022nov.onnx","license":"Apache-2.0","license_bytes":11371,"license_sha256":"0ec3668d3274bcf29e8a29e9576d5a2cd96fc78d3c5bec4387355a796e5d9088","license_url":"https://raw.githubusercontent.com/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/LICENSE","name":"object_detection_yolox_2022nov.onnx","revision":"47534e27c9851bb1128ccc0102f1145e27f23f98","role":"objects","sha256":"c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063","source":"https://github.com/opencv/opencv_zoo/blob/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/object_detection_yolox_2022nov.onnx"},{"bytes":3905734,"directory":"palm_detection_mediapipe","download_url":"https://media.githubusercontent.com/media/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/palm_detection_mediapipe_2023feb.onnx","license":"Apache-2.0","license_bytes":11358,"license_sha256":"cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30","license_url":"https://raw.githubusercontent.com/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/LICENSE","name":"palm_detection_mediapipe_2023feb.onnx","revision":"47534e27c9851bb1128ccc0102f1145e27f23f98","role":"palms","sha256":"78ff51c38496b7fc8b8ebdb6cc8c1abb02fa6c38427c6848254cdaba57fcce7c","source":"https://github.com/opencv/opencv_zoo/blob/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/palm_detection_mediapipe_2023feb.onnx"}],"nms_iou":{"objects":0.5,"palms":0.3},"runtime":{"numpy":"2.5.3","opencv":"5.0.0"},"score_kind":"uncalibrated_model_evidence","threshold":0.35},"recipe":{"automatic_outcome":"unknown","claim_scope":"sampled_2d_observations","id":"dvidia-observations-v1.1","max_duration_seconds":60,"max_frames":120,"max_image_side":640,"max_track_gap_seconds":0.8,"minimum_association_iou":0.2,"samples_per_second":2,"timebase":"zero-origin video and container; nonzero starts rejected"},"sources":[{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"shirt","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"8392c62f4e9590a15745311746114c2b6355e24c60fa87fef68e834f20762113","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010079.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010079.mp4","title":"Starting to fold a white shirt"},{"credit":"Alexey Gavryushin, Dingxi Zhang, Zhao Huang, Alexandros Delitzas, Jiaqi Chen, Ben Ellis, Cedric Z\u00f6llner, Manthan Patel, Manuel Kaufmann, Marc Pollefeys, and Xi Wang; CoMind (2026); excerpt starts 1s, original full hash not verified","id":"stir","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"4ffe78850d4e6475546a521e0eb425f7a34c632679183cf112aacb70ec1b0861","source_recording_id":"https://comind.ethz.ch/","source_url":"https://comind.ethz.ch/","title":"Stirring vegetables in a pan"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"pack","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"87a67e5b10f124a65df7d669086d4a9b2cb7ae25a5d758594b9b3aa7b0938f4c","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010087.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010087.mp4","title":"Moving food from a tray toward a container"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"onion","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"100e25698ee99e99dc648c41d9864144e312f0fce7c07dc9a7b06b487b9a2a5f","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010072.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010072.mp4","title":"Slicing an onion on a pink plate"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"melon","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"f32cf75d28602d169b4f302830ba633a351ac890596887c54c8e928efcf2650e","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010076.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010076.mp4","title":"Cutting a watermelon wedge"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"jeans","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"95845362951b3f78b7b731691df6d472bf176ced4b07ecef4d99d0b8a21da3e2","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010081.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010081.mp4","title":"Folding grey jeans"}]}
\ No newline at end of file
diff --git a/benchmarks/observation-20261010/hardened-run.json b/benchmarks/observation-20261010/hardened-run.json
new file mode 100644
index 0000000..30e5c2a
--- /dev/null
+++ b/benchmarks/observation-20261010/hardened-run.json
@@ -0,0 +1 @@
+{"accuracy_status":"not_assessed","attempted":6,"completed":6,"elapsed_seconds":4.635012916056439,"episodes":[{"episode_sha256":"26b67bb490a0853482817f840715a17ae827830f32898b5fed60ccbcd51fd1a0","id":"shirt"},{"episode_sha256":"8b18fbda42aa44fa008fd763c865a045a416947b00464126b2e31b1c2c1c5ac8","id":"stir"},{"episode_sha256":"bb76a1996112fcb0bed37b988a4698676788b068d77e33ac4bc8cd080f6042da","id":"pack"},{"episode_sha256":"b94f9d15c4ecfc8c843d6c2d0e22568dbe32337cfe03091b7ef1bb529e1ee449","id":"onion"},{"episode_sha256":"7be44932308e08032322cfff7d9ee264335c016266c36015b0a991fb3cef9dab","id":"melon"},{"episode_sha256":"dee34f090cd5697889a6f904e30be6669085bf4db79273c56af3e4959151168e","id":"jeans"}],"failures":[],"geometry":"image_coordinates_only","kind":"dvidia.observation-run.v1","protocol_sha256":"3cd74c261157101b459563ff2b449c847194055fa024e60a6ed56aed94cf1b9c","robot_skill_qualified":false,"sessions":{"unknown-shared-publisher-session":["shirt","stir","pack","onion","melon","jeans"]},"timing_scope":"intake, decode, inference, association and per-episode writes; excludes model setup and final receipt"}
\ No newline at end of file
diff --git a/benchmarks/observation-20261010/initial-protocol.json b/benchmarks/observation-20261010/initial-protocol.json
new file mode 100644
index 0000000..4425dd8
--- /dev/null
+++ b/benchmarks/observation-20261010/initial-protocol.json
@@ -0,0 +1 @@
+{"created_at":"2026-10-10T11:14:17.377906+00:00","execution":"local_offline","hardware":{"machine":"arm64","platform":"Darwin"},"kind":"dvidia.observation-protocol.v1","limits":{"cpu_threads":4,"workers":1},"manifest_sha256":"6e4e3a0b1b12eeb32cafcba0668d2614c80b022a91eafc7740018be880eddb29","model":{"backend":"opencv","coordinates":"normalized_image_xyxy","cpu_threads":10,"device":"cpu","hand_box_semantics":"Palm detector rectangle; not full hand segmentation or observed contact.","hand_detector_enabled":true,"hand_threshold":0.5,"implementation_sha256":"89cf6a3058661ec03acf78e3eaeab1182f235b0c2fe0d03b406fbaaafb29c2a4","implementation_version":1,"kind":"opencv-zoo-yolox-palm","limitations":["COCO labels can be wrong or absent, especially for small and occluded objects.","Palm boxes do not establish finger contact, grasp, force, depth, task success or robot actions.","Object identity and temporal relationships require a separate tracking stage."],"max_detections":30,"max_nms_candidates":1000,"models":[{"bytes":35858002,"directory":"object_detection_yolox","download_url":"https://media.githubusercontent.com/media/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/object_detection_yolox_2022nov.onnx","license":"Apache-2.0","license_url":"https://raw.githubusercontent.com/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/LICENSE","name":"object_detection_yolox_2022nov.onnx","revision":"47534e27c9851bb1128ccc0102f1145e27f23f98","role":"objects","sha256":"c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063","source":"https://github.com/opencv/opencv_zoo/blob/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox/object_detection_yolox_2022nov.onnx"},{"bytes":3905734,"directory":"palm_detection_mediapipe","download_url":"https://media.githubusercontent.com/media/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/palm_detection_mediapipe_2023feb.onnx","license":"Apache-2.0","license_url":"https://raw.githubusercontent.com/opencv/opencv_zoo/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/LICENSE","name":"palm_detection_mediapipe_2023feb.onnx","revision":"47534e27c9851bb1128ccc0102f1145e27f23f98","role":"palms","sha256":"78ff51c38496b7fc8b8ebdb6cc8c1abb02fa6c38427c6848254cdaba57fcce7c","source":"https://github.com/opencv/opencv_zoo/blob/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe/palm_detection_mediapipe_2023feb.onnx"}],"nms_iou":{"objects":0.5,"palms":0.3},"runtime":{"numpy":"2.5.3","opencv":"5.0.0"},"score_kind":"uncalibrated_model_evidence","threshold":0.35},"recipe":{"automatic_outcome":"unknown","claim_scope":"sampled_2d_observations","id":"dvidia-observations-v1","max_duration_seconds":60,"max_frames":120,"max_image_side":640,"max_track_gap_seconds":0.8,"minimum_association_iou":0.2,"samples_per_second":2},"sources":[{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"shirt","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"8392c62f4e9590a15745311746114c2b6355e24c60fa87fef68e834f20762113","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010079.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010079.mp4","title":"Starting to fold a white shirt"},{"credit":"Alexey Gavryushin, Dingxi Zhang, Zhao Huang, Alexandros Delitzas, Jiaqi Chen, Ben Ellis, Cedric Z\u00f6llner, Manthan Patel, Manuel Kaufmann, Marc Pollefeys, and Xi Wang; CoMind (2026); excerpt starts 1s, original full hash not verified","id":"stir","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"4ffe78850d4e6475546a521e0eb425f7a34c632679183cf112aacb70ec1b0861","source_recording_id":"https://comind.ethz.ch/","source_url":"https://comind.ethz.ch/","title":"Stirring vegetables in a pan"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"pack","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"87a67e5b10f124a65df7d669086d4a9b2cb7ae25a5d758594b9b3aa7b0938f4c","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010087.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010087.mp4","title":"Moving food from a tray toward a container"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"onion","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"100e25698ee99e99dc648c41d9864144e312f0fce7c07dc9a7b06b487b9a2a5f","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010072.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010072.mp4","title":"Slicing an onion on a pink plate"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"melon","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"f32cf75d28602d169b4f302830ba633a351ac890596887c54c8e928efcf2650e","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010076.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010076.mp4","title":"Cutting a watermelon wedge"},{"credit":"Taher Panbiharwala and Zainab Barwaniwala; EgoAnnotate v1; excerpt starts 0s, original full hash not verified","id":"jeans","kind":"public","license":"CC BY 4.0","permission":"local_analysis","session_id":"unknown-shared-publisher-session","sha256":"95845362951b3f78b7b731691df6d472bf176ced4b07ecef4d99d0b8a21da3e2","source_recording_id":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010081.mp4","source_url":"https://huggingface.co/datasets/TaherPanbiharwala/EgoAnnotate/blob/156d13adaee4110cf63e54fa940ec16ff199d12f/videos/GX010081.mp4","title":"Folding grey jeans"}]}
\ No newline at end of file
diff --git a/benchmarks/observation-20261010/initial-run.json b/benchmarks/observation-20261010/initial-run.json
new file mode 100644
index 0000000..5a57c5e
--- /dev/null
+++ b/benchmarks/observation-20261010/initial-run.json
@@ -0,0 +1 @@
+{"accuracy_status":"not_assessed","attempted":6,"completed":6,"elapsed_seconds":4.835139082977548,"episodes":[{"episode_sha256":"e5225fd8209e6a27f1b906d02cd642f52d4b21693a5d1ec671146aacaa7282b7","id":"shirt"},{"episode_sha256":"a4b1323797769802f7b945185748814038c1ef391fb0f94cf77a84cf5748a24e","id":"stir"},{"episode_sha256":"4ab53292050295dbd3fa3615b499b42a61013c7751fd992ba725266b241cec7b","id":"pack"},{"episode_sha256":"a4a4cf5b23ae679a712f79a522824b347560d5f0c1a6dcc246316244af6f900e","id":"onion"},{"episode_sha256":"a5cc8dd002f65cde1639771952fa0099415e22a2103e44cd1bfba7147e8e62ca","id":"melon"},{"episode_sha256":"dc0ace7b027e2260e791b278d9241c8b42f2e9243ce7d0ab9e47a00a25db734d","id":"jeans"}],"failures":[],"geometry":"image_coordinates_only","kind":"dvidia.observation-run.v1","protocol_sha256":"71629e04edd72e1daccb968e011f68db07ae87cf9fa083976cb1a73744a32926","robot_skill_qualified":false,"sessions":{"unknown-shared-publisher-session":["shirt","stir","pack","onion","melon","jeans"]},"timing_scope":"intake, decode, inference, association and per-episode writes; excludes model setup and final receipt"}
\ No newline at end of file
diff --git a/benchmarks/observation-20261010/ui-validation.json b/benchmarks/observation-20261010/ui-validation.json
new file mode 100644
index 0000000..e39c612
--- /dev/null
+++ b/benchmarks/observation-20261010/ui-validation.json
@@ -0,0 +1,22 @@
+{
+ "scope": "Local isolated UI QA copy; not model accuracy",
+ "checks": [
+ "Desktop loads real episode and unknown outcome without horizontal overflow",
+ "Video plays, sampled overlays render, and box visibility toggles",
+ "Corrected approval persists after reload while raw label and unknown outcome remain unchanged",
+ "A new blank manual event can be removed; a labeled manual event can be approved and saved",
+ "Downloaded review matches persisted revision and original source hash",
+ "Episode navigation protects unsaved changes, preserves Keep reviewing, and honors explicit discard",
+ "Structured model and source provenance render as readable text",
+ "390px phone layout has no horizontal overflow and visible controls have 44px targets",
+ "320px phone layout has no horizontal overflow and visible controls have 44px targets",
+ "No browser console or page errors",
+ "Accepted events preserve decimal timing, including 0.25s\u20130.50s",
+ "Overlay counts match timestamp-bounded real evidence at 12 sample and gap boundaries",
+ "Pending events block completion without writing an invalid revision",
+ "390px: event selection reveals editor, provenance follows review, no overflow, 44px controls",
+ "320px: event selection reveals editor, provenance follows review, no overflow, 44px controls"
+ ],
+ "console_errors": [],
+ "screenshots": "Kept locally with ignored QA run; no source frames redistributed"
+}
diff --git a/docs/observation-models.md b/docs/observation-models.md
new file mode 100644
index 0000000..5338367
--- /dev/null
+++ b/docs/observation-models.md
@@ -0,0 +1,32 @@
+# Observation model provenance
+
+Weights are downloaded explicitly into ignored local storage, never committed.
+The observation adapter's pre/postprocessing is adapted from OpenCV Zoo; that
+file is Apache-2.0 and retains upstream attribution. The rest of this repository
+retains its existing license. Original licenses are included in
+[`third_party/observation-models`](../third_party/observation-models/).
+
+Pinned upstream revision: `47534e27c9851bb1128ccc0102f1145e27f23f98`.
+
+| Role | Exact artifact | Bytes | SHA-256 |
+| --- | --- | ---: | --- |
+| COCO object detection | `object_detection_yolox_2022nov.onnx` | 35,858,002 | `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063` |
+| Palm detection | `palm_detection_mediapipe_2023feb.onnx` | 3,905,734 | `78ff51c38496b7fc8b8ebdb6cc8c1abb02fa6c38427c6848254cdaba57fcce7c` |
+
+The official [YOLOX directory](https://github.com/opencv/opencv_zoo/tree/47534e27c9851bb1128ccc0102f1145e27f23f98/models/object_detection_yolox)
+and [MediaPipe palm directory](https://github.com/opencv/opencv_zoo/tree/47534e27c9851bb1128ccc0102f1145e27f23f98/models/palm_detection_mediapipe)
+each state that all files in the directory are licensed under Apache 2.0.
+The downloaded license text is also pinned and hash-checked by the downloader.
+YOLOX's included license attributes Megvii Inc., 2021–2022.
+
+DVIDIA modifications include local weight verification, bounded deterministic
+suppression, clipped normalized image boxes, explicit palm-only semantics,
+metadata, bounded inputs and validation of model output. Inference time and peak
+RAM/thread use are not hard capped. No remote Python code
+is downloaded or executed. Source data licensing remains independent from model
+licensing. These license records do not establish rights to any user's footage.
+
+No SpatialLM, MANO, depth model or language model weights are part of this pilot.
+The exact detector recipes and runtime versions are recorded in each run. The
+first reports preserve the adapter implementation hash as it existed at run time;
+later source or notice changes do not retroactively change those receipts.
diff --git a/docs/observation-pilot-plan.md b/docs/observation-pilot-plan.md
new file mode 100644
index 0000000..5967f7b
--- /dev/null
+++ b/docs/observation-pilot-plan.md
@@ -0,0 +1,63 @@
+# DVIDIA observation pilot v1
+
+Planned 10 October 2026, before model execution. This is an engineering feasibility
+pilot, not a trained robot skill, production deployment, or accuracy benchmark.
+
+## Deliverables
+
+1. Local, bounded video decoding with actual sampled presentation timestamps,
+ original/frame hashes, source rights and session provenance.
+2. Optional local detector and temporal caption adapters. DVIDIA owns the
+ orchestration, association, annotation contract and review workflow; pretrained
+ components retain their own licenses. No SpatialLM weights are used.
+3. A mobile-friendly video/timeline review with editable proposals, explicit
+ outcomes, immutable review revisions and optimistic concurrency.
+4. A runnable pilot receipt with time, resource scope, failures and limitations.
+ An independent reference-label evaluator must report not assessed when no
+ references exist; model agreement is not human review.
+
+## Fixed experiment scope
+
+- First candidate task: move a cup to a marked surface. Available public sample
+ footage may instead establish adapter feasibility; record task mismatches.
+- Begin with five existing attributed EgoAnnotate excerpts (shirt, pack,
+ onion, melon, jeans) and one CoMind excerpt (stir), without selecting by model outputs. These are
+ excerpts, not independent complete cup-placement attempts.
+- At most six initial clips, 60 seconds per clip, 128 MiB per clip, 120 frames
+ per clip, four CPU threads, one inference worker, no concurrent model jobs.
+- Decode at most two samples/second, maximum image side 640. Proposals reference
+ actual decoded timestamps. No off-device media transfer or paid compute.
+- The first run uses object/palm detections and 2D overlap proposals. Temporal
+ language captions, if added, receive a separate frozen recipe: at most three
+ windows per clip and four frames per window, maximum 96 generated tokens per
+ window. Record model/load/inference time separately.
+- Download only official model artifacts and registry packages, capped at 2 GiB
+ of model weights for this first run. Inference is offline after preparation.
+- Model failure remains a failed attempt. Never fabricate detections, timestamps,
+ grasp states, successful completion, metric geometry, contact or robot actions.
+- Freeze the recipe before the initial run; adjustments produce another recipe
+ and result directory. Preserve the original attempt and failures.
+
+## Proposed cup cohort (not collected)
+
+Collect 30–50 complete permission-cleared attempts from several people and
+setups, including occluded cups, wrong targets, drops, incomplete attempts and
+recoveries. Group by original/session/person/object before a development/test
+split. Obtain independent references for object boxes, event boundaries and
+observable outcome, blinded to proposals. A marked target and known-size
+reference enable later geometry checks; no metric coordinates are implied now.
+
+Compare detector-only proposals, caption-only proposals, their combination and
+manual review at matched quality. Report detection precision/recall and IoU,
+boundary timing error, missed/extra events, false completion, correction time,
+abstention and full cost. Keep model scores separate from measured accuracy.
+Physical skill qualification and action learning require separate experiments.
+
+## Gates
+
+The engineering gate is readable real video, hash-bound finite annotations,
+successful correction/reload/export, offline execution, explicit unknowns,
+resource limits and tests. The accuracy gate remains open until independent
+references exist. Commercial release additionally requires review of exact
+model/data licenses and evaluation on consented DVIDIA footage. Repeated scans
+may later support a 3D view, but v1 supplies no 3D reconstruction.
diff --git a/docs/observation-ui.md b/docs/observation-ui.md
new file mode 100644
index 0000000..001ed50
--- /dev/null
+++ b/docs/observation-ui.md
@@ -0,0 +1,129 @@
+# Local observation review
+
+The dependency-free page served by the observation lab combines the original
+video, sampled boxes, timeline and review queue. It makes no requests beyond its
+local origin. There are no fallback detections or automatic action descriptions.
+
+## Grouped review
+
+The main queue groups observations with the same original label, evidence type,
+current review label and decision when their intervals overlap or have gaps no
+larger than 0.25 seconds. This is a presentation aid, not physical object identity,
+tracking, an action segment or proof of continuity through an occlusion.
+
+Each card shows its exact interval and number of included observations. Confirm
+or reject applies only to the members shown in the selected group, then advances
+to the next pending group. Confirming a group never approves an unrelated group.
+Edited descriptions apply to those members; their individual timings remain
+unchanged. Differently corrected or decided observations stay separate. Manual
+annotations always remain independent.
+
+Expand a group's individual observations to edit one label or interval. The
+original raw list remains under **Inspect original evidence**. Nothing is dropped,
+suppressed or removed from the stored model record. The immutable source IDs,
+raw labels, sample timestamps, scores and boxes remain inspectable.
+
+The six initial pilot clips present 72 original proposals as 33 initial groups:
+shirt 7 → 4, stir 25 → 8, pack 7 → 4, onion 10 → 5, melon 14 → 7 and jeans 9 → 5.
+These are counts of UI items, not measured reductions in review time or improved
+model accuracy. Corrections can split groups. All original proposals still require
+an explicit decision before a review can be finished.
+
+## Video and timeline
+
+The timeline is attached to the video, highlights the selected interval, and
+shows confirmed intervals separately. Selecting a card seeks to its start.
+**Replay moment** plays just that interval. Individual edits can set boundaries
+from the playhead or explicit numeric values.
+
+Boxes use normalized `[x1,y1,x2,y2]` image coordinates and respect letterboxing.
+Samples are indexed once per episode and looked up with binary search. DOM
+nodes are reused by track and updated only when the displayed sample changes;
+identical samples do not rebuild the overlay. No new tracking or interpolation
+was added. Boxes appear only within 0.25 seconds of a real sampled timestamp,
+retain raw model labels and disappear through unsupported gaps. Palm rectangles
+remain palm detections; overlap does not establish contact or grasp.
+
+## Automatic saving and completion
+
+Valid drafts save after a 650ms pause. Only one request is in flight at a time.
+The response advances the base revision without replacing newer edits; any
+changes made during the request are saved in the next revision. Invalid input
+remains visibly unsaved and cannot be overwritten by an older save response.
+
+Errors preserve the draft and expose an explicit retry. Conflicting tabs never
+silently overwrite one another. A conflict offers export of the unsaved draft
+(including current editor input) or explicit discard/reload. Episode navigation
+and browser closing protect unsaved changes. No shared browser cache of private
+annotations is added: a reload before successful persistence can lose unsaved
+input, so heed the leave warning or export the draft.
+
+Draft autosave uses `in_review`. **Finish review** is a separate deliberate
+operation and is available only after every proposal has a decision. Failed
+completion stays visibly unsaved. Outcomes default to `unknown`; a person can
+select visible completion or incomplete when finishing. None implies robot skill
+qualification. Export of the saved review is disabled while a draft is unsaved.
+
+The local server stores append-only revisions tied to the original source and
+annotation hashes. The public synthetic demo instead uses its existing browser
+storage adapter; its UI explicitly describes that storage boundary.
+
+## Mobile and accessibility
+
+Below 821px, selecting a card opens a native dialog styled as a bottom sheet,
+with a large Confirm & next action, a reject action and a replay control. The
+page brings the video into view above it where viewport height allows. Closing
+returns to the queue, or to the finish panel once every item is decided. The dialog
+traps focus and makes background content inert. Detailed timing remains collapsed.
+On desktop, the same editor stays beside the player. Resizing handles the dialog
+state; landscape uses the layout appropriate to its available width.
+
+Controls have at least 44px touch targets; main decision buttons are 50px. Inputs
+use 16px text on phones. Semantic buttons, native dialog behavior, labeled inputs,
+visible focus, text status labels and polite save feedback accompany the color
+cues. Playback is never automatically started by opening a group. Reduced-motion
+preferences are honored.
+
+## HTTP and data contract
+
+- `GET /api/episodes`: `{ "episodes": [{ "id": "..." }], "failures": [] }`.
+- `GET /api/episodes/{id}`: the episode, immutable observations and latest review.
+- `POST /api/episodes/{id}/review`: `{ "base_revision": 0, "events": [],
+ "outcome": "unknown", "status": "in_review" }`; responds with `{ "review": ... }`.
+- Same-origin video URLs support byte ranges. Writes require the local origin.
+
+Each review event retains `id`, `observation_id`, `label`, `start_seconds`,
+`end_seconds` and `decision` (`pending`, `approved`, `rejected`). New manual events
+use a `manual-` UUID and `observation_id: null`. No group ID or automatic approval
+is added to the backend contract; grouping is derived from the full event list.
+
+## Repeatable checks
+
+`tests/test_observation_review_core.py` runs the pure JavaScript grouping and
+sample-boundary invariants when Node.js is installed. The existing Python tests
+cover the real decoder, bounded HTTP API, media integrity and review revisions.
+
+For isolated browser tests, use a new synthetic fixture directory in one terminal:
+
+```sh
+PYTHONPATH=src .venv/bin/python tests/browser/serve_observation_fixture.py --directory runs/review-browser-fixture --port 8283
+```
+
+With Playwright and its Chromium browser already installed, in another terminal:
+
+```sh
+node tests/browser/observation_review.cjs
+```
+
+`PLAYWRIGHT_MODULE` can select an existing package location. `OBSERVATION_URL`
+selects another loopback fixture URL; `OBSERVATION_OUTPUT` selects an ignored
+receipt/screenshot directory. The tests require a fresh unreviewed fixture.
+They simulate overlapping saves, invalid edits, conflicts, failed writes and
+failed completion, then exercise persistence, export, manual events, phone and
+landscape layouts, repeated sheet opening, rotation and scroll-lock recovery.
+They never modify the original pilot run or production data.
+
+The overlay mutation check makes ten seeks within a single sample and observes
+zero DOM mutations. This is a regression invariant, not a phone FPS benchmark.
+Headless Chromium checks do not establish Safari or physical-device performance.
+Model accuracy and human annotation speed remain unmeasured.
diff --git a/docs/observations.md b/docs/observations.md
new file mode 100644
index 0000000..7bad3e9
--- /dev/null
+++ b/docs/observations.md
@@ -0,0 +1,137 @@
+# Local video observation lab
+
+This is a runnable research feature in DVIDIA Training. It does not replace
+the upload flow or activate a production analysis service. The implementation
+owns intake, timestamps, proposal association, reviews and provenance. The two
+optional pretrained detectors retain their own licenses; no new foundation
+model has been trained and no SpatialLM weights are included.
+
+## Setup
+
+The public Observation Lab release is pinned separately from the older training
+pilot. Obtain it with:
+
+```sh
+git clone --branch observation-lab-v0.1.0 https://github.com/Dvidia-Inference/dvidia-training.git
+cd dvidia-training
+```
+
+From this repository checkout, with Python 3.12+ and FFmpeg/ffprobe installed:
+
+```sh
+python3.12 -m venv .venv
+.venv/bin/python -m pip install '.[observations]'
+.venv/bin/python scripts/download_observation_models.py --download --output runs/observation-models
+```
+
+The explicit provisioning command downloads about 40 MB from pinned official
+OpenCV Zoo artifacts and verifies sizes and SHA-256 hashes. Inference never
+downloads anything. Install software and models before disconnecting from the
+network. The tested runtime is Python 3.12.9, OpenCV 5.0.0.93, NumPy 2.5.3 on
+macOS ARM64. Other platforms have not been benchmarked. Model loading and
+software installation are excluded from the published six-clip processing time.
+
+## Supply permitted videos
+
+Keep the following manifest next to your own files. Replace every example value,
+including the hash; it is deliberately not a runnable dataset. Do not add raw
+recordings, private metadata or local paths to Git.
+
+```json
+{
+ "kind": "dvidia.observation-source.v1",
+ "task": "Move a cup onto a marked surface",
+ "episodes": [{
+ "id": "attempt-001",
+ "path": "attempt-001.mp4",
+ "title": "Cup placement, first attempt",
+ "session_id": "participant-a-session-1",
+ "source_recording_id": "consented-original-001",
+ "kind": "user_authorized",
+ "credit": "Consenting contributor; local analysis only",
+ "license": "Private; local analysis permission recorded",
+ "source_url": "local-authorized-recording",
+ "permission": "local_analysis",
+ "sha256": "REPLACE_WITH_THE_ORIGINAL_FILE_SHA256"
+ }]
+}
+```
+
+The permission field records a declaration; it does not obtain consent or grant
+rights. Use the same session/person/original groups for related clips. Unknown
+grouping must remain unknown, not become a new independent sample per excerpt.
+Publisher titles are display metadata, never model inputs.
+
+Inputs are limited to 50 clips per run, 60 seconds and 128 MiB per clip, 512 MiB
+per batch and 4096 pixels per source side. Formats follow the existing footage
+reader. For this pilot, video and container timestamps must start at zero;
+offset recordings fail with a saved reason rather than silently shifting boxes.
+An explicit normalized derivative can be supplied separately with its provenance;
+the tool never rewrites the original.
+
+## Analyze, review, inspect
+
+```sh
+.venv/bin/dvidia-observe analyze /path/to/source.json --models runs/observation-models --output runs/observations-001
+.venv/bin/dvidia-observation-studio runs/observations-001 --port 8280
+```
+
+Open http://127.0.0.1:8280. Choose an episode, play or scrub, select an observation,
+correct its description/timing and approve or reject it. Add a missed event when
+needed. Save a draft or explicitly finish the review. Source proposals remain
+separate; edits create append-only revisions with stale-tab conflict detection.
+Download exports the last saved review. Outcomes default to unknown.
+
+```sh
+.venv/bin/dvidia-observe inspect runs/observations-001
+```
+
+This verifies the source/protocol/episode hashes and revision chain, then reports
+review progress. Without independent references it returns `accuracy: null` and
+`accuracy_status: not_assessed`. A review is an assessment by the local operator,
+not an authenticated identity or independent accuracy measurement.
+
+The lab listens only on 127.0.0.1, checks the local origin for writes, supports
+bounded video byte ranges, and performs no uploads. It serves no arbitrary files
+or model URLs. Run it only with bundles you trust; local users can read files in
+the same account. Hashes detect accidental changes but are not digital signatures.
+
+## What the observations mean
+
+- YOLOX suggests COCO object labels and image rectangles. Scores are uncalibrated.
+- A MediaPipe palm detector suggests palm rectangles, not full hand segmentation.
+- Greedy same-class image-overlap association joins nearby detections; it does
+ not prove physical identity and splits gaps. No boxes fill an occlusion.
+- Hand/object box overlap yields an explicit 2D proximity hypothesis. It cannot
+ establish touching, grasping, force, depth, task completion or robot commands.
+- Sampling is two frames/second, up to 120 frames and 640 pixels per side. The UI
+ displays boxes only within 0.25 seconds of a sampled timestamp. This is a
+ recorded evidence viewer, not a smooth live hand tracker.
+
+The run stores copied originals, sampled frames, raw proposals, failures, recipe,
+model hashes, source rights and review revisions. Allow scratch space beyond the
+input size. One inference worker is used and four OpenCV threads are requested;
+the six-clip pilot recorded `cv2.getNumThreads()` as ten after loading. Actual
+peak thread concurrency was not measured, so no four-thread cap is established.
+There is no hard inference-time, RAM or container limit. Do not install it on a production database machine
+without a separate resource-constrained deployment test.
+
+## Next gates
+
+1. Collect the [planned cup-placement cohort](observation-pilot-plan.md), including
+ complete successes, failures, occlusions and recoveries from several setups.
+2. Obtain independent human references before assessing labels or timing.
+3. Add a separate temporal caption recipe to describe observable actions; compare
+ it with detector-only and combined proposals, including correction effort.
+4. Add depth/geometry only with explicit scale and validation; then test a
+ separately supervised robot-action bridge and closed-loop safety/competence.
+5. Integrate queued analysis into the app only after quality, cost, retries,
+ permissions and mobile review are validated on consented DVIDIA recordings.
+
+See [model licenses](observation-models.md), [UI contract](observation-ui.md) and
+the [actual feasibility report](../benchmarks/observation-20261010/README.md).
+
+[Try the synthetic review demo](https://huggingface.co/spaces/Dvidia/observation-lab)
+or [read the research publication](https://research.dvidia.org/papers/dvidia-observation-lab/).
+The hosted demo has authored boxes and browser-local review state; live detection
+of permitted footage runs in the installed local tool.
diff --git a/publishing/huggingface/observation-lab/README.md b/publishing/huggingface/observation-lab/README.md
new file mode 100644
index 0000000..d44b7f1
--- /dev/null
+++ b/publishing/huggingface/observation-lab/README.md
@@ -0,0 +1,76 @@
+# Publishing the Observation Lab workflow demo
+
+This preparation package is separate from the original training-pilot publisher.
+It stages **only an authored synthetic static Space**, intended for
+`Dvidia/observation-lab`. It has no upload function, credential reader or model
+dependency. Do not pass its plan to the older three-repository publisher, which
+correctly refuses a different destination/layout.
+
+## Prepare and inspect
+
+From the training repository, using installed Python3.12+ and FFmpeg/libx264:
+
+```sh
+python3 publishing/huggingface/observation-lab/prepare.py --org Dvidia --output runs/observation-space-publication
+python3 -m http.server 8294 --bind 127.0.0.1 --directory runs/observation-space-publication/space
+```
+
+Open http://127.0.0.1:8294. Check playback, sampled boxes, label/time edits,
+approval/rejection, manual events, save/reload, review JSON download and reset.
+Check a second clean browser context starts at revision 0, while another tab in
+the same browser reports a stale revision instead of overwriting it. Blocked
+browser storage should show the temporary-memory notice.
+
+The preparation script has a fixed source allowlist; it never scans private
+`runs/`, user footage, model directories or recursively copies `src/`. It copies
+the exact review UI to `source/` and makes a labeled browser-only API derivative.
+It generates two 8-second 640×360 schematic clips using stdlib RGB drawing and
+FFmpeg/libx264. All demo state stays in browser storage or temporary memory.
+Nothing is posted to a server; media requests read only the included fixtures.
+
+Review `space/README.md`, `index.html`, `demo-adapter.js`, `provenance.json`,
+`demo/provenance.json` and the exact `publication-plan.json` inventory/hashes.
+The preparation budget is 2 MiB. No model weights, secrets, private clips or
+commercial server dependencies are bundled. A static Space has no container;
+therefore there is no container/image validation or model throughput claim.
+
+## Maintainer publication
+
+Use the official Hub SDK in an isolated maintainer environment and the existing
+secure credential path. Create public `Dvidia/observation-lab` with
+`repo_type='space'` and `space_sdk='static'`; upload only the reviewed `space/`
+directory with the plan's exact inventory. Do not upload the outer staging
+directory or local token files. No application token/secret, GPU allocation or
+persistent shared review store is needed.
+
+Before a remote mutation, freeze the reviewed bytes to a separate bounded
+snapshot, verify every digest/inventory entry, check destination organization
+membership and preserve any differing existing remote files rather than
+overwriting them silently. After publication, anonymously fetch every artifact
+at the returned exact commit and compare hashes; confirm the live Space starts,
+playback/download work and synthetic/local-state labels remain visible.
+
+The user explicitly authorized this public publication, but this preparation
+script performs no remote calls. Actual local ONNX tool installation and model
+notices are linked from the demo; its own code/data remain MIT.
+
+The dedicated publisher validates this single-Space plan, reads the named
+Keychain credential through the existing in-memory adapter, refuses differing
+existing remote bytes, and anonymously verifies every file at the new commit:
+
+```sh
+python3 publishing/huggingface/publish_observation.py check runs/observation-space-publication/publication-plan.json
+runs/huggingface-tools/.venv/bin/python publishing/huggingface/publish_observation.py publish runs/observation-space-publication/publication-plan.json
+```
+
+Provision the maintainer environment and Keychain entry using the parent
+[publishing guide](../README.md). Never paste a token into a command or the plan.
+
+```sh
+python3 -m unittest discover -s publishing/huggingface/observation-lab -p 'test_*.py'
+node publishing/huggingface/observation-lab/test_adapter.cjs
+```
+
+Official references: [Static Spaces](https://huggingface.co/docs/hub/spaces-sdks-static),
+[configuration](https://huggingface.co/docs/hub/spaces-config-reference),
+[Hub uploads](https://huggingface.co/docs/huggingface_hub/guides/upload).
diff --git a/publishing/huggingface/observation-lab/generate_demo.py b/publishing/huggingface/observation-lab/generate_demo.py
new file mode 100644
index 0000000..10748cd
--- /dev/null
+++ b/publishing/huggingface/observation-lab/generate_demo.py
@@ -0,0 +1,121 @@
+"""Generate only DVIDIA-authored schematic video and annotation fixtures."""
+from __future__ import annotations
+
+import argparse
+from hashlib import sha256
+import json
+from pathlib import Path
+import shutil
+import subprocess
+
+WIDTH, HEIGHT, FPS, SECONDS = 640, 360, 20, 8
+
+
+def rectangle(image, x, y, width, height, color):
+ row = bytes(color) * width
+ for yy in range(y, y + height):
+ image[(yy * WIDTH + x) * 3:(yy * WIDTH + x + width) * 3] = row
+
+
+def object_box(time):
+ x = int(96 + 344 * min(time / 5.5, 1))
+ return [x, 172, x + 64, 236]
+
+
+def frame(time, occluded):
+ image = bytearray(bytes([245, 244, 239]) * WIDTH * HEIGHT)
+ rectangle(image, 32, 80, 576, 228, [224, 226, 220])
+ for x, y, w, h in ((430, 153, 100, 3), (430, 253, 100, 3),
+ (430, 153, 3, 103), (527, 153, 3, 103)):
+ rectangle(image, x, y, w, h, [76, 124, 91])
+ x1, y1, x2, y2 = object_box(time)
+ rectangle(image, x1, y1, x2 - x1, y2 - y1, [61, 106, 191])
+ if occluded:
+ rectangle(image, 245, 108, 142, 173, [195, 194, 183])
+ for x in range(248, 385, 12):
+ rectangle(image, x, 110, 3, 169, [214, 214, 205])
+ return image
+
+
+def generate(output):
+ output = Path(output).absolute()
+ if any(p.is_symlink() for p in (output, *output.parents)) or output.exists():
+ raise ValueError('Use a fresh plain fixture directory.')
+ ffmpeg = shutil.which('ffmpeg')
+ if ffmpeg is None:
+ raise ValueError('Preparing schematic videos requires installed FFmpeg with libx264.')
+ output.mkdir(parents=True)
+ version = subprocess.run([ffmpeg, '-version'], check=True, capture_output=True, text=True).stdout.splitlines()[0]
+ fixtures = []
+ for name, occluded in [('visible', False), ('occluded', True)]:
+ video = output / (name + '.mp4')
+ args = [ffmpeg, '-v', 'error', '-nostdin', '-threads', '1', '-f', 'rawvideo',
+ '-pix_fmt', 'rgb24', '-s', f'{WIDTH}x{HEIGHT}', '-r', str(FPS),
+ '-i', 'pipe:0', '-an', '-c:v', 'libx264', '-threads', '1',
+ '-pix_fmt', 'yuv420p', '-movflags', '+faststart', '-map_metadata', '-1', str(video)]
+ process = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+ try:
+ for index in range(FPS * SECONDS):
+ process.stdin.write(frame(index / FPS, occluded))
+ process.stdin.close()
+ error = process.stderr.read()
+ if process.wait(timeout=30) != 0:
+ raise ValueError('Synthetic video encoding failed.')
+ finally:
+ if process.poll() is None:
+ process.kill()
+ process.wait()
+ source_hash = sha256(video.read_bytes()).hexdigest()
+ observations = []
+ intervals = [(0, 8)] if not occluded else [(0, 1.5), (5, 8)]
+ for number, (start, end) in enumerate(intervals, 1):
+ boxes = []
+ for index in range(SECONDS * 2):
+ t = index / 2
+ if not start <= t < end:
+ continue
+ bounds = object_box(t)
+ boxes.append({'time_seconds': t, 'xyxy': [bounds[0] / WIDTH, bounds[1] / HEIGHT,
+ bounds[2] / WIDTH, bounds[3] / HEIGHT], 'label': 'blue shape · authored'})
+ observations.append({'id': f'sample-{number}', 'label': 'Blue shape visible · authored example',
+ 'start_seconds': start, 'end_seconds': end, 'boxes': boxes,
+ 'evidence': 'authored_schematic_example', 'review_state': 'proposed'})
+ observations.append({'id': 'region', 'label': 'Marked region visible · authored example',
+ 'start_seconds': 0, 'end_seconds': 8, 'boxes': [
+ {'time_seconds': i / 2, 'xyxy': [430 / WIDTH, 153 / HEIGHT, 530 / WIDTH, 256 / HEIGHT],
+ 'label': 'marked region · authored'} for i in range(SECONDS * 2)],
+ 'evidence': 'authored_schematic_example', 'review_state': 'proposed'})
+ episode = {'kind': 'dvidia.observation-demo-episode.v1', 'id': name,
+ 'title': 'Blue shape crosses an occluder' if occluded else 'Blue shape moves into a marked region',
+ 'video_url': 'demo/' + name + '.mp4', 'duration_seconds': SECONDS,
+ 'source': {'kind': 'synthetic', 'sha256': source_hash, 'credit': 'DVIDIA contributors · authored geometric fixture',
+ 'license': 'MIT', 'session_id': 'synthetic-demo-one-dependent-group',
+ 'source_recording_id': 'authored-' + name, 'source_url': 'generate_demo.py',
+ 'notes': 'No person, robot, camera capture, detector or learned action is represented.'},
+ 'recipe': {'id': 'dvidia-authored-schematic-demo-v1',
+ 'model': {'kind': 'Authored fixture · No model ran', 'models': []},
+ 'notes': 'Deterministic example annotations derived from the drawing recipe. No live inference, accuracy score, task segmentation or robot skill is demonstrated.'},
+ 'observations': observations,
+ 'review': {'revision': 0, 'status': 'unreviewed', 'outcome': 'unknown', 'events': [
+ {'id': o['id'], 'observation_id': o['id'], 'label': o['label'],
+ 'start_seconds': o['start_seconds'], 'end_seconds': o['end_seconds'],
+ 'decision': 'pending'} for o in observations]}}
+ (output / (name + '.json')).write_text(json.dumps(episode, indent=2) + '\n')
+ fixtures.append({'id': name, 'title': episode['title'], 'duration_seconds': SECONDS,
+ 'video_url': episode['video_url'], 'status': 'unreviewed'})
+ (output / 'episodes.json').write_text(json.dumps({'episodes': fixtures, 'failures': []}, indent=2) + '\n')
+ files = [{'path': p.name, 'bytes': p.stat().st_size, 'sha256': sha256(p.read_bytes()).hexdigest()}
+ for p in sorted(output.iterdir()) if p.is_file()]
+ (output / 'provenance.json').write_text(json.dumps({'kind': 'dvidia.authored-demo-fixtures.v1',
+ 'generation': 'stdlib RGB rectangle drawing piped to local FFmpeg/libx264; no source footage input',
+ 'generator_sha256': sha256(Path(__file__).read_bytes()).hexdigest(), 'ffmpeg': version,
+ 'dimensions': [WIDTH, HEIGHT], 'fps': FPS, 'seconds_per_clip': SECONDS,
+ 'annotations': 'authored image rectangles every 0.5 seconds; no model inference',
+ 'occlusion': 'box annotations deliberately absent when the blue shape is behind the drawn panel',
+ 'independent_source_groups': 1, 'files': files}, indent=2) + '\n')
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--output', required=True, type=Path)
+ generate(parser.parse_args().output)
diff --git a/publishing/huggingface/observation-lab/prepare.py b/publishing/huggingface/observation-lab/prepare.py
new file mode 100644
index 0000000..c728dd5
--- /dev/null
+++ b/publishing/huggingface/observation-lab/prepare.py
@@ -0,0 +1,166 @@
+"""Prepare an explicit synthetic-only static Observation Lab Space; never upload."""
+from __future__ import annotations
+
+import argparse
+from hashlib import sha256
+import json
+from pathlib import Path
+import re
+import shutil
+import subprocess
+
+HERE = Path(__file__).resolve().parent
+ROOT = HERE.parents[2]
+MAX_TOTAL_BYTES = 2 * 1024 * 1024
+
+
+def record(path):
+ raw = path.read_bytes()
+ return {'bytes': len(raw), 'sha256': sha256(raw).hexdigest()}
+
+
+def plain(path):
+ if any(p.is_symlink() for p in (path, *path.parents)) or not path.is_file():
+ raise ValueError('Preparation accepts only the named plain source files.')
+ return path
+
+
+def fresh(path):
+ path = Path(path).absolute()
+ if any(p.is_symlink() for p in (path, *path.parents)):
+ raise ValueError('Publication output may not contain symlinks.')
+ if path.exists() and (not path.is_dir() or any(path.iterdir())):
+ raise ValueError('Choose a fresh or empty publication directory.')
+ path.mkdir(parents=True, exist_ok=True)
+ return path
+
+
+def replace_once(text, old, new):
+ if text.count(old) != 1:
+ raise ValueError('Observation UI changed; review the demo adapter before preparing.')
+ return text.replace(old, new)
+
+
+def stage(namespace, output):
+ if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]{0,95}', namespace):
+ raise ValueError('Expected a Hugging Face organization name.')
+ output = fresh(output)
+ space = output / 'space'
+ space.mkdir()
+ # Explicit inventory only: no private runs, clips, models, credentials or
+ # recursive source/package copies can enter the public Space.
+ sources = {
+ 'README.md': HERE / 'templates' / 'README.md',
+ 'demo-adapter.js': HERE / 'templates' / 'demo-adapter.js',
+ 'generate_demo.py': HERE / 'generate_demo.py',
+ 'LICENSE': ROOT / 'LICENSE',
+ 'THIRD_PARTY.md': HERE / 'templates' / 'THIRD_PARTY.md',
+ 'licenses/object_detection_yolox-LICENSE.txt': ROOT / 'third_party' / 'observation-models' / 'object_detection_yolox-LICENSE.txt',
+ 'licenses/palm_detection_mediapipe-LICENSE.txt': ROOT / 'third_party' / 'observation-models' / 'palm_detection_mediapipe-LICENSE.txt',
+ 'source/observation_studio.html': ROOT / 'src' / 'dvidia_training' / 'observation_studio.html',
+ }
+ source_records = []
+ for name, source in sources.items():
+ plain(source)
+ target = space / name
+ target.parent.mkdir(parents=True, exist_ok=True)
+ shutil.copyfile(source, target)
+ source_records.append({'source_path': source.relative_to(ROOT).as_posix(),
+ 'published_path': name, **record(source)})
+ readme = space / 'README.md'
+ readme.write_text(readme.read_text().replace('{{HF_ORG}}', namespace))
+ ui = (space / 'source' / 'observation_studio.html').read_text()
+ start = ui.index(' const api = {\n')
+ end = ui.index(' function error(message)', start)
+ if ' save(id,body)' not in ui[start:end]:
+ raise ValueError('Expected the documented Observation API.')
+ ui = ui[:start] + ' const api = window.DvidiaObservationDemo;\n' + ui[end:]
+ ui = replace_once(ui, '\n
+