Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
3aa5337
feat(ods): add owner-side flow control and simplify BC demo
yuecideng Oct 20, 2026
1f23b37
feat(ods): add real data-stream and decoded LeRobot showcase
yuecideng Oct 20, 2026
6a980a7
fix(ods): isolate showcase GPUs and require complete publication
yuecideng Oct 20, 2026
1869ba9
fix(ods): qualify explicit Vulkan auxiliary GPU resources
yuecideng Oct 20, 2026
cf18c61
fix(ods): drain decoded readers and scope consumer wait metrics
yuecideng Oct 20, 2026
6d86b81
docs(ods): publish validated stream comparison and reports
yuecideng Oct 20, 2026
b955fc5
perf(ods): remove redundant sampling copies and padding
yuecideng Oct 21, 2026
82efc91
feat(ods): add fixed-budget visual training comparison
yuecideng Oct 21, 2026
23ad072
fix(ods): scope GPU admission to each training pipeline
yuecideng Oct 21, 2026
33f25d6
fix(ods): overlap recording fairly in benchmark controls
yuecideng Oct 21, 2026
9b61675
feat(ods): audit full video frames and SDK pipeline activity
yuecideng Oct 21, 2026
0a2e0b9
Record ODS visual study and harden process-group shutdown
yuecideng Oct 21, 2026
915510b
fix(ods): budget visual prefetch and measure complete data supply
yuecideng Oct 22, 2026
2f4010e
docs(ods): retain budget-aware scheduling pilot and capacity guidance
yuecideng Oct 22, 2026
8f101b2
feat(ods): consume published subbatches with bounded archive ownership
yuecideng Oct 22, 2026
3829593
docs(ods): retain publication handoff pilot and audit evidence
yuecideng Oct 22, 2026
3a40b34
chore(ods): remove generated evidence from the code review
yuecideng Oct 23, 2026
14d9c3e
feat(ods): add owner-side flow control and harden sampling lifecycle
yuecideng Oct 23, 2026
9e99224
Merge standalone ODS foundation into visual showcase branch
yuecideng Oct 23, 2026
1a09cb8
docs(ods): describe the independently reviewable foundation layer
yuecideng Oct 23, 2026
4a8dbd2
fix(ods): isolate runtime test imports and clarify live-source scope
yuecideng Oct 24, 2026
40d7a1b
Merge isolated runtime tests and clarified ODS source scope
yuecideng Oct 24, 2026
2de54bd
Sync visual ODS showcase with the merged foundation on main
yuecideng Oct 24, 2026
51f1f0c
feat(ods): prepare dual-camera PickPlace learning recipe and preview
yuecideng Oct 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,8 @@ topics:
title: Differentiable Learning Layers
aliases: [learning layers, differentiable IK layer, 可微 IK, 训练层, 学习模块分层]
keywords: [DifferentiableIKLayer, KinematicsLayer, SerialChainKinematics, DifferentiableIKCfg,
solve_dls_ik, damped_least_squares_step, Jacobian gradient, dJ/dq, learn_grasp_yaw, grasp yaw, 抓取角度优化, 抓取姿态学习]
solve_dls_ik, damped_least_squares_step, Jacobian gradient, dJ/dq, learn_grasp_yaw, grasp yaw, 抓取角度优化, 抓取姿态学习,
ResNet10Encoder, ResNet10, visual encoder, 视觉编码器]
paths: [topics/learning-layers/learning-layers.md]
source_of_truth:
- embodichain/compute/kinematics/dls.py
Expand Down Expand Up @@ -442,7 +443,7 @@ topics:
status: active
- id: data-pipeline
title: Data Pipeline
aliases: [online data, online dataset, async recorder, dataset persistence, simple BC demo, ODS documentation, ODS 文档, 数据管线, 在线数据, 数据集持久化, 数据保存,
aliases: [online data, online dataset, async recorder, dataset persistence, simple BC demo, ODS documentation, ODS capability showcase, ODS 能力展示, ODS 文档, 数据管线, 在线数据, 数据集持久化, 数据保存,
深度视频, 数据集校验, 数据集转换, 数据集划分]
keywords: [OnlineDataEngine, OnlineDataset, sample_batch, sample_batch_with_context, lock_index, LeRobotRecorder, AsyncLeRobotRecorder,
DepthSidecarManager, depth_meta.json, fragment_id, finalize, expert action schema, qpos_slice, qvel_slice,
Expand All @@ -468,6 +469,7 @@ topics:
- docs/source/features/online_data.md
- docs/source/features/ods/
- docs/architecture/dataset-loading-design.md
- docs/architecture/ods-data-stream-showcase-design.md
- embodichain/lab/gym/envs/managers/dataset_manager.py
- embodichain/lab/gym/envs/managers/datasets.py
- embodichain/lab/gym/envs/managers/async_datasets.py
Expand All @@ -479,9 +481,10 @@ topics:
- tools/lerobot_export/
- embodichain/lab/scripts/run_task.py
- examples/ods/simple_bc_demo/
- examples/ods/data_stream_demo/
watch_paths: [embodichain/data_pipeline/, embodichain/lab/gym/envs/managers/dataset_manager.py, embodichain/lab/gym/envs/managers/datasets.py,
embodichain/lab/gym/envs/managers/async_datasets.py, embodichain/lab/gym/envs/embodied_env.py, embodichain/lab/scripts/run_task.py,
tests/data_pipeline/, embodichain/cli/dataset.py, tools/lerobot_export/, examples/ods/simple_bc_demo/,
tests/data_pipeline/, embodichain/cli/dataset.py, tools/lerobot_export/, examples/ods/simple_bc_demo/, examples/ods/data_stream_demo/,
docs/source/features/online_data.md, docs/source/features/ods/, docs/architecture/dataset-loading-design.md]
related_topics: [manager-functor, env-framework, task-programs, sensor-system]
status: active
Expand Down
31 changes: 31 additions & 0 deletions agent_context/topics/data-pipeline/data-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ belongs to [data-assets](../data-assets/data-assets.md).
| Shared sequence contract and sources | `embodichain/data_pipeline/datasets/sequence.py`, `sources.py`, `_sampling.py` |
| Bounded numeric LeRobot loading | `embodichain/data_pipeline/datasets/lerobot.py`: `LeRobotDataSource`, explicit field mapping |
| Local Franka ChunkBC demo and offline comparisons | `examples/ods/simple_bc_demo/train.py`; local task/config/model/trainer/evaluator and evidence in the same directory |
| Pure delivery capability and standard video decoding comparison | `examples/ods/data_stream_demo/run.py`, `suite.py`, `report.py`; no learner, real SDK/DataLoader decode and parent-owned prefetch quotas |
| Fixed-budget visual training, diversity and total pipeline time | `examples/ods/data_stream_demo/train.py`, `training_metrics.py`, `training_report.py`; single/dual-camera BC, standard dataset recording and completed-training accounting; `training_throughput.py` owns three-stage preflight guidance |
| Dual-camera learning and camera review | `examples/ods/data_stream_demo/converge.py`, `preview.py`; `CONVERGENCE.md` owns the candidate recipe, development selection and one held-out test |
| Commit/discard/finalization dispatch | `embodichain/lab/gym/envs/managers/dataset_manager.py` |
| Synchronous LeRobot persistence | `embodichain/lab/gym/envs/managers/datasets.py`: `LeRobotRecorder` |
| Background persistence | `embodichain/lab/gym/envs/managers/async_datasets.py`: `AsyncLeRobotRecorder` |
Expand All @@ -38,6 +41,34 @@ Example-local documentation owns its task binding, CLI/artifact names,
comparison protocol and historical evidence. Keep generic operational contracts
in the core guides.

The [ODS capability-showcase design](../../../docs/architecture/ods-data-stream-showcase-design.md)
owns the fixed-budget single/dual-camera BC comparison and the historical
state/single/dual delivery diagnostic. Its example-local decoded reader is
separate from the public resident numeric reader. Follow it for offline corpus
sizes, source-reuse protection, camera qualification and timing boundaries.
Training comparisons match actual updates and valid supervision; data volume
and trained-state diversity are outcomes. Record only completed optimizer
updates as training coverage, and include recording/finalization and normal
teardown in total pipeline time. Serial trace replay separately attributes
generation/training overlap.
Budget-aware prefetch and three-stage capacity guidance remain example-local;
they do not change public engine/controller scheduling. The
[visual demo README](../../../examples/ods/data_stream_demo/README.md)
owns calibration and pilot commands, including isolated eager controls.
The visual runner can ingest complete producer subbatches before generation
completion. Its raw training/archive/serial owners share one C-episode bound;
serial controls still archive during generation. This example-local handoff
and nested SDK-stage profiling preserve the public sampling contract.
All acquisition controls may overlap recording and generation within the same
storage bounds. Worker SDK service timestamps remain separate from owner
copy/wait spans. `training_audit.py` combines full numeric checks, source-window
SDK reads and `video_integrity.py`'s all-frame software video count/integrity
check; fresh post-timing receipts preserve earlier evidence.
The [experiment evidence index](../../../examples/ods/data_stream_demo/EXPERIMENTS.md)
links immutable historical studies and their workload-specific limitations.
Generated run evidence is excluded from the source tree; it does not establish
training efficiency or policy performance.

`BatchSource.sample_context()` returns owned TensorDicts under a shared
`SequenceSpecCfg`; `SequenceDataset` supplies the common iteration interface.
The package's `schema.py` owns common semantic validation and schema fingerprints;
Expand Down
14 changes: 11 additions & 3 deletions agent_context/topics/learning-layers/learning-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,12 @@ These modules do not import learning or simulation. `SerialChainKinematics`
returns TCP poses and geometric Jacobians from one FK pass, retaining dJ/dq.

`learning/layers/` owns `KinematicsLayer` and `DifferentiableIKLayer` Torch
module interfaces. `learning/models/` owns general MLPs, feature normalization
and configuration-driven MLP construction. These components do not import RL,
module interfaces. `learning/models/` owns general MLPs, feature normalization,
configuration-driven MLP construction, and the trainable `ResNet10Encoder`.
The encoder is imported lazily so torchvision is required only by visual clients;
it consumes float32 NCHW RGB in `[0, 1]` and owns ImageNet normalization.
The ODS visual demo owns the shared-view/history BC head and task losses, not
the general encoder. These components do not import RL,
TensorDict or simulation. RL policy interfaces, actor-critic composition and
policy registration/builders belong to `learning/rl/policies/`; RL code consumes
the shared networks and normalizer. See [RL learning](../rl-learning/rl-learning.md)
Expand Down Expand Up @@ -57,7 +61,11 @@ part of the training/evaluation entry point. The demo does not execute physical
motion; keep task predictors out of production policy registries until a reusable
contract exists.

Tests live under `tests/compute/kinematics/` and `tests/learning/layers/`.
Tests live under `tests/compute/kinematics/`, `tests/learning/layers/`, and
`tests/learning/models/`. The visual encoder tests verify reference-structure
forward/gradient equivalence, image ownership and checkpoint round trips;
the ODS demo's `qualify_encoder.py` additionally verifies pinned RLInf classes,
causal multiview inputs, image-only fitting, and explicitly selected CUDA.
Validate FK, dJ/dq, full IK and recurrent-state gradients separately; include
nonzero TCP offsets, repeated solves, checkpoint migration and import boundaries. GPU validation runs serially with `--run-gpu`.
The unified kinematic benchmark's `differentiable` mode measures training and
Expand Down
6 changes: 6 additions & 0 deletions docs/architecture/dataset-loading-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ For use and operation, start with the [ODS guide](../source/features/online_data
This document owns source/Dataset architecture and implementation boundaries; task parameters
and historical run evidence remain with the demo.

The [ODS data-stream showcase](ods-data-stream-showcase-design.md)
uses standard SDK/DataLoader video decoding for its offline comparison. That
example-local reader is separate from the shipped resident numeric backend; its
visual results cannot be obtained by dropping images or preloading decoded
datasets through the current numeric loader.

## Problem and acceptance criterion

`SequenceDataset` supplies one PyTorch context-iteration interface for live and
Expand Down
Loading