Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
63beff7
feat(dynamics): separate NeRD models, data generation and training
yuecideng Oct 24, 2026
a58de92
docs(dynamics): register the offline workflow in architecture navigation
yuecideng Oct 24, 2026
cc1a67e
feat(sl): add shared supervised batch and optimizer update contracts
yuecideng Oct 24, 2026
3e7a338
feat(sl): add shared supervised batch and optimizer update contracts
yuecideng Oct 24, 2026
dd52660
feat(sl): add shared supervised batch and optimizer update contracts
yuecideng Oct 24, 2026
8408d9f
feat(sl): add offline run recovery and recipe-based train-sl CLI
yuecideng Oct 24, 2026
11142d5
test(sl): register the supervised learning context domain
yuecideng Oct 24, 2026
a002fea
feat(sl): add offline run recovery and recipe-based train-sl CLI
yuecideng Oct 24, 2026
9430346
Merge standalone NeRD foundation for the supervised learning stack
yuecideng Oct 24, 2026
77869ee
fix(sl): accept argument lists from the unified CLI dispatcher
yuecideng Oct 24, 2026
f06dd79
feat(sl): integrate optional RigidFormer and qualify reference training
yuecideng Oct 24, 2026
3af58e9
Merge shared SL foundation into NeRD integration
yuecideng Oct 24, 2026
95b7506
Merge shared SL foundation and standalone NeRD dependency
yuecideng Oct 24, 2026
736a127
test(sl): isolate supervised test modules from dynamics tests
yuecideng Oct 24, 2026
1c1441f
feat(sl): integrate NeRD fitting and reference acceptance
yuecideng Oct 24, 2026
00b6a56
Merge branch 'codex/sl-foundation' into codex/sl-nerd
yuecideng Oct 24, 2026
1395871
fix(sl): isolate metadata and validate optimizer recovery before muta…
yuecideng Oct 24, 2026
3ebb4e2
Merge branch 'codex/sl-foundation' into codex/sl-rigidformer
yuecideng Oct 24, 2026
d5cc9d7
docs(sl): expose optional RigidFormer setup and public contracts
yuecideng Oct 24, 2026
23b4519
Merge branch 'codex/sl-foundation' into codex/sl-nerd
yuecideng Oct 24, 2026
8ed70f1
docs(sl): link the standalone NeRD training integration
yuecideng Oct 24, 2026
6e5a861
fix(sl): bind scheduler and tensor-tree semantics during recovery
yuecideng Oct 24, 2026
e1b3cb1
Merge branch 'codex/sl-foundation' into codex/sl-nerd
yuecideng Oct 24, 2026
4ec19fc
fix(sl): preserve prior experiment artifacts when starting or resumin…
yuecideng Oct 24, 2026
e94acb6
fix(sl): allow owned initial-policy snapshots before fitting
yuecideng Oct 24, 2026
4fc00c0
Merge finalized SL recovery guards into RigidFormer integration
yuecideng Oct 24, 2026
d591e78
fix(sl): reject duplicate NeRD acceptance seeds before startup
yuecideng Oct 25, 2026
3cce8f8
fix(sl): strengthen RigidFormer reference qualification
yuecideng Oct 25, 2026
8fe2341
fix(sl): weight rigid-body losses by valid object count
yuecideng Oct 25, 2026
a7668ad
Add shared SL updates, offline training and recovery (#857)
yuecideng Oct 25, 2026
9fd69c7
Merge latest main into NeRD and SL foundations
yuecideng Oct 25, 2026
31e2b90
Merge latest main before NeRD training hardening
yuecideng Oct 25, 2026
f03af34
Integrate NeRD with the shared supervised learner
yuecideng Oct 25, 2026
6dc47c3
fix(sl): preserve run identity, best models and NeRD recovery
yuecideng Oct 25, 2026
d9cd707
docs(sl): render NeRD adapters with explicit API directives
yuecideng Oct 25, 2026
2e99302
fix(nerd): validate statistics and preserve unnormalized model precision
yuecideng Oct 25, 2026
bed4b69
Merge latest main while preserving NeRD precision fixes
yuecideng Oct 25, 2026
60425ca
Merge latest main before moving NeRD out of the core
yuecideng Oct 25, 2026
048d147
refactor(models): move NeRD into extension and downloadable showcase
yuecideng Oct 25, 2026
fa16c60
Merge latest main after NeRD model extension migration
yuecideng Oct 25, 2026
60f9b07
fix(nerd): retain experiment root for comparison receipts
yuecideng Oct 25, 2026
2c9ee88
fix(nerd): repair asset downloads and remove dedicated unit tests
yuecideng Oct 25, 2026
f850a5e
docs(models): define policy and dynamics domain boundaries
yuecideng Oct 25, 2026
ddb7c59
Merge shared model and SL training base into RigidFormer
yuecideng Oct 25, 2026
5e0b3b1
Move RigidFormer into the official dynamics model family
yuecideng Oct 25, 2026
9b53fa7
Bound RigidFormer distances at autocast output precision
yuecideng Oct 25, 2026
c29b2c6
Use a verified index dependency for optional RigidFormer
yuecideng Oct 25, 2026
b5f4bf8
Merge main into RigidFormer model branch
yuecideng Oct 25, 2026
b1d8ab4
test: align robot catalog checks with vendor layout
yuecideng Oct 25, 2026
ac69eb8
Include vendor catalog test compatibility for main CI
yuecideng Oct 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -270,10 +270,11 @@ topics:
status: active
- id: neural-dynamics
title: Neural Dynamics Models and Training
aliases: [neural dynamics, neural robot dynamics, NeRD, 神经动力学, 动力学模型训练, NeRD 示例]
aliases: [neural dynamics, neural robot dynamics, NeRD, 神经动力学, 动力学模型训练, NeRD 示例, RigidFormer, RigidFormer 示例]
keywords: [NeRDStateCodec, NeRDModel, DynamicsGenerationCfg,
DynamicsShardWriter, DynamicsWindowDataset, DynamicsTrainer, embodichain_models,
learning/nerd, nerd reference assets,
learning/nerd, nerd reference assets, RigidFormerCfg, RigidFormerModel,
RigidBodySequenceSource, build_rigidformer_trainer, learning/rigidformer,
官方训练精度对比]
paths: [topics/neural-dynamics/neural-dynamics.md]
source_of_truth:
Expand All @@ -282,9 +283,13 @@ topics:
- examples/learning/nerd/_resources.py
- examples/learning/nerd/_support/
- examples/learning/nerd/configs/
- extensions/embodichain_models/embodichain_models/dynamics/rigidformer/
- examples/learning/rigidformer/
watch_paths: [extensions/embodichain_models/, examples/learning/nerd/,
examples/learning/rigidformer/, tests/learning/models/test_rigidformer.py,
tests/learning/models/test_rigidformer_data.py, tests/examples/rigidformer_reference/,
docs/source/features/neural_dynamics.md,
docs/source/api_reference/model_catalogs.rst]
docs/source/api_reference/model_catalogs.rst, docs/source/api_reference/model_rigidformer.rst]
related_topics: [supervised-learning, learning-layers, data-pipeline, data-assets]
status: active
- id: learning-layers
Expand Down
1 change: 1 addition & 0 deletions agent_context/topics/data-pipeline/data-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ belongs to [data-assets](../data-assets/data-assets.md).
| File-only validation and grouped quality splits | `embodichain/data_pipeline/datasets/inspection.py`; `embodichain dataset validate\|split` |
| Durable commit diagnostics and metadata repair | `embodichain/data_pipeline/recording/journal.py`; `embodichain dataset recover` |
| NeRD-specific HDF5 windows and teacher collection | `extensions/embodichain_models/embodichain_models/dynamics/nerd/data.py` and `examples/learning/nerd/_support/_generation.py`; contracts in [neural dynamics](../neural-dynamics/neural-dynamics.md) |
| RigidFormer aligned geometry windows | `extensions/embodichain_models/embodichain_models/dynamics/rigidformer/data.py`; model ownership in [neural dynamics](../neural-dynamics/neural-dynamics.md) |
| Isolated LeRobot 0.6.1 language/depth conversion | `tools/lerobot_export/` (Python >=3.12, independent dependencies) |

Read [online sampling](online-sampling.md) for worker states, refill, shared
Expand Down
8 changes: 4 additions & 4 deletions agent_context/topics/learning-layers/learning-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ returns TCP poses and geometric Jacobians from one FK pass, retaining dJ/dq.
`learning/layers/` owns `KinematicsLayer` and `DifferentiableIKLayer` Torch
module interfaces. `learning/models/` owns general MLPs, feature normalization
and configuration-driven MLP construction. These shared components do not import
RL, TensorDict or simulation. Concrete NeRD models, codecs and the shared-SL
recipe live in the bundled `embodichain_models.dynamics.nerd` extension;
teacher collection, historical training and research belong to
`examples/learning/nerd/`. See
RL, TensorDict or simulation. Concrete NeRD and RigidFormer models, model-specific codecs/data and shared-SL
recipes live in the bundled `embodichain_models.dynamics` families;
teacher collection, historical training and research belong to the corresponding
`examples/learning/` showcase. See
[neural dynamics](../neural-dynamics/neural-dynamics.md) for model, SL integration,
dataset and checkpoint ownership. RL policy interfaces, actor-critic composition and
policy registration/builders belong to `learning/rl/policies/`; RL code consumes
Expand Down
25 changes: 22 additions & 3 deletions agent_context/topics/neural-dynamics/neural-dynamics.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ runtime registry. Shared MLPs and feature normalization stay in core

The [model catalog contract](../../../extensions/embodichain_models/README.md)
owns domain and public-import naming. Policy models predict actions; dynamics
models predict future state or geometry. RWM and RigidFormer follow-ups use
`dynamics.rwm` and `dynamics.rigidformer`; only `dynamics.nerd` is implemented here.
models predict future state or geometry. NeRD and RigidFormer are implemented
at `dynamics.nerd` and `dynamics.rigidformer`; the RWM follow-up uses `dynamics.rwm`.
World-model naming does not introduce a parallel `world_models` domain.

| Request | Owner |
Expand All @@ -38,7 +38,26 @@ Scene API. Native learned-solver deployment and functional Gym integration are
separate consumers. There are no NeRD-specific core `compute/dynamics`,
`data_pipeline/dynamics`, `learning/dynamics` modules or core `dynamics` command.

## Resource and physical identity
## RigidFormer geometry model

`extensions/embodichain_models/embodichain_models/dynamics/rigidformer/` owns
`model.py`, aligned CPU windows in `data.py`, objective/batch conversion in
`supervised.py` and the installable
`sl_recipe:build_rigidformer_trainer` factory. Package imports stay lazy; only
model construction loads the pinned optional upstream dependency. Core SL and
generic data-pipeline modules do not import this family. The model has no action
input (`action_conditioned=False`) and does not qualify action gradients or APG.

`embodichain run-example learning/rigidformer` runs the private Box2D comparison
under the example's `_support/`. The [example guide](../../../examples/learning/rigidformer/README.md)
owns source/data provenance, finite-padding and pairwise-memory boundaries; the
[frozen protocol](../../../examples/learning/rigidformer/research/README.md) owns
reference seeds, budgets and acceptance gates. Existing tests in
`tests/learning/models/test_rigidformer*.py` cover the production geometry,
weighted loss/gradient, masking, budget and exact recovery contracts. New runs
bind executing source hashes; historical reports retain their original identity.

## NeRD resource and physical identity

`prepare --robot cartpole|ant --resource-root DIR` downloads the pinned dataset,
portable model and raw reference assets. Revisions and SHA256 checks are owned by
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,9 @@ operations remain in `compute`; source storage belongs to data adapters.
- `embodichain train-sl --config FILE [--resume FILE]` lazily resolves a
`factory: module:function`. The factory receives the full model/data/objective/
optimizer/trainer/evaluation configuration and returns a `SupervisedTrainer`.
- The [NeRD recipe](../neural-dynamics/neural-dynamics.md) is an installable factory
at `embodichain_models.dynamics.nerd.sl_recipe:build_supervised_trainer`.
The core does not import the model extension or example helpers. The NeRD
showcase's default `train` command uses this same common recipe.
- [Model-specific factories](../neural-dynamics/neural-dynamics.md) are installable
from their `embodichain_models` family packages. The core does not import the
model extension or example helpers; model recipes return the common trainer.

Objectives own loss masks, normalization, integration and multi-step graph
construction. The optimizer does not infer action semantics or detach histories.
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture/curated.json
Original file line number Diff line number Diff line change
Expand Up @@ -3342,13 +3342,13 @@
},
{
"id": "neural-dynamics",
"label": "NeRD model extension",
"label": "Official dynamics models",
"kind": "package",
"topic_ids": [
"neural-dynamics"
],
"summary": "Bundles the causal NeRD network, physical codec, trajectory windows and supervised recipe in the main wheel.",
"details": "embodichain_models.dynamics.nerd owns the concrete model, portable checkpoints, physical representations, HDF5 windows and installable SL recipe. Generic MLP/normalization and supervised orchestration stay in core. The model extension imports neither the showcase nor simulation; collection, historical training and reference research belong to the example.",
"summary": "Bundles NeRD and optional RigidFormer models, model-specific data adapters and shared-SL recipes in the main wheel.",
"details": "embodichain_models.dynamics.nerd owns portable neural dynamics and HDF5 windows; dynamics.rigidformer owns pinned geometry prediction, aligned point windows and the valid-object weighted objective. Both provide installable shared-SL factories. Generic MLP/normalization and supervised orchestration stay in core. The model extension imports neither the showcase nor simulation; collection, historical training and reference research belong to the example.",
"boundaries": [
"The core does not import concrete model implementations.",
"Data/model/reference-asset payloads are external resources; numerical and physical identities remain explicit."
Expand Down
12 changes: 12 additions & 0 deletions docs/source/api_reference/model_catalogs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -252,3 +252,15 @@ working directory and example-runner bootstrap.
:members: sample

.. autofunction:: build_supervised_trainer

RigidFormer geometry prediction
-------------------------------

The optional ``embodichain_models.dynamics.rigidformer`` family owns aligned
geometry windows, its pinned upstream adapter, valid-object weighted objective
and installable SL recipe. It predicts geometry without action inputs.

.. toctree::
:maxdepth: 1

model_rigidformer
114 changes: 114 additions & 0 deletions docs/source/api_reference/model_rigidformer.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
RigidFormer geometry dynamics
=============================

The lazy optional adapter binds the pinned RigidFormer implementation to causal
point-cloud windows and shared supervised updates. Reference geometry and point
correspondence are explicit. The model has no action input and does not qualify
action gradients, APG or a native physics-solver interface. Install the
``rigidformer`` extra for this family. The Box2D comparison belongs to the
``learning/rigidformer`` example and retains its frozen protocol and provenance.

Public entry points
-------------------

.. currentmodule:: embodichain_models.dynamics.rigidformer

.. autosummary::

RIGIDFORMER_REVISION
RigidFormerCfg
RigidFormerModel
RigidBodySequenceSource
RigidFormerObjective
rigidformer_batch
build_rigidformer_trainer

.. autodata:: RIGIDFORMER_REVISION

.. autoclass:: RigidFormerCfg

.. autoclass:: RigidFormerModel

.. autoclass:: RigidBodySequenceSource

.. autoclass:: RigidFormerObjective

.. autofunction:: rigidformer_batch

.. autofunction:: build_rigidformer_trainer

Aligned geometry windows
------------------------

The source owns a bounded CPU corpus and returns independent previous, current,
reference and next-frame point arrays without crossing episode boundaries.
The caller owns sampling RNG; material point identities and finite object/point
lengths remain explicit.

.. currentmodule:: embodichain_models.dynamics.rigidformer.data

.. autosummary::

RigidBodySequenceSource

.. autoclass:: RigidBodySequenceSource
:members: get_batch, sample_batch

Optional network adapter
------------------------

The optional upstream model loads at construction. Parameter keys and numerical
behavior follow the pinned upstream revision. The per-call budget bounds the
dense nearest-neighbour distance intermediate. Unequal point lengths require a
padded point axis of at most 128 for forward and rollout.

.. currentmodule:: embodichain_models.dynamics.rigidformer.model

.. autosummary::

RIGIDFORMER_REVISION
RigidFormerCfg
RigidFormerModel

.. autodata:: RIGIDFORMER_REVISION

.. autoclass:: RigidFormerCfg
:members:

.. autoclass:: RigidFormerModel
:members: forward, rollout

Objective and causal batch conversion
-------------------------------------

The objective preserves the upstream position/acceleration losses. Effective
weight is the sum of valid objects, including when scenes have different object
counts. Targets remain separate from causal model inputs.

.. currentmodule:: embodichain_models.dynamics.rigidformer.supervised

.. autosummary::

RigidFormerObjective
rigidformer_batch

.. autoclass:: RigidFormerObjective
:members: compute_loss

.. autofunction:: rigidformer_batch

Installable supervised recipe
-----------------------------

Use ``factory:
embodichain_models.dynamics.rigidformer.sl_recipe:build_rigidformer_trainer``
with ``embodichain train-sl``. The recipe samples fixed tensor-only archives and
returns the common trainer with source/content/configuration recovery identity.

.. currentmodule:: embodichain_models.dynamics.rigidformer.sl_recipe

.. autosummary::

build_rigidformer_trainer

.. autofunction:: build_rigidformer_trainer
5 changes: 5 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ embodichain run-example --list
embodichain run-example sim/motion/planners/curobo_planner --help
embodichain run-example ods/simple_bc_demo/train --help
embodichain run-example learning/nerd --help
embodichain run-example learning/rigidformer --help
embodichain run-example sim/atomic_action --skill place --variants 8
embodichain run-example sim/atomic_action/coordinated_pickment --help
```
Expand Down Expand Up @@ -42,3 +43,7 @@ The [Atomic Skill example catalog](sim/atomic_action/README.md) owns all per-ski
coordinated-manipulation and recovery demonstrations. The two source-checkout
walkthroughs under `scripts/tutorials/atomic_action/` teach single-arm and dual-arm
API composition; they reuse the canonical example setup.

The [RigidFormer showcase](learning/rigidformer/README.md) owns the pinned Box2D
reference comparison. Its installable `dynamics.rigidformer` model, geometry
source and objective use the shared SL trainer for fixed-corpus training.
64 changes: 64 additions & 0 deletions examples/learning/rigidformer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# RigidFormer geometry prediction

The installable model family is `embodichain_models.dynamics.rigidformer`. It
wraps `lucidrains/rigidformer` at the pinned revision
`4ab4edac7271b49484748d6bf8086a497f13272d` and trains through the shared
`embodichain.learning.sl` contracts. Install `embodichain[rigidformer]` only when
using this model. The upstream model and Box2D reference recipe retain the
complete MIT license in the model package's `THIRD_PARTY_LICENSES`.

The ordinary optional extra uses the public-index pin `rigidformer==0.2.0`,
without a Git dependency in EmbodiChain's published metadata:

```bash
python -m pip install 'embodichain[rigidformer]'
```

All four runtime Python files in the [PyPI 0.2.0 wheel](https://pypi.org/project/rigidformer/0.2.0/)
were verified byte-for-byte against the frozen commit above. This establishes
that distribution's runtime-source equivalence; the version number alone does
not qualify a reference environment. For exact Git installation in an isolated
`py311` environment, use [requirements-reference.txt](requirements-reference.txt)
and the [reference setup](research/README.md). The acceptance runner still
checks the checkout and installed Python bytes against the pinned Git blobs.

For a fixed tensor-only PyTorch geometry archive, use the installable factory:

```bash
embodichain train-sl --config examples/learning/rigidformer/configs/sl.train.yaml
embodichain train-sl --config my-recipe.yaml --resume outputs/rigidformer/sl/last.pt
```

Set `data.path` in the configuration to your archive. Paths resolve from the
caller's working directory. Its fields are `positions[E,T,O,P,3]`,
`properties[E,O,A]` (or `[E,O,P,A]`) and positive `step_dt` in seconds. Optional
`episode_lengths`, `object_lens` and `object_point_lens` describe finite padding.
Point identities remain aligned across frames, and the owned previous-frame
geometry is the reference for each sampled window. Sampling preserves episode
boundaries and explicit resident/batch byte budgets.

`RigidFormerModel.action_conditioned` is `False`. The adapter exposes future
geometry prediction and supervised parameter training. Action-gradient/APG and
native physics-solver deployment are outside its qualified capabilities. The
dense nearest-neighbour buffer is checked against `max_pairwise_bytes`; this
is a bound on that intermediate alone. Unequal point lengths require padded
`P <= 128`, including at rollout. Supplied point counts must be at least
`num_anchors`, including placeholder objects; fixed anchors index valid points.
The objective weights each scene contribution by its number of valid objects.

The reference experiment is an example, with private support outside the
installable model package:

```bash
embodichain run-example learning/rigidformer --help
embodichain run-example learning/rigidformer \
--reference /tmp/rigidformer-reference \
--output /tmp/rigidformer-acceptance --threads 4
```

See the [frozen reference protocol](research/README.md) for source/data checks,
friction/collision modes, seeds, budgets and acceptance thresholds. Every new
manifest records the executing adapter, SL and runner source hashes. Historical
reports remain bound to their original source and data; relocation does not
change or requalify that evidence. Downloaded data, upstream checkouts, weights
and generated reports remain outside Git and the wheel.
22 changes: 22 additions & 0 deletions examples/learning/rigidformer/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------


"""RigidFormer reference showcase."""

from __future__ import annotations

__all__: list[str] = []
26 changes: 26 additions & 0 deletions examples/learning/rigidformer/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

"""Run the pinned RigidFormer reference acceptance experiment."""

from __future__ import annotations

__all__: list[str] = []

if __name__ == "__main__":
from ._support._acceptance import _main

_main()
Loading
Loading