Skip to content

Add RigidFormer engine demo and evaluation workflows - #890

Merged
yuecideng merged 5 commits into
codex/rigidformer-official-trainingfrom
codex/rigidformer-official-native
Oct 28, 2026
Merged

yuecideng merged 5 commits into
codex/rigidformer-official-trainingfrom
codex/rigidformer-official-native

Conversation

@yuecideng

@yuecideng yuecideng commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add an engine-rendered RigidFormer sphere demo to the learning/rigidformer showcase, alongside pinned official-asset preparation, fixed-weight evaluation, raw/EMA export, independent author comparison and recovery checks.

embodichain run-example learning/rigidformer demo displays dynamic Newton XPBD spheres on the left and render-only model spheres on the right. The model receives four engine observations and performs one free rollout. The demo supports a native window or headless engine-camera recording, configurable resource paths and object selection, and writes MP4, a preview frame, aligned trajectories and source-identified metrics. Frames stream to the encoder. The README includes resource preparation and runnable commands.

Stack

  • Layer: 3/3
  • Base: codex/rigidformer-official-training
  • Depends on: #889
  • Followed by: none

Type of change

  • New feature (non-breaking)

Validation

  • Official Sphere weights and verified local MOVi-Sphere geometry: headless 8-frame smoke and default 51-frame collision/landing run; native-window path under Xvfb with six frames; two-object, five-frame smoke. All completed with cleanup.
  • Default output verified as 960x540, 51 frames at 12 fps. Trajectory checks verify finite predictions, aligned shapes, observed warmup agreement and reported center errors.
  • 39 existing showcase, example packaging and launcher tests pass. Demo help remains simulator-free; syntax checks pass. No unit tests of demo internals were added; the new example is exercised through the real CLI and engine.
  • After synchronizing the Train official RigidFormer through shared SL #889 fixes, 20 focused model/SL checks pass (three GPU cases skipped in that CPU invocation). The inherited changes have their own CPU/Gloo/CUDA/NCCL validation in Train official RigidFormer through shared SL #889.
  • Black 26.10.1 (black .), API coverage 3287/3287, context validation and 42 context tests pass. Neural-dynamics context and the example index were updated; simulator lifecycle and launcher guidance remain accurate.
  • Pinned model parity, installed-wheel and fixed-weight evaluation evidence is recorded in the acceptance protocol and qualification index. The engine demo uses new initial conditions and Newton physics; its errors describe that visual comparison. Full published-benchmark evaluation and full-budget retraining remain separate.

Checklist

  • Ran black . with Black 26.10.1.
  • Reviewed affected documentation and agent context and updated the owning guidance.
  • Public API documentation coverage passes.
  • Verified proportional behavioral coverage and reviewed test necessity.
  • The new example is preserved by the existing example packaging path; it uses existing engine and RigidFormer dependencies.

@yuecideng
yuecideng added this pull request to stack #891 October 27, 2026 14:49
@yuecideng yuecideng added benchmark Benchmark protocols, suites, metrics, evaluation tools and technical reports. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. docs Documentation, API references and example explanations. feature New user-facing capabilities. model Neural model architectures, inference, weights and model contracts. sl Supervised learning updates, training, recovery and behavior cloning workflows. labels Oct 27, 2026
@yuecideng
yuecideng force-pushed the codex/rigidformer-official-native branch 2 times, most recently from 2044b2a to 6ecec1c Compare October 28, 2026 01:37
@yuecideng
yuecideng force-pushed the codex/rigidformer-official-native branch from 6ecec1c to 59bb5e7 Compare October 28, 2026 01:38
@yuecideng yuecideng changed the title Add RigidFormer asset and numerical validation workflows Add RigidFormer engine demo and evaluation workflows Oct 28, 2026
@yuecideng yuecideng added the visualization Browser viewers, markers, trajectory previews and interactive debugging displays. label Oct 28, 2026
@yuecideng
yuecideng marked this pull request as ready for review October 28, 2026 16:36
@yuecideng
yuecideng merged commit c8875fe into main Oct 28, 2026
11 of 12 checks passed
@yuecideng
yuecideng deleted the codex/rigidformer-official-native branch October 28, 2026 16:36
@greptile-apps

greptile-apps Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The changes add operator-run workflows around RigidFormer without changing general simulator behavior.

The PR appears safe to merge, with non-blocking improvements to raw-data checks and report reliability.

Fix All in CodexFindings

  1. P2 Unlisted shards bypass checksum checks ▶
  2. P2 Optional metadata can block reports ▶
Fix with agent prompt
### Issue 1
examples/learning/rigidformer/_prepare.py:126-133
`prepare` discovers every local Parquet shard but checks hashes only for paths listed in the official manifest. An extra shard in a reused cache can therefore supply scenes without an official checksum check. Its converted contents then receive the pinned `data_revision`.

Before creating the output directory, reject selected shards missing from the manifest or select shards from the manifest itself.

### Issue 2
examples/learning/rigidformer/_support/_evidence.py:66-69
If Torch supports NCCL but `libnccl.so.2` cannot be loaded by name, `evidence` raises `OSError`. The Torch build check does not guarantee that this separate lookup succeeds. Even a completed single-GPU evaluation then loses its summary.

Catch the lookup failure and record an unavailable runtime version instead of failing the experiment.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds the RigidFormer showcase with asset preparation, shared-training recipes, weight export, fixed-weight evaluation, author comparison and recovery checks.

  • The RigidFormer showcase offers commands for its main workflows.
  • A side-by-side demo shows Newton spheres and a RigidFormer rollout.
  • Pinned MOVi assets become prepared data for RigidFormer workflows.
  • Fixed-checkpoint evaluation reports RigidFormer results with their inputs.
  • Comparison and recovery commands check RigidFormer training against references.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Pinned assets] --> B[Prepare verified geometry]
  B --> C[Shared SL training]
  C --> D[Export raw or EMA weights]
  D --> E[Fixed-weight evaluation]
  B --> F[Newton sphere demo]
  D --> F
  F --> G[Four observed frames]
  G --> H[One free model rollout]
  H --> I[Video and trajectory report]
Loading

Reviews (1) · Last reviewed commit: "style(examples): trim RigidFormer file e..." · Reviewed by Greptile

Comment on lines +126 to +133
selected = sorted((raw / "data" / dataset).glob("*.parquet"))
if not selected:
raise FileNotFoundError("No official numeric scenes found")
selected_names = {str(p.relative_to(raw)) for p in selected}
for rel, info in needed.items():
if not limit_per_split or rel.startswith("assets/") or rel in selected_names:
if not (raw / rel).is_file() or sha256(raw / rel) != info["sha256"]:
raise ValueError(f"Official raw data checksum differs: {rel}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unlisted shards bypass checksum checks

prepare discovers every local Parquet shard but checks hashes only for paths listed in the official manifest. An extra shard in a reused cache can therefore supply scenes without an official checksum check. Its converted contents then receive the pinned data_revision.

Before creating the output directory, reject selected shards missing from the manifest or select shards from the manifest itself.

Prompt To Fix With AI
This is a comment left during a code review.
Path: examples/learning/rigidformer/_prepare.py
Line: 126-133

Comment:
**Unlisted shards bypass checksum checks**

`prepare` discovers every local Parquet shard but checks hashes only for paths listed in the official manifest. An extra shard in a reused cache can therefore supply scenes without an official checksum check. Its converted contents then receive the pinned `data_revision`.

Before creating the output directory, reject selected shards missing from the manifest or select shards from the manifest itself.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

Comment on lines +66 to +69
if torch.distributed.is_nccl_available():
loaded = ctypes.c_int()
ctypes.CDLL("libnccl.so.2").ncclGetVersion(ctypes.byref(loaded))
nccl = [loaded.value // 10000, loaded.value % 10000 // 100, loaded.value % 100]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Optional metadata can block reports

If Torch supports NCCL but libnccl.so.2 cannot be loaded by name, evidence raises OSError. The Torch build check does not guarantee that this separate lookup succeeds. Even a completed single-GPU evaluation then loses its summary.

Catch the lookup failure and record an unavailable runtime version instead of failing the experiment.

Prompt To Fix With AI
This is a comment left during a code review.
Path: examples/learning/rigidformer/_support/_evidence.py
Line: 66-69

Comment:
**Optional metadata can block reports**

If Torch supports NCCL but `libnccl.so.2` cannot be loaded by name, `evidence` raises `OSError`. The Torch build check does not guarantee that this separate lookup succeeds. Even a completed single-GPU evaluation then loses its summary.

Catch the lookup failure and record an unavailable runtime version instead of failing the experiment.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark Benchmark protocols, suites, metrics, evaluation tools and technical reports. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. docs Documentation, API references and example explanations. feature New user-facing capabilities. model Neural model architectures, inference, weights and model contracts. sl Supervised learning updates, training, recovery and behavior cloning workflows. visualization Browser viewers, markers, trajectory previews and interactive debugging displays.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant