ART-Embodied

Trajectory-aware reinforcement learning for Physical AI, built on OpenPipe ART and designed for LeRobot workflows.

日本語 · 한국어 · 简体中文 · 繁體中文 · Embodied RL guide · Examples · Release validation · Upstream ART docs

Warning

ART-Embodied is a research preview. OpenVLA-OFT action-token GRPO has passed real LIBERO rollout, distributed LoRA training, checkpoint reload, and fixed evaluation. GSPO has passed the distributed process and checkpoint lifecycle gate. Its scorer-aligned objective gate and multi-update success-rate comparison are still in progress. PI0 and PI0.5 Flow-SDE GRPO have produced positive fixed-development-evaluation lift, but remain experimental until replicated across training seeds and a sealed test. SmolVLA is not yet a validated training path.

What it adds

LeRobot already provides policies, processors, datasets, and robot environments. ART-Embodied adds the experiment control plane needed to improve a policy from complete embodied rollouts:

LeRobot policy + processors + environment
                   │
                   ▼
       grouped trajectories and rewards
                   │
                   ▼
      trajectory/action-token GRPO or GSPO
                   │
                   ▼
       versioned LoRA checkpoints
                   │
                   ▼
 fixed evaluation + W&B Models + Weave

The success criterion is better behavior under the policy's native evaluator, not merely a lower surrogate loss.

ART lifecycle, LeRobot ownership

ART-Embodied uses the ART training lifecycle without forcing a robot policy into ART's text-generation interface. The native policy, processors, environment, and action sampler remain ordinary LeRobot objects. The embodied control plane owns grouping, training, step and checkpoint lineage, evaluation, and logging.

Contract	Implementation
Model identity and lifecycle	EmbodiedTrainableModel, an actual art.TrainableModel
Training call	backend.train(model, trajectory_groups, learning_rate=...)
Group collection	ART-style trajectory_group(...) and gather_trajectory_groups(...)
Results and step tracking	ART TrainResult and model-owned get_step()
Experiment logging	Model-owned logging with embodied W&B video and Weave trace rendering
Policy execution	Native LeRobot processors and sampler; no fake chat conversion

Embodied trajectories and policy-gradient backends remain specialized because images, robot states, action chunks, and sampler-specific likelihoods do not fit the text-trajectory contract honestly. The current OpenVLA backend therefore does not run through upstream ART LocalBackend, AOM, or Serverless Training. Those systems are architectural reference points and future integration targets, not current runtime dependencies.

There are two views of the same runtime: a LeRobot-first high-level helper and an ART-style lower-level lifecycle. The high-level helper is not a second training implementation; it registers the same embodied model and backend internally.

Validated results

All validated model-family results use the same comparison format:

Policy / suite	Objective	Update	SFT	ART-Embodied	Paired lift
OpenVLA-OFT / LIBERO Object	Action-token GRPO	200	34/100	100/100	+66 points
OpenVLA-OFT / LIBERO Spatial	Action-token GRPO	100	48/100	88/100	+40 points
PI0 / LIBERO Spatial	Flow-SDE GRPO	100	63/100	99/100	+36 points
PI0.5 / LIBERO Long	Flow-SDE GRPO	250 (best dev)	48/100	84/100	+36 points

Each row evaluates the SFT and trained policy on the same 100 fixed, generated-state development scenarios. The sets are state-held-out from the training reset bank, but are not sealed final tests or multi-seed claims.

For OpenVLA-OFT Object, the rank-32/alpha-32 LoRA checkpoint at update 200 matched the public RLinf GRPO checkpoint at 100/100.

The 100 initial states were generated independently and do not overlap the 500 states sampled during training. Pairing found 66 SFT-to-ART improvements and no regressions (exact McNemar p=2.71e-20). This establishes generalization to new initial states under the same ten tasks and instructions. The manifest has been used during development, so it is a fixed state-held-out development set rather than a sealed final test. It is not a held-out language or held-out-task claim. RLinf is used as a positive-control specification and evaluation oracle. The normal ART-Embodied runtime uses model_loader: native; RLinf is not a runtime dependency.

LIBERO Spatial is the second-task regression. It reuses the generic OpenVLA backend without core-code changes, but uses a different SFT checkpoint, task suite, fixed evaluation set, and W&B project. On a separately generated and frozen set of 100 initial states, update 100 improved SFT from 48/100 to 88/100: 43 paired improvements, three regressions, exact McNemar p=4.62e-10. Update 30 scored 82/100 on the same states. As with Object, repeated development use makes this a state-held-out development manifest, not a sealed final test. The Spatial acceptance run did not use the public RLinf-trained checkpoint as an oracle or import an RLinf source checkout at runtime.

The PI Flow-SDE path has now established a second policy-family result. With the RLinf-reference K4/noise-0.5 sampler and 1,024 trajectories per update:

For PI0, all 36 discordant pairs improved and none regressed (paired 95% CI [+27,+46] points; exact McNemar p=2.91e-11). Later checkpoints through update 130 remained at 97--99/100. The full metric history, media, model artifacts, and trajectory traces are in the public PI0 W&B run. For PI0.5 on LIBERO Long, the rank-32/alpha-32 LoRA run completed 300 updates. Its best development checkpoint scored 84/100 at update 250, with 38 paired improvements and two regressions (paired 95% CI [+26,+46] points; exact McNemar p=1.49e-9). The final update 300 checkpoint scored 81/100. The full metric history, media, model artifacts, and trajectory traces are in the public PI0.5 W&B run. These evaluations use fixed generated-state development manifests, not sealed tests or multi-seed claims. Because the PI0.5 manifest was evaluated repeatedly, 84/100 is explicitly a best-development-checkpoint result rather than an unbiased final estimate. Exact recipes and hash-verified manifests are included under examples/embodied/.

Current support

Capability	Status
OpenVLA-OFT action-token GRPO	Validated research path
OpenVLA-OFT action-token GSPO	Process/checkpoint gate passed; scorer-aligned objective validation in progress
OpenVLA-OFT LIBERO Spatial regression	Validated: generated-state SFT 48/100 to ART 88/100
LeRobot Gymnasium rollout adapter	Implemented
Fixed and paired evaluation	Implemented
Single-GPU and local multi-GPU LoRA training	Implemented
Batched shared-policy rollout	Implemented
Checkpoint/resume with optimizer and RNG state	Implemented
W&B metrics, videos, Tables, and Models artifacts	Implemented
Weave nested trajectory traces and file-backed video	Implemented
SmolVLA flow matching	Research only
pi0/pi0.5 Flow-SDE GRPO	Single-run validation: PI0 Spatial 63/100 to 99/100; PI0.5 Long 48/100 to best-dev 84/100 (final 81/100). Multi-seed/sealed validation pending

One GPU is a supported resource profile

ART-Embodied does not require dedicated rollout, training, and evaluation GPUs. Set both device lists to the same accelerator and use a non-persistent rollout lifecycle. The runtime then executes initial evaluation -> train rollout -> training -> periodic evaluation serially, releasing the rollout replica before the optimizer takes ownership of the device:

runtime:
  rollout_devices: [cuda:0]
  training_devices: [cuda:0]
  distributed_training: false
  rollout_execution:
    lifecycle: per_update

per_update minimizes persistent GPU and host-memory use by terminating rollout workers at each phase boundary. cpu_offload retains workers and trades more host RAM for lower startup overhead. Multi-GPU profiles are throughput optimizations, not an installation requirement. The complete PI0.5 example is pi05_libero_object_flow_sde_grpo_single_gpu.yaml.

Install

Validated compatibility

ART-Embodied builds as an independent add-on. It owns only the art_embodied package and declares OpenPipe ART as an exact dependency. The compatibility range remains intentionally narrow while additional policy families are brought through the same accelerator and learning-quality gates.

ART-Embodied code	OpenPipe ART	LeRobot	Validated policy path
0.1.0rc1; current 0.1.0rc2.dev0	0.5.18 / upstream 828b839b	>=0.4.4,<0.5	OpenVLA-OFT action-token LoRA GRPO

The package boundary is pip install openpipe-art art-embodied with import art and import art_embodied as embodied. It will not install a second distribution into the art namespace. See the add-on migration milestones.

The published ART 0.5.18 wheel and ART-Embodied both support Python 3.11, so the validated OpenVLA-OFT stack can host the ART control plane and native policy in one isolated environment. Upstream ART main has since raised its source-tree requirement to Python 3.12; support for that future release will be widened only after the same embodied regression gate passes.

Use an isolated environment. Robotics policy stacks often pin Torch, Transformers, and simulator versions that differ from ART's LLM backends.

To reproduce the add-on installation order in a clean environment:

git clone https://github.com/nejumi/ART-Embodied.git
cd ART-Embodied
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install 'openpipe-art==0.5.18'
python -m pip install '.[libero]'
art-embodied doctor --profile libero

This ART-first installation is an acceptance-tested path, not merely a package resolver check. A fresh Python 3.11 environment has loaded the real OpenVLA-OFT model, completed one LoRA GRPO optimizer update, saved its adapter and training state, and reloaded the checkpoint without importing the development worktree's src directory.

For uv-based development in the checkout:

git clone https://github.com/nejumi/ART-Embodied.git
cd ART-Embodied
uv sync --extra lerobot

For the validated LIBERO integration:

uv sync --extra libero

For the experimental PI0/PI0.5 Flow-SDE path, use a separate Python 3.12 environment. LeRobot 0.6 and its Transformers 5 runtime are intentionally not resolved into the behavior-pinned OpenVLA-OFT worker:

python3.12 -m venv .venv-pi
source .venv-pi/bin/activate
python -m pip install '.[pi-libero]'
art-embodied doctor --profile pi

LeRobot's PI processor uses the gated google/paligemma-3b-pt-224 tokenizer. Accept its terms once, then run hf auth login or export HF_TOKEN. ART-Embodied resolves this small tokenizer before the multi-gigabyte policy checkpoint and reports an actionable error if access is missing. The coordinator completes the initial download before worker startup, so users do not need to prewarm each rollout worker manually.

This extra defines the runtime used by the PI Flow-SDE positive controls. The current evidence is positive on fixed generated-state development manifests; it is not yet a multi-seed or sealed-test claim. The model-family table above remains the source of truth for that evidence boundary.

The lerobot extra installs the simulator-neutral LeRobot package without selecting its rapidly moving model extras. The libero extra instead recreates the behavior-pinned OpenVLA-OFT/LIBERO worker stack used by the positive control, including Torch 2.6.0, Transformers 4.40.1, PEFT 0.11.1, and NumPy 1.26.4. Do not replace it with lerobot[libero,peft]: LeRobot 0.4.4's current extras require newer Transformers and PEFT versions that load the checkpoint but change its action logits. The lerobot and libero extras are therefore separate worker profiles and must not be installed into the same environment. The pi and pi-libero extras are a third, mutually isolated LeRobot 0.6 profile for native PI samplers.

Check the installed control-plane stack before allocating a GPU:

uv run art-embodied doctor

# Use this in the generic LeRobot worker profile.
uv run art-embodied doctor --require-lerobot

# Use this before allocating a GPU for the OpenVLA-OFT/LIBERO profile.
uv run art-embodied doctor --profile libero

For a process-isolated policy environment, use --worker; this validates the selected package profile without requiring ART in that worker process. Add --require-lerobot only to the generic LeRobot profile. Add --json in CI or launch scripts to consume the same report programmatically.

OpenVLA-OFT v0.1 requires the dedicated libero environment or an equivalent container with its validated stack. The runtime contract fails early on dependency drift instead of silently changing action logits.

When native policy dependencies still require process isolation, install the art-embodied wheel in both environments and select the policy environment explicitly:

runtime:
  # Used by rollout actors, batched inference servers, and training workers.
  worker_python_executable: /opt/venv/openvla/bin/python

Leaving this value null uses the current interpreter. The command is launched as an argv list without a shell. The selected environment must contain art-embodied and the policy/simulator dependencies; worker-only imports do not initialize ART. This is an optional dependency-isolation boundary rather than a requirement for the validated Python 3.11 profile.

Portable Slurm launch

Slurm is an optional launcher, not part of the ART-Embodied runtime contract. Do not embed a user home, checkout path, or cluster-specific account name in reusable job files. The repository wrapper resolves the checkout from its own location and loads that checkout's .env:

sbatch --gres=gpu:h100:8 --cpus-per-task=96 --mem=690G \
  scripts/slurm/run-in-repo.sh \
  uv run python examples/embodied/libero/train_openvla_oft.py \
  --config examples/embodied/openvla_oft_libero_grpo_rlinf_positive_control.yaml

Set ART_EMBODIED_ENV_FILE at submission time only when secrets live outside the checkout:

ART_EMBODIED_ENV_FILE="${HOME}/.config/art-embodied/secrets.env" \
  sbatch --export=ALL scripts/slurm/run-in-repo.sh COMMAND [ARG ...]

Slurm exports the submission environment and original SLURM_SUBMIT_DIR by default. The wrapper uses that directory after Slurm copies the script into its spool, and falls back to the script location when run directly. It therefore works after cloning under another user, home directory, cluster, or region without editing the job file. Set ART_EMBODIED_REPO_ROOT only when submitting from outside the checkout. Experiment conditions remain in YAML; only secrets and machine-local locations may use environment overrides.

The GPU image must expose the NVIDIA driver and a GLVND EGL loader (libEGL.so.1; package libegl1 on Ubuntu) for headless LIBERO rendering. Cluster images should install it system-wide. For an immutable image, an administrator-provided compatible loader may instead be placed in the ignored repo-relative .runtime/lib/ directory; the portable wrapper prepends that directory without recording a machine-specific path in YAML.

Run the OpenVLA-OFT control

Experiment conditions live in YAML. Environment variables are reserved for secrets such as WANDB_API_KEY.

uv run art-embodied validate \
  examples/embodied/openvla_oft_libero_grpo_rlinf_positive_control.yaml

uv run python examples/embodied/libero/train_openvla_oft.py \
  --config examples/embodied/openvla_oft_libero_grpo_rlinf_positive_control.yaml \
  --preflight

uv run python examples/embodied/libero/train_openvla_oft.py \
  --config examples/embodied/openvla_oft_libero_grpo_rlinf_positive_control.yaml

validate and --preflight check trajectory geometry, optimizer rows, devices, evaluation scenarios, and simulator assets before loading a model. The positive control is intentionally compute-heavy. For a new integration, start from lerobot_action_token_grpo.template.yaml.

For a claim-facing long run, generate and freeze the state-held-out manifest before starting training, evaluate the SFT baseline on it once, and use that same manifest and baseline report for periodic evaluation. Rollout success is already the train diagnostic; a train-matched fixed set must not be presented as held-out validation. Set evaluation.data_role: development while choosing recipes or checkpoints. After freezing the method and selection rule, use a new manifest with data_role: sealed_test and checkpoint_selection: last. ART-Embodied rejects selecting a best checkpoint from sealed outcomes. The manifest workflow is documented in the examples guide.

Every fixed evaluation writes a machine-regenerable evidence index containing the raw-outcome hash, manifest hash, exact package source-tree hash, Git state, dependency versions, runtime/container identity, and evaluated base/adapter or transactional-checkpoint identity. With W&B enabled, log_evaluation_artifacts: true publishes the index and raw outcomes as one versioned evaluation artifact; without W&B, the same files remain local.

Evaluate the fixed SFT baseline without constructing an optimizer:

uv run python examples/embodied/libero/train_openvla_oft.py \
  --config examples/embodied/openvla_oft_libero_object_sft_baseline_eval.yaml \
  --evaluate-only --evaluation-step 0

Evaluate a saved ART-Embodied policy snapshot without resuming its optimizer:

uv run python examples/embodied/libero/train_openvla_oft.py \
  --config experiment-eval.yaml --evaluate-only --evaluation-step 5 \
  --policy-checkpoint outputs/my-run/checkpoints/step-000005/policy

The evaluation YAML must describe the same policy family, base revision, processor, environment, and fixed scenario contract as the checkpoint.

See examples/embodied/README.md for candidate preparation, paired evaluation, generated state manifests, and conformance tools.

Connect an existing LeRobot workflow

ART calls the same LeRobot policy, preprocessor, postprocessor, and environment used by the application. The high-level LeRobot-first helper runs through ART's model lifecycle internally: model registration, grouped trajectories, backend.train(model, groups, ...), model-owned logging, checkpoint lineage, and backend close. The only policy-specific seam for an action-token model is recording sampled tokens and rollout-time log probabilities.

import asyncio

import art_embodied as embodied
from my_robot_app import (
    evaluation_scenarios,
    make_environment,
    policy,
    postprocessor,
    preprocessor,
    record_action_tokens,
    train_scenarios,
)


async def main() -> None:
    config = embodied.EmbodiedExperimentConfig.from_yaml("experiment.yaml")
    adapter = embodied.LeRobotPolicyAdapter(
        policy=policy,
        preprocessor=preprocessor,
        postprocessor=postprocessor,
        device=config.policy.device,
        record_action_fn=record_action_tokens,
    )
    result = await embodied.run_lerobot_experiment(
        config=config,
        policy=policy,
        train_scenarios=train_scenarios,
        evaluation_scenarios=evaluation_scenarios,
        environment_factory=make_environment,
        policy_adapter=adapter,
    )
    print(result.config_fingerprint)


asyncio.run(main())

ART users can use the same runtime through the lower-level lifecycle without giving up native LeRobot policy ownership:

import art_embodied as embodied

config = embodied.EmbodiedExperimentConfig.from_yaml("experiment.yaml")
model = embodied.EmbodiedTrainableModel(policy=policy, config=config)
native_backend = embodied.make_embodied_backend(config, policy=policy)
backend = embodied.EmbodiedBackend(native_backend, config=config)
await model.register(backend)

groups = await embodied.gather_trajectory_groups(
    [
        embodied.trajectory_group(
            (rollout(model.policy, scenario) for _ in range(config.algorithm.group_size)),
            metadata={"scenario_id": scenario.id},
        )
        for scenario in train_scenarios
    ]
)
result = await backend.train(
    model,
    groups,
    learning_rate=config.training.optimizer.learning_rate,
)
await model.log(groups, split="train", metrics=result.metrics, step=result.step)
await model.close()

EmbodiedTrainableModel is an actual art.TrainableModel, but it deliberately does not create an OpenAI-compatible text inference client. Rollouts continue to use the policy's native LeRobot processors and action sampler.

Continuous, diffusion, and flow policies require objectives aligned with their native samplers. ART-Embodied does not treat them as token policies merely to reuse the OpenVLA path.

Scale without changing the experiment

Slurm is optional. Devices, actors, inference replicas, worker lifecycle, and microbatch size are YAML settings; the task and learning contract remain the same on a workstation or cluster.

Use this order:

1. Complete one update with one actor and one model replica per rollout device.
2. Increase actors until simulator work no longer hides inference latency.
3. Use batched_server so actors share a policy instead of duplicating weights.
4. Add inference replicas only when measured HBM headroom and throughput justify it.
5. Enable local distributed training when optimizer wall time is material.
6. Use cpu_offload when synchronous rollout and training share GPUs; size host memory for every offloaded replica.

On 80 GB H100s, the measured OpenVLA-OFT profile uses three inference replicas and six actors per GPU, batch size two, and training microbatch 12. It collected 1,024 trajectories at 1.536 trajectories/s, over twice the original research runtime. This is a measured profile, not a portable default. Keep 10–20% HBM headroom when deriving a profile for another policy or accelerator.

W&B and Weave

W&B/Weave emission is optional, but enabled paired runs put the immutable SFT baseline at native W&B Step 0 and every periodic RL evaluation on the same validation/success_rate curve. They then provide one training history row per completed optimizer update, stable video panels, fixed-evaluation Tables, and versioned LoRA/training-state artifacts. Weave records one trace root per update with group and trajectory children. Raw media bytes are never embedded in trace JSON.

The validation/* section is intentionally reserved for the small set of aggregate metrics used to judge a run. Per-task and per-episode results live in bounded Tables and evaluation Artifacts, while videos use stable media/simulation/* keys. This keeps the primary evaluation charts readable even for large held-out manifests.

Training follows the same human-facing contract. The top-level train/* section contains only train/success_rate and train/reward_mean. Sample and advantage health lives under signal/*, optimizer internals under optimization/*, timing and throughput under performance/*, and episode counts and durations under train_details/*. W&B groups by the first path component, so adding deeper paths below train/ would bury the primary curve rather than organize it.

OpenPipe ART 0.5.18 already installs the Weave client as a core dependency. ART-Embodied does not initialize a Weave project or make observability network calls unless the YAML enables them; install the observability extra to add the validated W&B client integration.

observability:
  delivery_failure_policy: fail_run
  wandb:
    enabled: true
    project: art-embodied
    mode: online
    log_model_artifacts: true
    log_evaluation_table: true
  weave:
    enabled: true
    project: art-embodied
    trace_trajectories: true
    max_groups_per_update: 4
    max_trajectories_per_group: 4
  videos_per_update: 2
  videos_per_evaluation: 8

Disable both integrations to train locally; checkpoints, local videos, and JSON evaluation outcomes remain available.

Long-run checkpoints are immutable transactions, not partially populated directories. ART-Embodied hashes the policy, optimizer, and RNG payloads, writes a completion marker, and atomically publishes the milestone. Resume verifies both payload integrity and the training-math contract before loading model state. Public recipes use delivery_failure_policy: fail_run: a W&B or Weave delivery exception is recorded in bounded local telemetry_failures.jsonl and then aborts the claim-facing run. Set best_effort explicitly when durable local evidence is preferable to mandatory external telemetry; completed optimizer updates and checkpoints remain intact. When resuming an output directory that already contains the measured Step-0 evaluation, set evaluation.baseline_outcomes_path to that outcome file while disabling evaluate_before_training. Preflight rejects an omitted reference so later validation cannot silently lose paired lift, confidence intervals, and McNemar statistics.

Boundaries

• Partial rollout groups fail the update rather than entering GRPO silently.
• Training diagnostics and claim-facing native evaluation are separate contracts.
• Videos and traces are bounded by YAML to avoid unbounded storage growth.
• Simulator-specific code stays in examples/adapters, not generic trajectory code.
• OpenVLA-OFT is the current positive control; other VLA families require their own sampler-aligned training mathematics and validation.

Documentation

• Embodied RL concepts and configuration
• Runtime and scaling architecture
• Release validation contract
• OpenVLA-OFT and LIBERO examples

Acknowledgements

ART-Embodied's OpenVLA-OFT positive control was made possible by the public RLinf implementation, training recipes, and released checkpoints. They provided a credible embodied-GRPO reference against which we could audit rollout semantics, action-token masking, advantage and loss aggregation, and native LIBERO evaluation. We are grateful to the RLinf authors for making those artifacts available.

The product runtime is an independent ART implementation and does not import or vendor RLinf code. References to rlinf_v01 identify an explicitly documented conformance profile, not a runtime dependency or a claim that this repository is an official RLinf distribution.

Relationship to ART

ART-Embodied tracks upstream ART and now uses an ART TrainableModel lifecycle, ART-shaped backend calls, grouped rollouts, TrainResult, model-owned logging, and checkpoint steps. Its multimodal trajectory schema and sampler-aligned VLA backends are embodied extensions rather than aliases for text trajectories or LLM backends. Embodied dependencies remain optional, and ordinary ART LLM-agent workflows do not import LeRobot or simulator packages.

The long-term packaging direction is a thin add-on installed beside ART rather than a permanently independent framework. Compatibility work should reduce duplicated lifecycle code, but must not move policy construction, processors, environment semantics, or native action sampling out of LeRobot.

Contributions are welcome for reproducible fixed benchmarks, policy-family adapters with valid likelihoods or scores, simulator adapters, and storage-safe W&B/Weave observability. See CONTRIBUTING.md.