PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning

Official project repository for PRICE — Physical Relations for Inferring Credit from Episodes.

This repository presents the paper, method, and experimental highlights. It contains no implementation, checkpoints, or training data.

Authors

Yangang Zou^1,2,* · Jiajun Lu^3,* · Weitao Zhou^4,6,†,‡ · Haibao Yu^5,7,‡ · Bozhou Zhang¹ · Jiawei Wang⁴ · Honglong Tian⁴ · Minglei Li⁴ · Li Zhang^1,2,‡

¹School of Data Science, Fudan University · ²Shanghai Innovation Institute · ³Beihang University · ⁴Simple AI · ⁵Tuojing Intelligence · ⁶Tsinghua University · ⁷The University of Hong Kong

^*Equal contribution. ^†Project leader. ^‡Corresponding authors.

Overview

Outcome-based reinforcement learning post-trains vision–language–action policies using terminal success signals, but typically assigns the same trajectory-level advantage to every action chunk. A failed episode can therefore penalize useful early actions alongside the actions that caused the failure.

PRICE derives local action-chunk credit from physical relations across trajectories. Rollouts that reach corresponding physical situations serve as references for one another: their terminal outcomes provide evidence about local progress, without an auxiliary evaluator or additional progress annotations.

Method

PRICE combines two components:

1. Physical relational graph. Match corresponding physical situations at action-chunk boundaries and pool current and historical terminal outcomes to estimate success potentials.
2. Confidence-gated credit assignment. Use changes in these potentials to refine trajectory-level supervision, retaining local credit when the evidence supports its direction.

The paper connects oracle potential changes to the terminal-success objective and gives a finite-sample directional bound for outcome-independent evidence pools. Independent continuation tests assess whether retained credits align with local progress.

Experimental Highlights

Evaluation	PRICE result
LIBERO, average across four suites	99.0% success
LIBERO-Long	97.6% success
RoboTwin 2.0, six selected tasks	90.0% average success
Real robots, Remote Insertion	55.0% success with PRICE-AWR
Real robots, Produce Sorting	67.5% success with PRICE-AWR

At the 90% success milestone, PRICE reduces training steps relative to SimpleVLA-RL by 34.5% on LIBERO-Goal and 37.1% on LIBERO-Spatial, with corresponding GPU-hour savings of 38.3% and 32.8% under the paper's matched settings.

On real robots, PRICE-AWR reaches 61.25% macro-average success, compared with 47.5% for GRPO-AWR, using the same offline data and optimization budget. See the paper for evaluation protocols and comparison details.

Citation

@article{zou2026price,
  title   = {{PRICE} the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning},
  author  = {Zou, Yangang and Lu, Jiajun and Zhou, Weitao and Yu, Haibao and Zhang, Bozhou and Wang, Jiawei and Tian, Honglong and Li, Minglei and Zhang, Li},
  journal = {arXiv preprint arXiv:2609.38890},
  year    = {2026},
  doi     = {10.48550/arXiv.2609.38890},
  url     = {https://arxiv.org/abs/2609.38890}
}