Jusheng Zhang¹ · Keze Wang² · Liang Lin²
¹ Stanford University · ² X-Era AI
Paper · First release · Experiment logs · Reproduce · 中文
First research release (v1). This repository accompanies arXiv v1. Version 2 and real-robot experiments are in progress. The results released here are from simulation. Project status →
GPT-6 Astra reads a pretrained robot policy's weights and repairs a class of instruction failures by editing 65 token-embedding rows. The editing process uses weight inspection and budgeted simulator feedback, with no gradient updates or new demonstrations.
Same instruction, scene, seed, and initial-state index; original π₀.₅ on the left, edited policy on the right.
“position the glass container of fermented grapes atop the storage unit for dishes and utensils”
The original times out at 300 steps; the edited policy succeeds at 85. This is a selected held-out example, not an estimate of average performance. The successful rollout stays on its terminal frame while the original continues. Original video · Edited video · Episode and rendering provenance
On 765 held-out LIBERO-Plus paraphrases, the strongest of three independent editing sessions raises π₀.₅ success from 86.3% to 92.0%: 48 repairs and 4 regressions, exact paired p = 1.31 × 10⁻¹⁰.
| Scope of session C | Original | Edited |
|---|---|---|
| Instructions containing edited tokens · 335 episodes | 74.3% | 87.5% |
| Other instructions · 430 episodes | 411 successes | 411 successes |
The other 430 episodes have zero binary outcome changes. This refers to success/failure, not identical trajectories. All three language sessions improve significantly on the same held-out population. Full results, controls, and limits →
This release includes the experiment code, all 16 editing-session transcripts, executed tool calls and their outputs, saved notes, episode-level evaluation records, and the evaluated sparse edit. Each session has a README linking its notes, final answer, and tool log.
| Question | Policy | Sessions and records |
|---|---|---|
| Can a small weight edit repair unfamiliar wording? | π₀.₅ | A · B · C — all three improve; evaluated edit |
| Can it repair distracting instruction suffixes? | FastWAM | First search · Follow-up A · B · C — no significant follow-up repair |
| Can it repair visual or language perturbations? | FastWAM | Camera · Background · Language |
| Can it improve recovery from failure states? | π₀.₅ | A · B · C — no retained repair |
| Can it improve broader robustness? | π₀.₅ | A · B · C — candidates declined; target gains uncertain |
Read the unsuccessful searches too. The paper distinguishes a promising internal signal, a candidate left on disk, and an edit the editor decides to retain. Some cleanup operations were not executed in the archived runs; a saved candidate alone does not establish retention. How to read the logs →
Check the numbers without a GPU. This script uses only the Python standard library and the released episode records:
git clone https://github.com/jushengzhang/robot-weight-debugging.git
cd robot-weight-debugging
python3 evidence/audit_new_results.py| If you want to… | Open |
|---|---|
| Inspect or apply the actual 65-row edit | Payload, recipe, and application script |
| Recompute the primary and supplementary results | Evidence guide |
| Follow one complete editing session | Language session C |
| Run the backend or a new editing session | Reproduction guide · Experiment sources |
| Download the published paper and LaTeX source | Paper artifacts · Release downloads |
The base models and simulator assets come from their upstream projects. The reproduction guide documents the required environments and path configuration; this first release preserves the research code and is not a containerized simulator installation.
We build on Sparks of Weight-Space Programming and its released experiment logs. Here the object being edited is a pretrained robot policy, and success is tested through paired robot rollouts. The supported result is precise lexical repair; the unsuccessful searches help define its current limits.
@article{zhang2026debugrobots,
title = {Can LLMs Debug Robots by Editing Their Weights?},
author = {Zhang, Jusheng and Wang, Keze and Lin, Liang},
journal = {arXiv preprint arXiv:2611.09376},
year = {2026},
doi = {10.48550/arXiv.2611.09376},
url = {https://arxiv.org/abs/2611.09376}
}Original project code is released under Apache-2.0. Model-derived payloads, external checkpoints, datasets, and the paper retain their applicable terms; see third-party notices.
