Skip to content

About

First release for Can LLMs Debug Robots by Editing Their Weights? Code, 65-row edit, 16 experiment logs and evidence. V2 and real-robot experiments in progress.

Topics

Resources

Stars

204 stars

Watchers

9 watching

Forks

Repository files navigation

Can LLMs Debug Robots by Editing Their Weights?

Jusheng Zhang¹ · Keze Wang² · Liang Lin²
¹ Stanford University · ² X-Era AI

Paper · First release · Experiment logs · Reproduce · 中文

First research release (v1). This repository accompanies arXiv v1. Version 2 and real-robot experiments are in progress. The results released here are from simulation. Project status →

GPT-6 Astra reads a pretrained robot policy's weights and repairs a class of instruction failures by editing 65 token-embedding rows. The editing process uses weight inspection and budgeted simulator feedback, with no gradient updates or new demonstrations.

Actual paired simulator replay: original policy and edited embeddings

Same instruction, scene, seed, and initial-state index; original π₀.₅ on the left, edited policy on the right.

“position the glass container of fermented grapes atop the storage unit for dishes and utensils”

The original times out at 300 steps; the edited policy succeeds at 85. This is a selected held-out example, not an estimate of average performance. The successful rollout stays on its terminal frame while the original continues. Original video · Edited video · Episode and rendering provenance

What changes after the edit?

On 765 held-out LIBERO-Plus paraphrases, the strongest of three independent editing sessions raises π₀.₅ success from 86.3% to 92.0%: 48 repairs and 4 regressions, exact paired p = 1.31 × 10⁻¹⁰.

Scope of session C Original Edited
Instructions containing edited tokens · 335 episodes 74.3% 87.5%
Other instructions · 430 episodes 411 successes 411 successes

The other 430 episodes have zero binary outcome changes. This refers to success/failure, not identical trajectories. All three language sessions improve significantly on the same held-out population. Full results, controls, and limits →

View the paper's result overview

Held-out language repair results

Explore the experiments

This release includes the experiment code, all 16 editing-session transcripts, executed tool calls and their outputs, saved notes, episode-level evaluation records, and the evaluated sparse edit. Each session has a README linking its notes, final answer, and tool log.

Question Policy Sessions and records
Can a small weight edit repair unfamiliar wording? π₀.₅ A · B · C — all three improve; evaluated edit
Can it repair distracting instruction suffixes? FastWAM First search · Follow-up A · B · C — no significant follow-up repair
Can it repair visual or language perturbations? FastWAM Camera · Background · Language
Can it improve recovery from failure states? π₀.₅ A · B · C — no retained repair
Can it improve broader robustness? π₀.₅ A · B · C — candidates declined; target gains uncertain

Read the unsuccessful searches too. The paper distinguishes a promising internal signal, a candidate left on disk, and an edit the editor decides to retain. Some cleanup operations were not executed in the archived runs; a saved candidate alone does not establish retention. How to read the logs →

Start here

Check the numbers without a GPU. This script uses only the Python standard library and the released episode records:

git clone https://github.com/jushengzhang/robot-weight-debugging.git
cd robot-weight-debugging
python3 evidence/audit_new_results.py
If you want to… Open
Inspect or apply the actual 65-row edit Payload, recipe, and application script
Recompute the primary and supplementary results Evidence guide
Follow one complete editing session Language session C
Run the backend or a new editing session Reproduction guide · Experiment sources
Download the published paper and LaTeX source Paper artifacts · Release downloads

The base models and simulator assets come from their upstream projects. The reproduction guide documents the required environments and path configuration; this first release preserves the research code and is not a containerized simulator installation.

Relation to weight-space programming

We build on Sparks of Weight-Space Programming and its released experiment logs. Here the object being edited is a pretrained robot policy, and success is tested through paired robot rollouts. The supported result is precise lexical repair; the unsuccessful searches help define its current limits.

Citation

@article{zhang2026debugrobots,
  title   = {Can LLMs Debug Robots by Editing Their Weights?},
  author  = {Zhang, Jusheng and Wang, Keze and Lin, Liang},
  journal = {arXiv preprint arXiv:2611.09376},
  year    = {2026},
  doi     = {10.48550/arXiv.2611.09376},
  url     = {https://arxiv.org/abs/2611.09376}
}

License

Original project code is released under Apache-2.0. Model-derived payloads, external checkpoints, datasets, and the paper retain their applicable terms; see third-party notices.

About

First release for Can LLMs Debug Robots by Editing Their Weights? Code, 65-row edit, 16 experiment logs and evidence. V2 and real-robot experiments in progress.

Topics

Resources

Stars

204 stars

Watchers

9 watching

Forks

Releases

Packages

Contributors

Languages