Skip to content

Align tactile naming with lerobot-xense - #16

Merged
Vertax42 merged 1 commit into
mainfrom
refactor/tactile-naming-alignment
Aug 26, 2026
Merged

Vertax42 merged 1 commit into
mainfrom
refactor/tactile-naming-alignment

Conversation

@Vertax42

Copy link
Copy Markdown
Collaborator

lerobot-xense renamed both the tactile knob and the tactile camera keys. This repo still spoke the old vocabulary, and two of the three call sites were already broken at runtime — this is a bug fix as much as a rename.

1. enable_tactile_sensors → enable_tactile

Upstream moved the knob onto the typed gripper: block at c49d4aa5 (what a gripper carries is a property of the gripper, not of the arm holding it). BiFlexivRizon4RTConfig no longer declares it, so every Flexiv run died in the recipe loader — reproduced before touching anything:

TypeError: BiFlexivRizon4RTConfig.__init__() got an unexpected keyword argument 'enable_tactile_sensors'

flexiv_recipe.load_robot_config() now routes each override to whichever dataclass declares it. Callers still pass a flat kwarg; the loader decides the nesting level, rebuilds the gripper block, and lets __post_init__ re-clone it per side. A name neither level knows is now an error, not a silent no-op:

forward-04.yaml: override(s) ['enable_tactile_sensors'] name no field on
BiFlexivRizon4RTConfig or its TaccapFollowerConfig block.

The override warning gained the level it landed on: gripper.enable_tactile: True -> False.

The ARX5 path is renamed to match — its lerobot config is renamed in the sister-repo PR (Vertax42/lerobot-xense, branch refactor/arx5-enable-tactile). Land that one first, or bi_arx5 inference breaks on the new kwarg.

2. <arm>_tactile_{0,1} → <arm>_tactile_<finger>

The suffix now names the jaw the pad sits on (serial parity, odd → left) rather than USB enumeration order, which was a stable order but not an identity.

bi_arx5's camera mapping still asked for left_tactile_0, a key lerobot stopped emitting at 1146d034. get_images() only warns on a missing camera — so tactile inference on ARX5 had been feeding the model nothing, one log line per step. There was a second latent crash behind it: real_env emitted left_tactile_0 while AlohaTactileInputs.EXPECTED_CAMERAS wanted cam_left_tactile, which fails the subset check. Both are gone: lerobot's key is carried through unchanged, robot → transform → model slot, with no renaming step anywhere.

This is a relabel, not a retrain

embed_prefix() selects a branch by name and applies one shared encoder per branch (PaliGemma.img / tactile_img), so no weight is keyed by image name. The only thing a rename can disturb is token order — which is sorted-key order, since JAX canonicalises dict keys. Verified it is unchanged:

base_0_rgb          -> base_0_rgb             same slot
left_wrist_0_rgb    -> left_wrist_0_rgb       same slot
right_wrist_0_rgb   -> right_wrist_0_rgb      same slot
left_tactile_0_rgb  -> left_tactile_left_rgb  same slot
right_tactile_0_rgb -> right_tactile_left_rgb same slot

preprocess_observation_tactile's "tactile" in key augmentation branch still matches.

The model consumes one pad per arm; this bench wires the left jaw of each (OG000337, OG000339 — both odd). The other two pads are recorded and unused. Feeding all four is a change of input shape, not of naming, and is not attempted here.

Interaction with #15

None, but worth stating: #15 selects tactile streams by substring (TACTILE_KEY_MARKER = "tactile"), which the new keys still match, and the two branches touch disjoint files. #15's docs and its _XENSE_VIDEO_KEYS test fixture still spell _tactile_{0,1}; those are cosmetic and can be swept in a follow-up.

Verification

  • ruff check src/ examples/ clean
  • pytest examples/ src/openpi/policies → 67 passed; the one failure (policy_rtc_test) reproduces on a clean tree
  • All 5 Flexiv recipes decode and route enable_tactile=False to gripper / left_gripper / right_gripper
  • BiARX5Config(enable_tactile=True) emits exactly the keys real_env now asks for

🤖 Generated with Claude Code

lerobot-xense renamed both the tactile knob and the tactile camera keys.
This repo still spoke the old vocabulary, and two of the three call sites
were already failing at runtime.

`enable_tactile_sensors` -> `enable_tactile`. Upstream moved the knob onto
the typed `gripper:` block (c49d4aa5), so `BiFlexivRizon4RTConfig` no
longer declares it and every Flexiv run died in the recipe loader:

    TypeError: BiFlexivRizon4RTConfig.__init__() got an unexpected
    keyword argument 'enable_tactile_sensors'

`flexiv_recipe.load_robot_config()` now routes each override to whichever
dataclass declares it, so callers still pass a flat kwarg and a name
neither level knows is an error rather than a silent no-op. The ARX5 path
is renamed to match; its lerobot config is renamed in the sister repo.

`<arm>_tactile_{0,1}` -> `<arm>_tactile_<finger>`. The suffix names the
jaw the pad sits on, from serial parity, instead of USB enumeration order.
bi_arx5's camera mapping still asked for `left_tactile_0`, a key lerobot
stopped emitting at 1146d034 — and get_images() only warns on a missing
camera, so tactile inference had been feeding the model nothing at all.
The model slots and the Aloha tactile transform now use lerobot's key
end to end, with no renaming step in between.

Renaming the model slots is a relabel, not a retrain: embed_prefix()
picks a branch by name and applies one shared encoder per branch, so no
weight is keyed by image name. Token order is sorted-key order, and
`left_tactile_left_rgb` / `right_tactile_left_rgb` sort into the same
positions the `_0_rgb` names held.

The model consumes one pad per arm; this bench wires the left jaw of each
(OG000337, OG000339 — both odd). The other two pads are recorded, unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Vertax42
Vertax42 merged commit 7c51dc5 into main Aug 26, 2026
1 check passed
@Vertax42
Vertax42 deleted the refactor/tactile-naming-alignment branch August 26, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant