isaaclab2lerobot exports wrong action dimension for Mimic datasets
Bug description
When converting an IsaacLab HDF5 dataset to LeRobot format, the conversion path currently reads actions from episode_data._data["obs"]["actions"].
This is problematic for Mimic-generated / Mimic-postprocessed datasets.
In those datasets:
- top-level
actions may already be converted back to joint-space actions, for example shape (T, 6)
- but
obs/actions may still keep the intermediate Mimic action representation, for example shape (T, 8)
As a result, isaaclab2lerobot.py / isaaclab2lerobotv3.py can export the wrong action representation and produce 8D actions in the final LeRobot dataset, even though the final top-level actions are already 6D.
Affected code
Current logic in task templates:
obs_data = episode_data._data["obs"]
action = obs_data["actions"][-1]
This appears in:
source/leisaac/leisaac/tasks/template/single_arm_env_cfg.py
source/leisaac/leisaac/tasks/template/bi_arm_env_cfg.py
source/leisaac/leisaac/tasks/template/direct/single_arm_env.py
source/leisaac/leisaac/tasks/template/direct/bi_arm_env.py
Reproduction
Input dataset structure
For a final Mimic-processed dataset, I observed:
- top-level
actions: (263, 6)
obs/actions: (263, 8)
Example file:
final_generated_mimic-lift-cube-example.hdf5
Conversion
Run:
python scripts/convert/isaaclab2lerobotv3.py \
--task_name <task_name> \
--hdf5_root ./datasets \
--hdf5_files final_generated_mimic-lift-cube-example.hdf5 \
--repo_id <repo_id> \
--fps 30
Expected behavior
The converter should prioritize the final top-level actions field when it exists:
episode_data._data["actions"]
and only fall back to:
episode_data._data["obs"]["actions"]
when top-level actions does not exist.
If both exist and have different shapes, it would be helpful to print a warning.
Actual behavior
The converter reads obs/actions, so the exported LeRobot dataset uses the 8D Mimic action representation instead of the final 6D joint action representation.
Why this matters
For Mimic workflows, the final dataset may intentionally contain:
- top-level
actions: final replay / joint-space actions
obs/actions: intermediate action representation used during generation
Using obs/actions silently changes the semantics of the exported action field.
Suggested fix
Use action resolution logic like:
if "actions" in episode_data._data:
action = episode_data._data["actions"][-1]
elif "actions" in episode_data._data["obs"]:
action = episode_data._data["obs"]["actions"][-1]
isaaclab2lerobot exports wrong action dimension for Mimic datasets
Bug description
When converting an IsaacLab HDF5 dataset to LeRobot format, the conversion path currently reads actions from
episode_data._data["obs"]["actions"].This is problematic for Mimic-generated / Mimic-postprocessed datasets.
In those datasets:
actionsmay already be converted back to joint-space actions, for example shape(T, 6)obs/actionsmay still keep the intermediate Mimic action representation, for example shape(T, 8)As a result,
isaaclab2lerobot.py/isaaclab2lerobotv3.pycan export the wrong action representation and produce 8D actions in the final LeRobot dataset, even though the final top-level actions are already 6D.Affected code
Current logic in task templates:
This appears in:
source/leisaac/leisaac/tasks/template/single_arm_env_cfg.pysource/leisaac/leisaac/tasks/template/bi_arm_env_cfg.pysource/leisaac/leisaac/tasks/template/direct/single_arm_env.pysource/leisaac/leisaac/tasks/template/direct/bi_arm_env.pyReproduction
Input dataset structure
For a final Mimic-processed dataset, I observed:
actions:(263, 6)obs/actions:(263, 8)Example file:
final_generated_mimic-lift-cube-example.hdf5Conversion
Run:
Expected behavior
The converter should prioritize the final top-level
actionsfield when it exists:and only fall back to:
when top-level
actionsdoes not exist.If both exist and have different shapes, it would be helpful to print a warning.
Actual behavior
The converter reads
obs/actions, so the exported LeRobot dataset uses the 8D Mimic action representation instead of the final 6D joint action representation.Why this matters
For Mimic workflows, the final dataset may intentionally contain:
actions: final replay / joint-space actionsobs/actions: intermediate action representation used during generationUsing
obs/actionssilently changes the semantics of the exported action field.Suggested fix
Use action resolution logic like: