Hi, we have a bimanual task dataset collected in LeIsaac simulation and converted to LeRobot format, and another bimanual dataset collected directly in our real-world setup with lerobot_record, and we want to concatenate them into a new mixed sim+real dataset for fine-tuning while keeping the original datasets unchanged; after checking, the two datasets seem to match on the main schema points such as robot type, fps, action/state shapes, and image keys (observation.images.left_wrist, right_wrist, top), but when we try the LeRobot merge flow it fails because the feature metadata is not exactly identical, especially in the video feature fields where one dataset uses info and the other includes both info and video_info, so our question is: what is the recommended way to combine a LeIsaac-converted bimanual simulation dataset with a directly recorded bimanual real-world LeRobot dataset for fine-tuning, and is there an official or preferred way to normalize these metadata differences before merging?
Hi, we have a bimanual task dataset collected in LeIsaac simulation and converted to LeRobot format, and another bimanual dataset collected directly in our real-world setup with lerobot_record, and we want to concatenate them into a new mixed sim+real dataset for fine-tuning while keeping the original datasets unchanged; after checking, the two datasets seem to match on the main schema points such as robot type, fps, action/state shapes, and image keys (observation.images.left_wrist, right_wrist, top), but when we try the LeRobot merge flow it fails because the feature metadata is not exactly identical, especially in the video feature fields where one dataset uses info and the other includes both info and video_info, so our question is: what is the recommended way to combine a LeIsaac-converted bimanual simulation dataset with a directly recorded bimanual real-world LeRobot dataset for fine-tuning, and is there an official or preferred way to normalize these metadata differences before merging?