Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 7 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,13 +240,13 @@ python scripts/dump_config_to_yaml.py <name> --output-dir configs
Before we can run training, we need to compute the normalization statistics for the training data. Run the script below with the name of your training config (e.g., for Xense):

```bash
python scripts/compute_norm_stats.py --config-name pi05_base_arx5_lora
python scripts/compute_norm_stats.py --config-name pi05_base_xtac_umi_pick_up_cube_0807_h200
```

Now we can kick off training with the following command (the `--overwrite` flag is used to overwrite existing checkpoints if you rerun fine-tuning with the same config):

```bash
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py pi05_base_arx5_lora --exp-name=my_experiment --overwrite
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py pi05_base_xtac_umi_pick_up_cube_0807_h200 --exp-name=my_experiment --overwrite
```

The command will log training progress to the console and save checkpoints to the `checkpoints` directory. You can also monitor training progress on the Weights & Biases dashboard. For maximally using the GPU memory, set `XLA_PYTHON_CLIENT_MEM_FRACTION=0.9` before running training -- this enables JAX to use up to 90% of the GPU memory (vs. the default of 75%).
Expand All @@ -258,7 +258,7 @@ The command will log training progress to the console and save checkpoints to th
Once training is complete, we can run inference by spinning up a policy server and then querying it from your robot runtime. Launching a model server is easy (we use the checkpoint for iteration 20,000 for this example, modify as needed):

```bash
python scripts/serve_policy.py policy:checkpoint --policy.config=pi05_base_arx5_lora --policy.dir=checkpoints/pi05_base_arx5_lora/my_experiment/19999
python scripts/serve_policy.py policy:checkpoint --policy.config=pi05_base_xtac_umi_pick_up_cube_0807_h200 --policy.dir=checkpoints/pi05_base_xtac_umi_pick_up_cube_0807_h200/my_experiment/19999
```

This will spin up a server that listens on port 8000 and waits for observations to be sent to it. We can then run an evaluation script (or robot runtime) that queries the server.
Expand Down Expand Up @@ -440,11 +440,6 @@ XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
#### BiFlexiv — assemble box with phone stand

```bash
python scripts/compute_norm_stats.py --config-name pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0403
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0403 \
--exp-name=bi_flexiv_assemble_box_with_phone_stand_lora_20260403 --overwrite

python scripts/compute_norm_stats.py --config-name pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0422_merged_fixed_h100
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0422_merged_fixed_h100 \
Expand All @@ -460,15 +455,6 @@ XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
--exp-name=pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0430_merged_fixed_h100_0510 --overwrite
```

#### BiFlexiv — earbuds case sequential insertion task

```bash
python scripts/compute_norm_stats.py --config-name pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100 \
--exp-name=pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100_0513 --overwrite
```

#### BiFlexiv - shoe_insole_retrieval_and_packing_0

```bash
Expand All @@ -481,11 +467,6 @@ python scripts/compute_norm_stats.py --config-name pi05_base_bi_flexiv_shoe_inso
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
pi05_base_bi_flexiv_shoe_insole_retrieval_and_packing_0515_h100 \
--exp-name=pi05_base_bi_flexiv_shoe_insole_retrieval_and_packing_0515_h100_0519 --overwrite

python scripts/compute_norm_stats.py --config-name pi05_base_bi_flexiv_shoe_insole_retrieval_and_packing_0607_h100
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \
pi05_base_bi_flexiv_shoe_insole_retrieval_and_packing_0607_h100 \
--exp-name=pi05_base_bi_flexiv_shoe_insole_retrieval_and_packing_0607_h100_0607 --overwrite
```

#### BiFlexiv - shoe_insole_retrieval_and_packing_1
Expand All @@ -508,25 +489,9 @@ XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 python scripts/train.py \

### Deployment Commands (latest per platform)

#### BiARX5 — training-time RTC inference

```bash
python scripts/serve_policy.py \
--default-prompt="pick rgb cubes and place them into the blue box" \
policy:checkpoint \
--policy.config=pi05_base_arx5_lora_training_time_rtc \
--policy.dir=checkpoints/pi05_base_arx5_lora_training_time_rtc/training_time_rtc_20251209/39999
```

#### BiFlexiv — assemble box inference

```bash
python scripts/serve_policy.py \
--default-prompt="assemble the box with the phone stand" \
policy:checkpoint \
--policy.config=pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0410_merged_fixed \
--policy.dir=checkpoints/pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0410_merged_fixed/bi_flexiv_assemble_box_with_phone_stand_lora_0410_merged_fixed_20260413/79999

python scripts/serve_policy.py \
--default-prompt="assemble the box with the phone stand" \
policy:checkpoint \
Expand All @@ -546,16 +511,6 @@ python scripts/serve_policy.py \
--policy.dir=checkpoints/pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0430_merged_fixed_h100/pi05_base_bi_flexiv_assemble_box_with_phone_stand_lora_0430_merged_fixed_h100_0510/66000
```

#### BiFlexiv — earbuds case assembly with lid operation inference

```bash
python scripts/serve_policy.py \
--default-prompt="Pick up the earbuds from the acrylic plate, open the charging case, precisely align and gently insert the earbuds using contact feedback, then close the lid securely" \
policy:checkpoint \
--policy.config=pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100 \
--policy.dir=checkpoints/pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100/pi05_base_bi_flexiv_earbuds_case_sequential_insertion_teleop_rtc_0513_h100_0513/19999
```

#### BiFlexiv - shoe_insole_retrieval_and_packing_0 inference

```bash
Expand All @@ -580,8 +535,8 @@ python scripts/serve_policy.py \
python scripts/serve_policy.py \
--default-prompt="Take the shoe out of the shoebox, open the shoe tongue, remove and reinsert the insole, then place the shoe into the shoebox." \
policy:checkpoint \
--policy.config=pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100 \
--policy.dir=checkpoints/pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100/pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100_0701/77000
--policy.config=pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100 \
--policy.dir=checkpoints/pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100/pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100_0701/77000
```

#### BiFlexiv - bag_inspection_0611 inference
Expand Down Expand Up @@ -653,8 +608,8 @@ The shoe-insole packing demo runs across three machines: the **inference server*
python scripts/serve_policy.py \
--default-prompt="Take the shoe out of the shoebox, open the shoe tongue, remove and reinsert the insole, then place the shoe into the shoebox." \
policy:checkpoint \
--policy.config=pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100 \
--policy.dir=checkpoints/pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100/pi05_base_bi_flexiv_newbalance_shoe_insole_retrieval_and_packing_0616_h100_0701/77000
--policy.config=pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100 \
--policy.dir=checkpoints/pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100/pi05_base_bi_flexiv_newbalacne_shoe_insole_retrieval_and_packing_0616_h100_0701/77000
```

**② Screen PC (video-playback laptop) — run on the screen pc side**
Expand Down
Loading