Peiyu Yang*, Mohamad S. Moudallal*, Antonio Langella, Giulio Turrisi, Victor Barasuol, and Claudio Semini
All the authors are with the Dynamic Legged Systems lab, Istituto Italiano di Tecnologia (IIT), 16163, Genova, Italy.
E-mail: name.surname@iit.it
* Peiyu Yang and Mohamad S. Moudallal contributed equally.
Reinforcement learning , Grippers and other end-effectors, Non-contact manipulation
Overview · Demonstrations · Results · Installation · Training · Project website
Contact-based robotic manipulation is constrained by end-effector reachability and the mechanical properties of objects. Airflow can exert forces without physical contact, enabling manipulation of objects beyond the workspace or when direct contact is undesirable. Here we propose BeyondContact, a reinforcement learning framework for airflow-based non-contact manipulation.
The framework includes:
- An airflow extension: Develop a configurable and computationally efficient airflow-interaction module used in Isaac Lab, supporting massively parallel robot-learning environments.
- Gripper-compatible hardware: Design a lightweight, high-speed blower that coexists with the gripper, delivering a maximum air velocity of 78m/s and a static thrust of up to 5.9N (600gf).
- Airflow modeling: Experimentally identify centerline-decay and radial-spreading models for custom blowers and nozzles.
- RL training frameworks: Propose RL training frameworks that enable two different tasks.
The paper evaluates zero-shot sim-to-real deployment, with no real-world policy fine-tuning.
| Isaac Lab extension | Hardware design | Training tasks | Real-world HPB success | Real-world RR heading error |
|---|---|---|---|---|
| 1 · reusable airflow extension | 1 · gripper-compatible blower design | 2 · HPB and RR | 80% · 16 / 20 trials | 2.6° MAE · 4 headings |
The policy first pushes the cube into a position suitable for blowing, then adjusts the nozzle direction and airflow to guide it into the target region. Once the cube arrives, the airflow command decreases to zero. For targets within gripper reach, the policy primarily uses contact-based manipulation.
See the looping HPB demonstration in Results. The full experiment compilation is available on the project website.
The robot continuously adjusts the blowing direction to rotate a cuboid toward a target heading.
See the looping demonstrations for 60°, 90°, and 135° in Results. Full trial recordings are available on the project website.
A 40 mm EDF is housed in an enclosure beneath the gripper. A magnetic connection supports interchangeable nozzles. An ESP32 microcontroller regulates the blower through an ESC. The hardware design files will be released on the project website after the paper is published.
The blower can achieve a maximum air velocity of 78 m/s and a static thrust of 5.9 N. We limit the blower output during identification and training to accommodate the anemometer's measurement range and the task configuration.
We experimentally identify the airflow profile of the blower and nozzle, then use surface integration to convert airflow into object forces and torques.
Both tasks use PPO with an asymmetric actor–critic. The details of the training configurations are:
- Actions: robot joint-position commands and airflow action.
- Architecture reported in the paper: actor and critic MLPs with
[512, 256, 128]hidden dimensions and ELU activations. - Robustness: domain randomization over object mass, friction, robot gains, and airflow parameters.
- Reported training setup: 4,096 environments, 5,000 iterations, NVIDIA RTX 5090 and Intel U7 265K.
We validated both tasks on physical Piper-L robotics arm.
|
Hybrid push–blow HPB 
|
Reorientation · 90° Airflow only 
|
|
Reorientation · 60° Airflow only 
|
Reorientation · 135° Airflow only 
|
For HPB, the target is a 20 × 20 cm square centered 1.3 m in front of the robot base. For RR, we setted four trails commanded at different final headings:
HPB: measured joint angles and airflow command through pushing, blowing, and settling.
RR: the 90° deployment sequence, joint and airflow traces, and heading trajectories for the other evaluated targets.
| Path | Contents |
|---|---|
external_blower/blower |
Reusable airflow model, surface integration, object assets, and visualization |
blower_arm_rl |
Piper-L task configurations, PPO training, and checkpoint evaluation |
docs |
Static project website and optimized presentation media |
Registered training tasks:
| Paper task | Gym ID |
|---|---|
| Hybrid push–blow (HPB) | Isaac-Blower-Piper-GripperAssistBoxPush-v0 |
| Rectangular reorientation (RR) | Isaac-Blower-Piper-GripperAssistCuboidOrientation-v0 |
Use an environment with Isaac Sim 5.1.0, an Isaac Lab installation, Python 3.11, PyTorch, and RSL-RL. Follow the Isaac Lab installation guide.
Run from this repository root using these Python commands:
python -m pip install -e external_blower/blower/source/blower
python -m pip install -e blower_arm_rl/source/blower_arm_rl
python blower_arm_rl/scripts/list_envs.py# Hybrid push–blow (HPB)
python blower_arm_rl/scripts/rsl_rl/train.py --task Isaac-Blower-Piper-GripperAssistBoxPush-v0 --headless
# Rectangular reorientation (RR)
python blower_arm_rl/scripts/rsl_rl/train.py --task Isaac-Blower-Piper-GripperAssistCuboidOrientation-v0 --headlessReplace /path/to/model.pt with your trained checkpoint:
# Hybrid push–blow (HPB)
python blower_arm_rl/scripts/rsl_rl/play.py --task Isaac-Blower-Piper-GripperAssistBoxPush-Play-v0 --num_envs 1 --checkpoint /path/to/model.pt
# Rectangular reorientation (RR)
python blower_arm_rl/scripts/rsl_rl/play.py --task Isaac-Blower-Piper-GripperAssistCuboidOrientation-Play-v0 --num_envs 1 --checkpoint /path/to/model.ptThe research page uses local assets and requires no frontend build step.
python -m http.server 8000 --directory docsVisit http://localhost:8000. The site is self-contained in docs/, including its optimized figures, looping GIFs, and videos.