Teleoperation of a concentric tube robot by hand gesture, using a Leap Motion sensor to track the operator's hand and map it onto the robot's tip.
Status: archived. Research code from 2017, not maintained. See the note on the Leap Motion SDK below before attempting to build it.
This is the implementation behind:
A. Razjigaev, R. Crawford, J. Roberts and L. Wu, "Teleoperation of a concentric tube robot through hand gesture visual tracking," 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2017, pp. 1175–1180. doi:10.1109/ROBIO.2017.8324577
This gesture control was used in a workshop with orthopaedic surgeons comparing three off-the-shelf interfaces for driving a continuum robot: a force-feedback device, this hand-gesture tracking, and a gamepad.
J. Opie, Development of Interfaces for Orthopaedic Surgeons to Control Minimally Invasive Surgical Robots during Knee Arthroscopy, PhD thesis, Queensland University of Technology, Chapter 6. https://eprints.qut.edu.au/207127/
The gesture and gamepad control implementations are mine, as is the continuum robot they drive.
Hand gesture teleoperation of a concentric tube robot — the system running: hand motion tracked by the Leap Motion sensor, mapped onto the robot's tip.
A Microsoft Visual Studio project, built as Debug x86. The main implementation is in AndrewsLeapAttempt.cpp.
- Visual Studio, configuration Debug x86.
- Eigen, via the NuGet package.
- The Leap Motion SDK, for hand tracking.
A caveat on the Leap Motion dependency. This was written against the Leap Motion desktop SDK as it existed in 2017. Leap Motion has since become Ultraleap and the tracking software has moved on considerably, so the original SDK is legacy and this project will not build against the current one without changes. The gesture-to-tip mapping in AndrewsLeapAttempt.cpp is the part worth reading regardless of whether the sensor layer still runs.
The same concentric tube robot, from other angles:
- gamepad_snakebot_control — this system modified to take gamepad input instead of hand tracking. Same kinematics and control underneath.
- SnakeBot_Simulink — Simulink/Simscape model of the direct and differential mapping algorithms.
- ArduinoSnakeRobot — a three-tube concentric tube robot built at IIT Madras in 2018, modelled on the QUT robot teleoperated here.
MIT — see LICENSE.
Written by Andrew Razjigaev. Questions: andrew_razjigaev@outlook.com