A 3D-printed robotic arm that mimics your movements in real time using just a camera.
🌐 Language Note: This README is written in English. However, the full technical report (PDF) is in Portuguese. All source code files have two versions — an English version and a Portuguese version (files ending in
_PT). Both versions are functionally identical; only comments and variable names differ. Use whichever you prefer.
This is a humanoid robotic arm that imitates your arm and hand movements in real time. Instead of buttons or joysticks, you simply stand in front of a camera — the system recognizes your gestures and the robot copies them automatically.
The arm has 7 independent degrees of freedom: 5 fingers that open and close, an elbow that goes up and down, and a rotating base that turns left and right. The entire mechanical structure was 3D-printed using the open-source InMoov project as a foundation.
This README is a complete tutorial. If you follow every step, you'll be able to build your own robotic arm — even without much experience in programming or electronics.
demo.mp4
Before starting, it's useful to know where everything is:
HumanoidRoboticArmVision/
│
├── 📄 README.md ← You are here! The full tutorial
├── 📄 REQUIREMENTS.txt ← Software versions and libraries
├── 📄 LICENSE ← MIT License
├── 📄 RIA_G7_Relatorio_Final_Braco_Robotico.pdf ← Full technical report (Portuguese)
│
├── 📂 code/
│ ├── 📂 Arduino/Final_Ard/
│ │ ├── 📂 MotorDriver/ ← Arduino MEGA firmware (English)
│ │ ├── 📂 MotorDriver_PT/ ← Arduino MEGA firmware (Portuguese)
│ │ └── 📂 Arduino_Uno_CNC_Final/ ← Arduino Uno firmware (stepper motor)
│ │
│ ├── 📂 RPi/Final_RPI/
│ │ ├── ArmController.py ← Computer vision for Raspberry Pi (English)
│ │ └── ArmController_PT.py ← Computer vision for Raspberry Pi (Portuguese)
│ │
│ └── 📂 PC/
│ ├── VisionDebugger_PC.py ← Debug tool for PC (English)
│ ├── VisionDebugger_PC_PT.py ← Debug tool for PC (Portuguese)
│ ├── hand_landmarker.task ← MediaPipe model (hand)
│ └── pose_landmarker_lite.task ← MediaPipe model (body)
│
├── 📂 Codigos_PDFs/ ← Source code as PDF (report annexes, Portuguese)
│
└── 📂 media/ ← Project images and GIFs
Note: Each code file exists in two versions — English and Portuguese (
_PTsuffix). They are functionally identical; only the comments and variable names change.
Here's everything you need to build the arm. Some components can be swapped for equivalents.
| Component | Reference | Qty | Notes |
|---|---|---|---|
| Servo Motor (elbow) | DS5160 (60 kgf·cm) | 1 | Needs to be powerful — supports the arm's weight |
| Servo Motor (fingers + wrist) | MG996R (9-11 kgf·cm) | 6 | 5 for fingers + 1 for wrist rotation |
| Stepper Motor (base) | 17HS4401S (NEMA 17) | 1 | Precise rotation of the base |
| Servo Driver | PCA9685 (16-channel, I2C) | 1 | Controls all servos using just 2 pins |
| Stepper Driver | A4988 | 1 | Plugs into the CNC Shield |
| CNC Shield | CNC Shield V3 | 1 | Mounts directly on the Arduino Uno |
| Raspberry Pi | Raspberry Pi 5 (8GB) | 1 | The "brain" — processes the vision |
| Camera | Raspberry Pi Camera Module 3 | 1 | Dedicated camera for the RPi |
| Arduino | Arduino MEGA 2560 | 1 | Central motor controller |
| Arduino | Arduino Uno | 1 | Dedicated to the stepper motor |
| Component | Specification | Qty | Notes |
|---|---|---|---|
| PLA Filament | 1.75mm, any color | ~500g | For 3D printing all parts |
| Springs | 3/16″ x 1-3/4″ (4.8mm x 44.5mm) | 5 | Finger return mechanism |
| Tendons | Braided fishing line, 0.8mm, 200LB | 5x 50cm | Pull the fingers closed |
| Teflon Tubes | ID 1.5mm x OD 2.5mm | ~160cm | Guide the tendons |
| Bearing | (commercial, not printed) | 1 | Connects the fixed base to the rotating part |
| Magnets | Ø 2.5mm, height 1mm | 5 | Magnetic attachment |
| Component | Specification | Notes |
|---|---|---|
| Power Supply 1 | 7V / 3A | For the servo motors (via PCA9685) |
| Power Supply 2 | 12-13V / 2A | For the stepper motor (via CNC Shield) |
| USB Cables | USB-A to USB-B | To power the Arduinos |
| RPi 5 Power | Official 27W USB-C | For the Raspberry Pi 5 |
| Component | Notes |
|---|---|
| Heat shrink tubing | To insulate soldered joints |
| Hookup wire | For connections between components |
| Polyimide tape (Kapton) | Heat protection |
| Assorted screws (M3, M4) | To secure printed parts |
| Ecoflex™ 00-10 (RTV Silicone) | Optional — for fingertip grip pads |
The arm's structure is based on the open-source InMoov project. You need to download the STL files and print them.
| Part | Link |
|---|---|
| 🖐️ Hand and Forearm | InMoov — Hand and Forearm |
| 🖐️ Hand (I2 version) | InMoov — Hand I2 |
| 🏗️ Custom Base | Custom Base STL Files |
💡 Note: The base parts are not from InMoov. They were exclusively created by us for this project. You can find the STL files in the
Custom_Base_STLfolder in this repository.
💡 Tip: The Hand I2 version is the most recent and includes design improvements for the fingers. We recommend using this one.
We used a Bambu Lab A1 printer, but any FDM printer will work with these settings:
| Parameter | InMoov Parts | Custom Parts (Base) |
|---|---|---|
| Material | PLA | PLA |
| Nozzle Temperature | 220°C | 220°C |
| Bed Temperature | 65°C | 65°C |
| Layer Height | 0.20mm | 0.20mm |
| Infill Density | 30% | 25% |
| Wall Loops | 2 | 2 |
| Speed | Standard (100%) | Standard (100%) |
During printing, part edges may lift off the bed (called warping). If this happens:
- Enable "Brim" in your slicer — this creates a thin rim around the part that increases bed adhesion.
- You don't need to raise the bed temperature or lower the speed. With Brim enabled, standard settings work just fine.
After printing all the parts, it's time to assemble.
Follow the official InMoov instructions:
Each InMoov page has step-by-step photos and videos explaining how to thread the tendons (fishing line), install the return springs, and route the teflon tubes.
The base is not part of InMoov — we designed it ourselves in SolidWorks for this project. It includes:
- Main structure — houses the stepper motor, Raspberry Pi, and both Arduinos.
- Motor spacers — ensure the motor shaft reaches the rotary coupling.
- Arm coupling interface — two pieces (lower base + upper base) that connect the bearing to the arm.
- Rotary coupling — transmits motion from the stepper motor to the rotating platform.
- Motor mount bracket — secures the motor to the main structure.
💡 Why a custom base? The InMoov project includes a full torso, but we only needed the arm. So we designed a compact base that houses all the electronics and supports the arm's weight without overloading the motor.
| Assembly Process | Assembled Arm |
|---|---|
 |
 |
This is the part that requires the most attention. The system uses three processors working together:
┌──────────────┐ UART ┌──────────────────┐ UART ┌──────────────┐
│ Raspberry Pi │ ──────────► │ Arduino MEGA │ ──────────► │ Arduino Uno │
│ 5 (8GB) │ GPIO14→RX1 │ 2560 │ TX1→RX │ │
│ │ │ │ │ + CNC Shield │
│ Vision (AI) │ │ Central Control │ │ Stepper Motor│
└──────────────┘ └──────────────────┘ └──────────────┘
│
│ I2C (SDA/SCL)
▼
┌──────────────────┐
│ PCA9685 │
│ Servo Driver │
│ (16 channels) │
└──────────────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
MG996R Servos DS5160 Servo ...
(fingers/wrist) (elbow)
💡 Why three processors instead of one? The Raspberry Pi needs all its processing power for the AI-based vision. If it also had to control motors at the same time, everything would lag and the movements would be jerky. By splitting the tasks, each processor does what it's best at — no delays.
| From (Source) | To (Destination) | Type | Source Pins | Dest. Pins | What It Does |
|---|---|---|---|---|---|
| Raspberry Pi 5 | Arduino MEGA | UART | GPIO 14 (TX) | Pin 19 (RX1) | Sends vision data |
| Arduino MEGA | Arduino Uno | UART | Pin 18 (TX1) | Pin 0 (RX) | Sends base commands ('0','1','2') |
| Arduino MEGA | PCA9685 | I2C | SDA (20) / SCL (21) | SDA / SCL | Controls the 7 servo motors |
| Arduino Uno | CNC Shield V3 | Shield | Pins 2, 5, 8 | STEP, DIR, EN | Controls the stepper motor |
| 7V/3A Supply | PCA9685 | Wire | — | V+ terminal | Powers the servo motors |
| 12V/2A Supply | CNC Shield | Wire | — | 12V terminal | Powers the stepper motor |
⚠️ IMPORTANT: Connect the GND (ground) of all components together. Without a common ground, serial communication will not work.
The PCA9685 is a key component. Without it, you'd need 7 PWM pins from the Arduino to control 7 servos — and the Arduino doesn't have that many stable PWM pins. With the PCA9685:
- You control up to 16 servos using just 2 pins (I2C: SDA and SCL).
- The motor power comes from an external supply (7V), not from the Arduino. This protects the Arduino from current spikes.
The CNC Shield plugs directly into the Arduino Uno and accepts the A4988 driver. The connections are automatic — just plug it in and wire the stepper motor to the X-axis terminals.
💡 Why separate power supplies? Motors draw a lot of current and generate electrical noise. If they shared the same supply as the Arduinos, the Arduinos could randomly restart or behave erratically. Separate supplies = stable system.
For the full interactive wiring diagram, visit the Cirkit Designer project.
The firmware is the code that runs inside the Arduinos. You need to upload two different programs.
File: Arduino_Uno_CNC_Final.ino
What it does: Receives simple commands ('0', '1', or '2') from the Arduino MEGA and rotates the stepper motor to the correct position (left, center, or right).
How to upload:
- Open the Arduino IDE.
- Connect the Arduino Uno to your PC via USB.
- Go to Tools > Board > Arduino Uno.
- Open the
Arduino_Uno_CNC_Final.inofile. - Click Upload (→).
File: MotorDriver.ino (English) or MotorDriver_PT.ino (Portuguese)
What it does: Receives a data string from the Raspberry Pi (via UART), parses the values, and sends the correct PWM signals to each servo motor through the PCA9685.
How to upload:
- Install the required library:
- In Arduino IDE: Sketch > Include Library > Manage Libraries
- Search for "Adafruit PWM Servo Driver" and install it.
- Connect the Arduino MEGA to your PC via USB.
- Go to Tools > Board > Arduino Mega 2560.
- Open the
.inofile and click Upload (→).
The two Arduinos talk to each other via UART (serial port). The MEGA acts as "Master" and the Uno as "Slave":
- The MEGA receives a complete data string from the Raspberry Pi.
- It extracts the base value and sends it to the Uno.
- The Uno interprets the command and moves the stepper motor.
This is the most exciting part — the brain of the system.
File: ArmController.py (English) or ArmController_PT.py (Portuguese)
In the Raspberry Pi terminal, run:
pip install mediapipe opencv-python pyserial numpy --break-system-packagesThe
--break-system-packagesflag is required on Raspberry Pi OS Bookworm because it's very restrictive with pip packages by default.
Connect the GPIO 14 (TX) on the Raspberry Pi to Pin 19 (RX1) on the Arduino MEGA with a wire. Also connect GND between the two.
⚠️ Voltage levels: The Raspberry Pi operates at 3.3V and the Arduino at 5V. In practice, the Arduino reads 3.3V as HIGH without issues (RPi TX → Arduino RX). If you need to send data from the Arduino to the RPi, use a voltage divider.
python3 ArmController.pyThe system will:
- Start the camera.
- Detect your hand and shoulder using AI (MediaPipe).
- Calculate angles and finger states.
- Send the commands to the Arduino MEGA via UART.
The Raspberry Pi can run without a monitor. Use the Raspberry Pi Connect service to access the screen remotely from any browser. We recommend using a 5G mobile hotspot for better speed and lower latency.
If you want to test the computer vision without having the robot assembled, you can use the PC script.
File: VisionDebugger_PC.py (English) or VisionDebugger_PC_PT.py (Portuguese)
pip install opencv-python mediapipe numpyThe PC script uses a different MediaPipe API that needs model files. These are already included in the code/PC/ folder:
hand_landmarker.taskpose_landmarker_lite.task
These files must be in the same folder as the Python script.
python VisionDebugger_PC.pyYou'll see a window with your webcam feed and the detected points (hand and body) drawn on top. The calculated values (angles, finger states, etc.) appear as text on screen, making calibration and debugging easy.
💡 Recommendation: Use this tool to understand how the vision works before setting everything up on the Raspberry Pi. It's much easier to experiment and debug on a PC.
The system uses Google's MediaPipe framework, which contains pre-trained AI models that detect 21 hand points and 33 body points in real time.
Instead of measuring the distance between the fingertip and the wrist (which changes if you move closer to or further from the camera), the system calculates the angle at the finger's middle joint:
- Large angle (> 160°) → finger extended → Open
- Small angle (< 160°) → finger bent → Closed
💡 Why angles instead of distances? Angles don't change with your distance from the camera. If you step back, the points get closer together on screen, but the angle at the joint stays the same. This makes the system much more reliable.
The system analyzes your elbow angle (using the shoulder, elbow, and wrist points):
- Arm folded (angle < 70°) → Base rotates Left
- Arm extended (angle > 130°) → Base rotates Right
- In between → Base stays at Center
It compares the height of your wrist to your shoulder:
- Wrist far below shoulder → Elbow stays down
- Wrist at shoulder level → Elbow goes up
Since MediaPipe doesn't provide palm rotation directly, the system compares the relative position of the thumb and the pinky finger. Depending on your arm position (extended or bent), it automatically switches between using the X or Y axis to calculate rotation, and maps the result to 0°–180°.
The Raspberry Pi sends a formatted string to the Arduino MEGA every video frame. The structure is:
$<Base>,<Elbow>,<D1>,<D2>,<D3>,<D4>,<D5>,<Rotation>\n
| Field | Values | Meaning |
|---|---|---|
| Base | 0, 1, 2 | Left, Center, Right |
| Elbow | 0, 1 | Extended, Bent |
| D1 to D5 | 0, 1 | State of each finger (0=open, 1=closed) |
| Rotation | 0–180 | Wrist rotation servo angle |
Example:
$1,0,1,1,1,1,1,90\n
Base at center, elbow extended, all fingers closed, wrist rotated to 90°.
The Arduino MEGA uses the $ symbol to know where a message starts and \n to know where it ends. This ensures only complete messages are processed — if a message arrives cut off, it's simply ignored.
| Problem | Likely Cause | Solution |
|---|---|---|
| Parts lifting off the print bed | Warping | Enable Brim in your slicer |
| Servos jitter or don't move | Insufficient power | Check that the 7V supply is connected to PCA9685 (V+ terminal, not VCC) |
| Arduino resets on its own | Servos drawing current from Arduino | Use separate external power supplies; never power servos from the Arduino |
| Serial communication doesn't work | No shared ground | Connect GND of all devices together |
Serial port not found on RPi |
UART not enabled | Enable UART in raspi-config → Interface Options > Serial Port |
| MediaPipe is slow on RPi | Heavy processing | Verify multithreading is active (CameraStream class in the code) |
| Hand not detected | Poor lighting | Improve ambient lighting; avoid backlight |
- Frame rate: 15–20 FPS on the Raspberry Pi 5, sufficient for real-time control.
- Latency: Noticeable but low — suitable for telepresence applications.
- Finger accuracy: The angle-based method is robust and works regardless of distance to the camera or hand rotation.
- Stability: The UART protocol didn't drop any packets during testing — movements were smooth.
system-working.mp4
- Fingers only have two states (open/closed) — no intermediate positions.
- Shoulder doesn't include the forward pitch movement — it wasn't implemented.
- Camera is not fixed to the robot; it can misalign if the table is bumped.
- No haptic feedback — the operator can't feel what the robot touches.
If you want to take the project further, here are some suggestions:
- Proportional finger control: Instead of "open" or "closed", map your real finger angle directly to the servo angle. This way the robot copies the exact position.
- Integrated camera mount: Design a part that attaches the camera to the robot's base, eliminating misalignment issues.
- More degrees of freedom: Add the shoulder servo (Forward Pitch) so the arm can reach objects in front of it.
- Haptic feedback: Install pressure sensors on the fingertips and vibration motors in a glove, so the operator "feels" what the robot touches.
- Professional base mounting: Use clamps instead of adhesives to secure the robot to the workbench.
| Name | Contact |
|---|---|
| Henrique Abrantes | GitHub |
| Christian Rodrigues | GitHub |
| Rodrigo Maria | GitHub |
Project developed as part of the Robotics and Artificial Intelligence program at Escola Superior Náutica Infante D. Henrique (Portugal).
- InMoov — inmoov.fr — Gael Langevin's open-source project for the mechanical design of the hand and forearm.
- MediaPipe — Google AI — Computer vision framework for hand and pose detection.
- OpenCV — opencv.org — Image processing library.
- Raspberry Pi — raspberrypi.com — Raspberry Pi 5 documentation.
- Arduino — docs.arduino.cc — Arduino MEGA 2560 documentation.
- Cirkit Designer — Interactive wiring diagram — Full schematic.
For the complete technical analysis (torque calculations, mathematical foundations, detailed engineering decisions), see the full report in PDF included in this repository (Portuguese).
This project is licensed under the MIT License — you are free to use, modify, and distribute it.
Made with ❤️, PLA, and a lot of patience.
⭐ If this project helped you, leave a star on the repository!