A real-time hand gesture controlled drone simulator built using Python, MediaPipe, OpenCV, and Pygame. The system detects hand gestures through a webcam, recognizes the number of fingers shown, and converts them into drone movement commands inside a 2D drone simulation.
Traditional drone control requires joysticks or remote controllers. This project demonstrates an alternative Human-Computer Interaction (HCI) approach where a user controls a virtual drone using only hand gestures captured through a webcam.
The application performs real-time hand landmark detection using Google's MediaPipe framework and maps recognized gestures into drone navigation commands. A custom-built drone simulator developed in Pygame visualizes the drone movement while displaying command status and system information through an interactive HUD.
This project was developed as part of the Design Engineering (DE) coursework.
- Build a touchless drone control interface.
- Detect hand landmarks in real time.
- Count raised fingers accurately.
- Map gestures into navigation commands.
- Simulate drone movement without physical hardware.
- Demonstrate Human Computer Interaction using Computer Vision.
- 📷 Real-time webcam hand tracking
- ✋ Finger counting using MediaPipe
- 🚁 Gesture controlled drone simulation
- 🎮 Interactive 2D drone environment
- 📊 Live command display (HUD)
- ⚡ Smooth real-time movement
- 🔄 Animated propellers
- 🎨 Custom drone graphics created entirely in Pygame
- 🟢 Takeoff/Landing status indicator
- 📈 FPS monitoring
| Category | Technology |
|---|---|
| Programming Language | Python |
| Computer Vision | OpenCV |
| Hand Tracking | MediaPipe Tasks API |
| Graphics | Pygame |
| Mathematics | Python Math Library |
| Webcam | OpenCV VideoCapture |
Hetanjali Vaghela — Robotics & Automation Engineering