🖐️ Gesture Based Controller

Control Your Computer Using Hand Gestures

A real-time hand gesture recognition system built with Python, MediaPipe, OpenCV, and PyAutoGUI that transforms your webcam into a virtual mouse, presentation remote, and game controller.

---

📖 Overview

Gesture Based Controller is a computer vision project that allows users to control their computer without touching a mouse or keyboard.

Using a webcam, the system detects hand landmarks in real time and converts predefined gestures into mouse movements and keyboard shortcuts.

The project supports three operating modes:

• 🖥 Desktop Control
• 🎤 Presentation Control
• 🎮 Game Control

---

✨ Features

🖥 Desktop Mode

• 🖱 Cursor Movement
• 👆 Left Click
• 👆👆 Right Click (Double Pinch)
• 📜 Scroll Up
• 📜 Scroll Down
• 🔄 Alt + Tab Application Switcher
• 📑 Ctrl + Tab Browser Switching
• ⌨ Press Enter

---

🎤 Presentation Mode

Control presentations without touching the keyboard.

• 👉 Swipe Right → Next Slide
• 👈 Swipe Left → Previous Slide
• ✋ Open Palm → Start Slideshow (F5)
• ✊ Closed Fist → Exit Slideshow (Esc)

---

🎮 Game Mode

Play arrow-key based games using gestures.

Gesture	Keyboard Action
☝ Index Finger	Right Arrow
✌ Peace Sign	Left Arrow
🖐 Open Palm	Up Arrow (Jump)
✊ Closed Fist	Down Arrow (Duck)

Works great with games like:

• Subway Surfers
• Temple Run
• Chrome Dino
• Any Arrow-Key Based Game

---

🔄 Mode Switching

Raise only your Pinky Finger for 1 second to switch between modes.

Desktop
   ↓
Presentation
   ↓
Game
   ↓
Desktop

---

🧠 Technologies Used

• Python
• OpenCV
• MediaPipe Tasks API
• PyAutoGUI
• NumPy

---

📦 Installation

1. Clone the Repository

git clone https://github.com/manisbhusal/Gesture-Based-Controller.git
cd Gesture-Based-Controller

---

2. Create Virtual Environment (Recommended)

Windows

python -m venv .venv
.venv\Scripts\activate

Linux / macOS

python3 -m venv .venv
source .venv/bin/activate

---

3. Install Dependencies

python -m pip install opencv-python
pip install opencv-python mediapipe pyautogui numpy

`` If pip isn't found

Install it first:

sudo apt update
sudo apt install python3-pip

---

## 4. Run the Project

```bash
python gesture.py

`` If you're using a virtual environment (recommended)

Create one:

python3 -m venv .venv

Activate it:

source .venv/bin/activate

Then install everything:

pip install opencv-python mediapipe pyautogui numpy

Run your program:

python gesture.py

The MediaPipe hand tracking model will automatically download during the first execution.

Press Q to quit.

---

🎮 Gesture Reference

Desktop Mode

Gesture	Action
☝ Index Finger	Move Cursor
🤏 Pinch	Left Click
🤏🤏 Double Pinch	Right Click
✌ Index + Middle	Scroll Up
✊ Closed Fist	Scroll Down
🖐 Four Fingers	Alt + Tab
🤟 Middle + Ring + Pinky	Ctrl + Tab
🤟 Index + Middle + Ring	Enter

---

Presentation Mode

Gesture	Action
👉 Swipe Right	Next Slide
👈 Swipe Left	Previous Slide
🖐 Open Palm	Start Presentation
✊ Closed Fist	Exit Presentation

---

Game Mode

Gesture	Keyboard
☝ Index	Right Arrow
✌ Peace	Left Arrow
🖐 Open Palm	Up Arrow
✊ Closed Fist	Down Arrow

---

⚙️ Smart Features

• ✅ Cursor Smoothing
• ✅ Dynamic Hand Scaling
• ✅ False Gesture Protection
• ✅ Gesture Cooldowns
• ✅ Double Pinch Detection
• ✅ Swipe Detection
• ✅ Gesture Latching
• ✅ Automatic MediaPipe Model Download

---

📁 Project Structure

Gesture-Based-Controller/
│
├── gesture.py
├── README.md
├── requirements.txt
└── hand_landmarker.task

---

💻 Requirements

• Python 3.10+
• Webcam
• Windows, Linux or macOS

Note: Windows provides the best compatibility with PyAutoGUI. On Linux, using X11 is recommended instead of Wayland.

---

⚠️ Tips for Best Performance

• Use a well-lit environment.
• Keep only one hand visible.
• Position your webcam directly in front of you.
• Avoid cluttered backgrounds.
• Maintain a consistent distance from the camera.

---

🚀 Future Roadmap

• Multi-Hand Support
• Volume Controller
• Brightness Controller
• Media Controls
• Drag & Drop
• Virtual Keyboard
• Custom Gesture Training
• AI Gesture Recognition
• Voice + Gesture Hybrid Control
• Settings GUI
• User Profiles

---

🤝 Contributing

Contributions are welcome!

1. Fork this repository

2. Create a new branch

git checkout -b feature-name

3. Commit your changes

git commit -m "Add new feature"

4. Push your branch

git push origin feature-name

5. Open a Pull Request.

---

🐞 Known Limitations

• Requires adequate lighting.
• Supports only one hand.
• Extremely fast hand movements may not be detected.
• Webcam quality directly affects tracking accuracy.
• Some keyboard shortcuts may differ between operating systems.
• PyAutoGUI has limited support on Wayland (Linux).

---

📄 License

This project is licensed under the MIT License.

Feel free to use, modify, and distribute it.

---

🙏 Acknowledgements

Special thanks to:

• MediaPipe
• OpenCV
• PyAutoGUI
• NumPy
• Python Community

---

⭐ If you like this project

Give it a ⭐ on GitHub and consider sharing it with others.