An intelligent, webcam-based eye-controlled cursor system that enables hands-free computer interaction using real-time facial landmark tracking, blink detection, and gaze-based interaction zones.
This project demonstrates how modern computer vision techniques can be used to build accessible and low-cost human-computer interaction systems without requiring expensive eye-tracking hardware.
Built using Python, OpenCV, and MediaPipe Face Landmarker Task API, the system transforms a standard laptop webcam into a functional eye-tracking interface capable of:
- 🖱️ Cursor Movement
- 👁️ Blink-to-Click Interaction
- 📜 Gaze-Based Scrolling
- 🔒 Click Lock / Unlock Zones
- 🎯 Real-Time Face & Gaze Visualization
- ⚡ Adjustable Smoothing & Sensitivity Controls
Read the full project post on LinkedIn:
Traditional eye-tracking systems often require:
- Specialized IR cameras
- Dedicated sensors
- Expensive proprietary hardware
This project aims to provide a hardware-efficient and accessible alternative that works using only a standard laptop webcam.
Potential applications include:
- ♿ Assistive technology for users with limited mobility
- 🧠 Human-Computer Interaction (HCI) research
- 👁️ Gaze-based UI systems
- 🖥️ Hands-free computing environments
- 🎓 Educational & research purposes
Tracks facial landmarks and maps eye/head movement to screen coordinates.
Moves the system cursor using gaze/head tracking with configurable:
- Sensitivity
- Smoothing
- Dead-zone stabilization
Detects eye blinks using Eye Aspect Ratio (EAR) calculations.
Supports:
- Single-click interactions
- Configurable blink threshold
- Frame-based blink validation
Custom gaze-controlled zones enable scrolling without touching the mouse or keyboard.
- Move cursor into scroll zone
- Hold gaze briefly
- Continuous scrolling begins
- Exit zone to stop scrolling
Allows users to temporarily disable clicking while still moving the cursor freely.
Useful for:
- Reading
- Browsing
- Watching content
- Preventing accidental clicks
Visual overlays include:
- Nose/gaze tracking point
- Interaction zones
- Status indicators
- Debug information
- FPS counter
All major parameters can be customized from a single settings file.
Examples:
CURSOR_SENSITIVITY
SMOOTHING
BLINK_THRESHOLD
BLINK_FRAMES
SCROLL_SPEED
ZONE_WIDTH
ZONE_HEIGHT- Python
- OpenCV
- MediaPipe Task API
- NumPy
- PyAutoGUI
- Facial Landmark Detection
- Eye Aspect Ratio (EAR)
- Gaze Mapping
- Cursor Smoothing
- Motion Filtering
- Real-Time Tracking
- Human-Computer Interaction (HCI)
flowchart TD
A[Webcam Feed] --> B[MediaPipe Face Landmark Detection]
B --> C[Eye & Head Tracking]
C --> D[Cursor Mapping Engine]
D --> E[Cursor Movement]
D --> F[Blink Detection]
D --> G[Interaction Zones]
F --> H[Mouse Click]
G --> I[Scroll System]
G --> J[Click Lock / Unlock]
E --> K[System Cursor Control]
I --> K
H --> K
J --> K
.
├── config
│ └── settings.py
├── core
│ ├── blink_detector.py
│ ├── calibration.py
│ ├── camera.py
│ ├── cursor_controller.py
│ ├── eye_tracker.py
│ ├── face_mesh.py
│ └── interaction_zones.py
├── models
│ └── face_landmarker.task
├── ui
│ ├── calibration_screen.py
│ ├── overlay.py
│ ├── zones_overlay.py
│ └── debug_overlay.py
├── utils
│ ├── drawing_utils.py
│ ├── fps_counter.py
│ └── math_utils.py
├── main.py
├── requirements.txt
└── README.md
git clone https://github.com/your-username/eye-controlled-cursor.git
cd eye-controlled-cursorpip install -r requirements.txtOr manually:
pip install opencv-python mediapipe pyautogui numpy pygetwindowpython main.pyPress:
ESC
to exit the application.
All major system parameters are configurable from:
config/settings.py
Example:
# Cursor Movement
CURSOR_SENSITIVITY = 1.5
SMOOTHING = 5
# Blink Detection
BLINK_THRESHOLD = 0.21
BLINK_FRAMES = 2
# Interaction Zones
REST_ZONE_DWELL_TIME = 1.2
SCROLL_ZONE_DWELL_TIME = 0.8
# Scroll
SCROLL_SPEED = 3
# Zone Dimensions
ZONE_WIDTH = 120
ZONE_HEIGHT = 300The project uses Eye Aspect Ratio (EAR) to detect blinks.
EAR = (vertical_distance_1 + vertical_distance_2) / (2 * horizontal_distance)When eyes close:
- Vertical distance decreases
- EAR drops below threshold
- Blink is detected
The left edge of the screen acts as a scroll activation zone.
- Move cursor into scroll zone
- Hold briefly
- Scroll activates
- Move out of zone to stop
Upper half:
- Scroll Up
Lower half:
- Scroll Down
The right edge of the screen acts as a click toggle zone.
- Move cursor into rest zone
- Hold briefly
- Clicking gets disabled/enabled
- Cursor movement remains active
flowchart LR
A[Webcam Input]
--> B[Face Landmark Detection]
--> C[Eye Tracking]
--> D[Gaze Mapping]
--> E[Cursor Smoothing]
--> F[Mouse Actions]
F --> G[Move Cursor]
F --> H[Blink Click]
F --> I[Scroll Zones]
Eye.Screen.recording.1.mp4
- Face tracking
- Gaze point
- Scroll zones
- Click status
- FPS counter
- Debug information
This project is also being explored from a research and accessibility perspective.
- Low-cost eye tracking systems
- Human-computer interaction
- Accessibility technology
- Webcam-based gaze estimation
- Real-time computer vision interfaces
Some of the major engineering challenges included:
- Cursor drift stabilization
- Webcam noise reduction
- Blink detection accuracy
- Smooth scrolling behavior
- Real-time performance optimization
- Balancing sensitivity and usability
- 🎯 Advanced calibration system
- 🧠 AI-based adaptive smoothing
- 👄 Gesture controls
- 🖱️ Dwell-click interaction
- 📱 Multi-monitor support
- ⚡ Better gaze estimation
- 🎨 Enhanced UI/UX
- 🔊 Voice-assisted controls
- Works best under good lighting
- Webcam quality affects accuracy
- Rapid head movement may reduce stability
- Long usage may require recalibration
This project demonstrates concepts from:
- Computer Vision
- Human-Computer Interaction
- Real-Time Systems
- Accessibility Engineering
- Facial Landmark Tracking
- Gaze Estimation
- UI/UX Interaction Design
Built by Harsh Tripathi in collaboration with Iqra Tabassum.
Special thanks to the open-source tools and libraries that made this project possible:
- MediaPipe
- OpenCV
- NumPy
- PyAutoGUI
- Python Community
Consider giving the repository a star ⭐
It helps support future improvements and accessibility-focused innovation.