A high-performance, real-time interactive Computer Vision Camera Engine powered by MediaPipe Tasks API and OpenCV. RetroLens enables users to project multidimensional visual "portals", paint neon strokes in mid-air, and trigger particle explosions through natural hand gestures—running smoothly at 45–60 FPS on standard CPUs without GPU acceleration.
flowchart TB
subgraph VideoInput["1. Video Capture & Threading"]
Webcam["Laptop / USB Webcam\n(DirectShow / CAP_DSHOW)"]
DroidCam["DroidCam / Virtual Camera\n(MJPG 720p / 1080p)"]
IPStream["IP Camera Stream\n(RTSP / HTTP Video)"]
ThreadedCam["ThreadedCamera Worker\n(Non-blocking I/O, Buffer Flush)"]
end
subgraph VisionPipeline["2. Hand Tracking & Kinematics"]
RGB["SRGB Frame Conversion"]
MPTask["MediaPipe HandLandmarker\n(21 3D Landmarks per Hand)"]
EMA["PointEMA Smoother\n(Zero-Heap Exponential Moving Average)"]
end
subgraph GesturePortal["3. Gesture Engine & Geometry"]
GiantPortal["10-Finger Giant Portal\n(Convex Hull Dual Hands)"]
SinglePortal["5-Finger Portal\n(Single Hand Geometry)"]
DualPortal["3D Mesh Mode\n(Dual Stapped Portals)"]
PinchDraw["Air Drawing Canvas\n(Thumb + Index Pinch)"]
BloomParticle["Neon Bloom Explosion\n(Fist-to-Palm Shockwave)"]
end
subgraph ShaderEngine["4. 15-Filter Bank Shader"]
LUT["Precomputed LUT Shaders\n(Dual-Tone, Thermal)"]
Kernel["Vectorized Matrix Kernels\n(Sepia, Sketch, Glitch, Neon)"]
Downscale["Hierarchical Downscaled Blur\n(4x Accelerated Creamy Bokeh)"]
end
subgraph OutputSink["5. Compositor & Output"]
AlphaBlend["Alpha Mask Compositor\n(Convex Hull Mask + Portal Edge Glow)"]
Display["OpenCV High-Speed Viewport\n(Real-Time HUD & Gesture Badges)"]
Recorder["MP4 VideoWriter & Snapshot Capture\n(Dynamic Framerate Sync)"]
end
Webcam --> ThreadedCam
DroidCam --> ThreadedCam
IPStream --> ThreadedCam
ThreadedCam --> RGB --> MPTask --> EMA
EMA --> GiantPortal
EMA --> SinglePortal
EMA --> DualPortal
EMA --> PinchDraw
EMA --> BloomParticle
GiantPortal --> AlphaBlend
SinglePortal --> AlphaBlend
DualPortal --> AlphaBlend
PinchDraw --> AlphaBlend
BloomParticle --> AlphaBlend
ShaderEngine <--> AlphaBlend
LUT --> ShaderEngine
Kernel --> ShaderEngine
Downscale --> ShaderEngine
AlphaBlend --> Display
AlphaBlend --> Recorder
- 👐 The 10-Finger Giant Portal: Spread all ten fingers from both hands to span a giant, non-crossing convex hull portal across your viewport.
- 🖐️ 5-Finger Single Hand Portal: Project a dynamic pentagonal portal tracking individual fingertip vectors with one hand.
- 💠 3D Mesh Mode (Dual Layer Portals): Clench both fists or press
Mto split the portal into two 3D-stacked geometric layers with distinct filter effects. - 💥 Neon Bloom Particle Explosion: Clench your fist and snap your palm wide open to trigger an explosion of glowing neon particles from the center of your hand.
- 🎨 Air Drawing (Neon Pinch): Pinch your thumb and index finger together to paint glowing neon ribbons directly in mid-air.
- ⚡ Ultra-Fast 15 Filter Bank:
MONO— High-contrast cinematic monochrome.INVERT— Negative photographic film.BLUR— Fast hierarchical downscaled blur (creamier look, 4x faster).SEPIA— Accurate BGR sepia transformation matrix (zero green/blue tint bias).DUAL-TONE— Synthwave Violet & Warm Gold accelerated via precomputed Look-Up Tables (1.2ms).PIXELATE— 8-bit retro mosaic with safe coordinate boundary clamps.THERMAL— Infrared heat map visualization using OpenCV Jet colormaps.SKETCH— Hand-drawn graphite pencil sketch.GLITCH— RGB chromatic aberration with horizontal scanline displacement.NEON— High-speed edge glow contour extraction (< 4ms).CARTOON— Comic book cel-shaded aesthetic.RAINBOW— Dynamic sinusoidal rainbow spectrum wave.MAGNIFY— Optical magnifying glass distortion with boundary protection.MATRIX— Sci-fi cascading digital rain.GALAXY— Deep-space nebula background with AI Selfie Segmenter blending.
- 📸 Snapshot & MP4 Video Recording: Capture high-res photos (
S) with screen flash effects, and record MP4 videos (R) with auto-synchronized framerate pacing. - 📷 Universal Camera Driver (
camera.py): Dedicated multi-threaded background worker with DirectShow (--dshow) support, MJPG codec unlock, and IP camera streaming.
| Key / Gesture | Action |
|---|---|
| Both Hands Spread (10 Fingers) | Open 10-Finger Giant Portal |
| One Hand Spread (5 Fingers) | Open 5-Finger Dynamic Portal |
| Thumb + Index Pinch | Mid-Air Neon Drawing |
| Fist ➔ Open Palm | Trigger Neon Bloom Particle Explosion |
Space |
Toggle portal activation (Open / Close) |
N / P |
Switch to Next / Previous visual filter |
M |
Toggle 3D Mesh Mode (Dual Stapped Portals) |
S |
Take snapshot (saved to screenshots/) |
R |
Start / Stop MP4 video recording |
C |
Clear air drawing canvas |
Q / Esc |
Quit application safely |
git clone https://github.com/stenlysayd/retrolens.git
cd retrolens
pip install -r requirements.txt-
Default Laptop / USB Webcam:
python main.py
-
DroidCam / USB Virtual Camera (DirectShow backend on Windows):
python main.py --source 1 --dshow
-
Mobile Camera via IP Webcam App / DroidCam WiFi Client:
python main.py --source http://192.168.1.50:8080/video
-
Force 16:9 Widescreen Crop:
python main.py --crop169
Custom performance parameters can be adjusted via config.json or config.py:
{
"EMA_ALPHA": 0.75,
"PROCESS_WIDTH": 960,
"PROCESS_HEIGHT": 540,
"DISPLAY_WIDTH": 1280,
"DISPLAY_HEIGHT": 720,
"GESTURE_COOLDOWN_SEC": 0.4,
"PINCH_RATIO_THRESHOLD": 0.25,
"MAX_PARTICLES": 120,
"SCREENSHOT_DIR": "screenshots"
}Contributions are welcomed! Check out CONTRIBUTING.md for guidelines on adding custom shaders to filters.py.
RetroLens processes all camera frames locally. See SECURITY.md for details on camera privacy.
This project is licensed under the MIT License © 2026 Stenly Sayd.