Sign It or Miss It is an interactive Unity game that use computer vision and hand tracking to provide an immersive experience. The game uses OpenCV and MediaPipe to track hand movements and generate a corresponding 3D hand in Unity. Players must pass their virtual hand through walls with cutout shapes to progress through the game.
- Python
- OpenCV
- MediaPipe
- cvzone
- Unity
- C#
- Blender
- Hand Tracking with OpenCV & MediaPipe: Real-time hand landmark detection for accurate gesture recognition.
- Dynamic Wall Cutouts: Walls with varying hand-shaped cutouts challenge the player's precision and reaction time.
- UDP Communication: Hand tracking data is sent from Python to Unity using UDP sockets.
- Engaging Gameplay: The game gets progressively challenging with different wall patterns and speeds.
- Python 3.11.x
- Unity (2023+ recommended)
- A webcam or external camera for hand tracking
-
Clone the Repository
git clone https://github.com/4SumMNNIT/SIMI.git cd SIMI -
Set Up Python Environment
cd backend python -m venv myenv ./myenv/Scripts/activate pip install -r requirements.txt -
Run the Hand Tracking Script
python main.py
-
Open the Unity Project
- Open Unity and load the
Unityproject. - Open the
Scenesfolder and drag and drop theMenuscene to heirarchy window. - Unity will prompt to install
TextMesh Pro EssentialsandExamples & Extrasmodules. - Install both of these.
- Run the game in Unity while keeping the Python script running.
- Open Unity and load the
-
Play the Game!
- Move your hand in front of the camera.
- Try to match your hand with the cutout shape in the walls.
- Advance through the level and test your precision!
Team Name: 4Sum
