🤟 ISL Interpreter

Real-time Indian Sign Language to Text Converter

No app install required — works directly in your browser

---

🧠 About The Project

India has 63 lakh+ deaf and hard-of-hearing individuals who face communication barriers daily. Existing solutions require expensive hardware or app installations.

ISL Interpreter is a browser-based web application that:

• Detects hand gestures in real-time using your webcam
• Converts Indian Sign Language gestures to text and speech
• Works on any device with a browser — zero installation needed
• Saves your translation history for later reference

---

✨ Features

Feature	Description
🎥 Real-time Detection	Live webcam feed with instant gesture recognition
🤖 AI-Powered	MediaPipe Hands + TensorFlow.js for accurate classification
📝 Sentence Builder	Collects letters → forms complete words and sentences
🔊 Text to Speech	Converts translated text to audio output
📊 Confidence Score	Shows how confident the model is for each gesture
🗂️ History	Saves all translated sessions for logged-in users
🔐 Authentication	Secure user login and registration

---

🖼️ Screenshots

Screenshots will be added once the UI is complete

Translator Page	History Page

---

🛠️ Tech Stack

Frontend

• React 18 — UI framework
• Tailwind CSS v4 — Styling
• MediaPipe Hands — Hand landmark detection (21 points)
• TensorFlow.js — Gesture classification using pre-trained ISL model
• Axios — API communication

Backend

• Java Spring Boot — REST API server
• Spring Security — Authentication
• Spring Data JPA — Database ORM
• MySQL — Data storage

Deployment

• Vercel — Frontend hosting
• Render.com — Backend hosting
• Railway.app — Database hosting

---

🚀 Getting Started

Prerequisites

• Node.js v20+
• Java JDK 17+
• MySQL
• Maven

Frontend Setup

# Clone the repo
git clone https://github.com/ManuStu-web/ISL-Interpreter.git

# Go to frontend
cd ISL-Interpreter/frontend

# Install dependencies
npm install

# Run development server
npm run dev

Open http://localhost:5173 in your browser.

Backend Setup

# Go to backend
cd ISL-Interpreter/backend

# Run with Maven
./mvnw spring-boot:run

Configure your MySQL credentials in application.properties before running.

---

🏗️ Project Architecture

ISL-Interpreter/
├── Frontend/                      # Unified React App (Vite)
│   ├── public/                    # Static global assets (videos, meta icons)
│   ├── src/
│   │   ├── assets/                # Images and internal assets
│   │   ├── Components/            # Core specific components (NavBar, Interpreter, Footer)
│   │   ├── ComponetsAbout/        # Informational specific components (About, Contact, Heading)
│   │   ├── pages/                 # Unified page views (Home.jsx, Home.css)
│   │   ├── App.jsx                # Main Routing setup / Layout Wrapper
│   │   ├── index.css              # Global styles & Tailwind injections
│   │   └── main.jsx               # React DOM injection point
│   ├── package.json               # Frontend dependencies & scripts
│   └── vite.config.js             # Vite configuration
│
├── homepage/                      # OBSOLETE (Can be safely deleted)
│   └── ...                        # Old code we successfully merged into Frontend
│
└── README.md                      # Global GitHub Documentation

---

🔄 How It Works

User shows hand gesture
        ↓
getUserMedia (webcam access)
        ↓
Canvas API (frame capture)
        ↓
MediaPipe Hands (21 landmarks)
        ↓
TensorFlow.js (gesture classification)
        ↓
React UI (display text)
        ↓
Spring Boot API (save history)

---

🌐 Live Demo

🔗 Coming Soon — Will be deployed on Vercel

---

📄 License

This project is built for educational purposes as part of Project-I (Semester VI).

---

🙏 Acknowledgements

• MediaPipe by Google
• TensorFlow.js
• ISL Dataset — Pre-trained model

---

Made with ❤️ for 63 lakh+ deaf and hard-of-hearing Indians

⭐ Star this repo if you find it useful!