A real-time facial emotion detection system built with Python, TensorFlow/Keras, and OpenCV. It uses a custom-trained Convolutional Neural Network to classify seven different emotions from a live webcam feed.
Point your webcam at a face and the model tells you what emotion it's reading — in real time. It draws a bounding box around each face it detects and overlays the predicted emotion along with a confidence score. If the model isn't confident enough (below 60%), it just says "Uncertain" instead of guessing.
The seven emotions it can classify are:
- Angry
- Disgust
- Fear
- Happy
- Neutral
- Sad
- Surprise
The CNN was trained from scratch on the FER2013 dataset, which contains 48×48 grayscale images of faces labeled with one of the seven emotion categories.
The architecture is a three-block convolutional network:
| Block | Filters | Operations |
|---|---|---|
| 1 | 64 | Conv → BN → ELU → Conv → BN → ELU → MaxPool → Dropout |
| 2 | 128 | Conv → BN → ELU → Conv → BN → ELU → MaxPool → Dropout |
| 3 | 256 | Conv → BN → ELU → Conv → BN → ELU → MaxPool → Dropout |
After the conv blocks, it flattens and passes through a fully connected layer (Dense 512 + BN + ELU + Dropout) before the final softmax output layer with 7 classes.
Training used:
- Optimizer: Adam
- Loss: Categorical Crossentropy
- Data augmentation: random rotation (±10°), zoom (10%), horizontal flip
- Checkpoint: saves the best model based on validation accuracy
The live detection script (emotion_recogniser.py) uses OpenCV's Haar Cascade classifier to find faces in each frame. For every face it finds, it crops and resizes the region to 48×48, normalises the pixel values, runs a prediction, and renders the result back on the frame.
FacialEmotionRecogniser/
│
├── main/
│ ├── emotion_recogniser.py # Run this for live detection
│ ├── final_model.ipynb # Training notebook
│ └── haarcascade_frontalface_default.xml
│
├── fix_model.py # Re-saves model to fix serialization issues
├── new.py # Alternative model fix with TF import
├── requirements.txt
└── README.md
Note: The trained model (
best_emotion_model.keras) and the FER2013 dataset are not included in the repo due to file size. You'll need to train the model yourself using the notebook, or download a pre-trained version separately.
- Python 3.8+
- A working webcam
- The trained model file (
.keras) placed insaved_model/
git clone https://github.com/Krishnagangwal/FacialEmotionRecogniser.git
cd FacialEmotionRecogniser
pip install -r requirements.txtThe main dependencies are:
tensorflow
keras
opencv-python
numpy
pandas
matplotlib
seaborn
scikit-learn
Open main/final_model.ipynb in Jupyter and run the cells top to bottom. Make sure the FER2013 CSV (fer2013.csv) is at ../dataset/fer2013.csv relative to the notebook. The best model gets saved to saved_model/best_emotion_model.keras.
Update the paths in emotion_recogniser.py to point to your saved model and the Haar Cascade XML, then run:
python main/emotion_recogniser.pyA window will open showing your webcam feed with emotion labels. Press Q to quit.
If you run into Keras deserialization warnings when loading the model, run:
python fix_model.pyThis re-saves the model in a clean format (fixed_model.keras) that avoids optimizer mismatch issues.
- The model was trained on FER2013, which is a fairly noisy dataset — don't expect perfect accuracy in all lighting conditions
- Predictions can get shaky with partial occlusion or extreme head angles
- The absolute paths in
emotion_recogniser.pyare currently hardcoded — you'll need to update them to match your local setup before running