DeepFake Sentinel

DeepFake Sentinel is a complete Flask + TensorFlow web application for deepfake detection across images, videos, and live webcam frames. The Flask backend serves the primary UI and exposes JSON APIs for upload prediction, webcam frame analysis, Grad-CAM visualization, health checks, and evaluation metrics.

The app runs immediately in deterministic demo mode when no trained checkpoint exists. For production use, train on a real dataset and save the model to models/deepfake_detector.keras.

Features

• Image and video fake/real detection
• Browser drag-and-drop uploads
• Live webcam frame analysis
• REAL / FAKE verdict, confidence, fake/real probabilities
• Multi-face detection with per-face verdicts
• Annotated face bounding-box outputs
• Grad-CAM overlays for trained Keras models
• PDF report generation
• SQLite upload history, audit logs, and analytics dashboard
• Optional Socket.IO live prediction channel
• Forensic residual heatmaps in demo mode and as Grad-CAM fallback
• TensorFlow/Keras transfer-learning training pipeline
• EfficientNetB0 or Xception backbone
• OpenCV image preprocessing and video frame extraction
• Evaluation metrics, confusion matrix, ROC curve, and training charts
• Flask API routes connected to the frontend
• Optional Vite React + Framer Motion frontend
• Docker and Docker Compose support
• Black, white, and yellow glassmorphism UI

Project Structure

deepfake-detector/
  backend/
    app.py
    api/
    model/
    preprocessing/
    utils/
    requirements.txt
  frontend/
    templates/index.html
    static/css/styles.css
    static/js/app.js
  react-frontend/
  data/
  models/
  notebooks/
  scripts/
  Dockerfile
  docker-compose.yml
  .env
  README.md

Screenshots

Run the Flask app and open http://127.0.0.1:5000 to capture:

• Upload detector with image or video preview
• Prediction card with confidence meter
• Grad-CAM viewer
• Webcam live detector
• Metrics panel after evaluation

Quick Start

Localhost will not work just by opening the folder. Start the Flask server first, then open the localhost URL.

Windows easiest option:

Double-click RUN_APP.bat

Keep that terminal window open, then open:

http://127.0.0.1:5000

Manual setup:

cd deepfake-detector
python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
python -m backend.app

macOS/Linux:

source .venv/bin/activate
pip install -r requirements.txt
python -m backend.app

Open:

http://127.0.0.1:5000

Environment

Configuration lives in .env.

MODEL_PATH=models/deepfake_detector.keras
UPLOAD_FOLDER=backend/uploads
OUTPUT_FOLDER=backend/outputs
REPORT_FOLDER=backend/reports
METRICS_FOLDER=models/metrics
DATABASE_PATH=data/deepfake_sentinel.sqlite3
IMAGE_SIZE=224
MODEL_THRESHOLD=0.65
THRESHOLD_PATH=models/metrics/threshold.json
CALIBRATION_PATH=models/metrics/calibration.json
REAL_RECALL_TARGET=0.94
MAX_FALSE_POSITIVE_RATE=0.08
MAX_VIDEO_FRAMES=18
DEMO_MODE=true
FACE_DETECTOR=auto
FACE_CROP_MARGIN=0.36
WEBCAM_SMOOTHING=0.35

Set DEMO_MODE=false in production if the app must fail fast when the model checkpoint is missing.

API Usage

Health:

curl http://127.0.0.1:5000/api/health

Upload prediction:

curl -X POST http://127.0.0.1:5000/api/predict \
  -F "file=@sample.jpg"

Webcam frame prediction accepts a JSON data URL:

curl -X POST http://127.0.0.1:5000/api/webcam/predict \
  -H "Content-Type: application/json" \
  -d "{\"image\":\"data:image/jpeg;base64,...\"}"

Metrics:

curl http://127.0.0.1:5000/api/metrics

Analytics dashboard data:

curl http://127.0.0.1:5000/api/analytics

Scan history:

curl "http://127.0.0.1:5000/api/history?label=FAKE&limit=20"

Download a PDF report:

curl -L http://127.0.0.1:5000/api/reports/<job_id> --output report.pdf

Optional Socket.IO live channel:

emit webcam_frame with { image: "data:image/jpeg;base64,..." }
listen for prediction or prediction_error

Smoke test a running server:

python scripts/api_smoke_test.py --base-url http://127.0.0.1:5000

Dataset Layout

The training loader expects this structure:

data/processed/
  train/
    real/
    fake/
  val/
    real/
    fake/
  test/
    real/
    fake/

Class labels are fixed as real = 0 and fake = 1.

Dataset Sources

FaceForensics++:

• Request/download access from the official FaceForensics++ project.
• Extract original videos into data/raw/faceforensics/real.
• Extract manipulated videos into data/raw/faceforensics/fake.
• Convert videos into frames with backend/preprocessing/preprocess_videos.py.

Celeb-DF:

• Download Celeb-DF according to the dataset license.
• Put authentic videos in data/raw/celebdf/real.
• Put synthesis videos in data/raw/celebdf/fake.
• Extract balanced frames before splitting.

DeepFake Detection Challenge:

• Download DFDC from Kaggle.
• Use metadata JSON to route videos into real and fake.
• Extract frames and split into train/val/test.

Preprocessing

Extract frames from videos:

python -m backend.preprocessing.preprocess_videos \
  --input-dir data/raw/dfdc \
  --output-dir data/interim/dfdc_frames \
  --frames-per-video 24

Preprocess images with optional face cropping and augmentation:

python -m backend.preprocessing.preprocess_images \
  --input-dir data/interim/dfdc_frames \
  --output-dir data/interim/dfdc_faces \
  --image-size 224 \
  --face-crop \
  --face-detector auto \
  --face-margin 0.36 \
  --augment

Split into train/val/test:

python scripts/split_dataset.py \
  --input-dir data/interim/dfdc_faces \
  --output-dir data/processed \
  --train 0.7 \
  --val 0.15

Balance an already split dataset when one class dominates:

python scripts/balance_dataset.py \
  --input-dir data/processed \
  --output-dir data/processed_balanced \
  --strategy oversample

Training

EfficientNetB0:

python -m backend.model.train \
  --data-dir data/processed \
  --output-dir models \
  --backbone efficientnet_b0 \
  --weights imagenet \
  --balance-strategy class_weight \
  --real-priority 1.35 \
  --real-recall-target 0.94 \
  --max-false-positive-rate 0.08 \
  --epochs 14 \
  --fine-tune-epochs 5

Xception:

python -m backend.model.train \
  --data-dir data/processed \
  --output-dir models \
  --backbone xception \
  --weights imagenet

The checkpoint is saved to:

models/deepfake_detector.keras

Training also saves:

• models/metrics/threshold.json
• models/metrics/calibration.json
• models/metrics/latest_training.json
• TensorBoard logs in models/metrics/tensorboard

The selected threshold prioritizes real-image recall and lower false positives, so inference no longer blindly uses 0.5.

Evaluation

python -m backend.model.evaluate \
  --data-dir data/processed \
  --model-path models/deepfake_detector.keras \
  --metrics-dir models/metrics \
  --real-recall-target 0.94 \
  --max-false-positive-rate 0.08

Tune only the threshold after training:

python -m backend.model.tune_threshold \
  --data-dir data/processed \
  --model-path models/deepfake_detector.keras \
  --metrics-dir models/metrics

Outputs:

• models/metrics/latest_metrics.json
• models/metrics/confusion_matrix.png
• models/metrics/roc_curve.png
• models/metrics/threshold.json
• models/metrics/calibration.json
• models/metrics/accuracy_curve.png
• models/metrics/loss_curve.png

The Flask UI reads models/metrics/latest_metrics.json.

Optional React Frontend

The production-connected Flask UI is in frontend/. The optional React/Vite frontend uses the same backend APIs.

cd react-frontend
npm install
npm run dev

If Flask is not on the same origin, set:

VITE_API_BASE=http://127.0.0.1:5000

Docker

Build and run:

docker compose up --build

Open:

http://127.0.0.1:5000

The compose file mounts:

• ./models
• ./data
• ./backend/uploads
• ./backend/outputs

Production Notes

• Train on licensed, representative datasets before using predictions operationally.
• Set a strong SECRET_KEY.
• Set FLASK_ENV=production and DEMO_MODE=false.
• Store trained checkpoints in models/.
• Put the app behind a reverse proxy with HTTPS.
• Restrict upload size with MAX_CONTENT_LENGTH_MB.
• Monitor false positives and false negatives by dataset, demographic slice, compression level, and video source.

Deployment Guide

Render:

• Create a Web Service from this repository.
• Build command: pip install -r requirements.txt
• Start command: gunicorn -w 2 -b 0.0.0.0:$PORT backend.app:app --timeout 180
• Add persistent disk storage for models/, data/, backend/outputs/, and backend/reports/.

Railway:

• Add the Python service.
• Set environment variables from .env.
• Use start command: gunicorn -w 2 -b 0.0.0.0:$PORT backend.app:app --timeout 180.

AWS:

• Use the Dockerfile with ECS/Fargate or EC2.
• Mount S3/EFS-backed storage for trained models and generated reports.
• Put the service behind HTTPS using an Application Load Balancer.

Vercel:

• Deploy only the optional React frontend to Vercel.
• Keep Flask on Render/Railway/AWS and set VITE_API_BASE to the Flask URL.

Troubleshooting

• Localhost cannot connect: start the server first with RUN_APP.bat.
• Real images marked fake: use the selected threshold from models/metrics/threshold.json, and train with --real-priority 1.35.
• No model found: the app runs in conservative demo mode until models/deepfake_detector.keras exists.
• PDF report fails: run pip install -r requirements.txt so reportlab is installed.
• Webcam blocked: allow camera permission in the browser and use http://127.0.0.1:5000.