AI Audio Detector

This project focuses on detecting AI-generated speech using a deep learning pipeline built on Wav2Vec2 embeddings and spectrogram-based features. The goal is to accurately distinguish between real human speech and synthetic audio generated by modern TTS or voice cloning models.

While this version is audio-only, the pipeline may later extend to multimodal detection using lip-sync or video features (e.g., from VoxCeleb).

Project Objectives

• Detect whether speech audio is real (human) or fake (AI-generated)
• Use Wav2Vec2 to extract high-dimensional audio embeddings
• Train binary classifiers using:
  • A baseline MLP
  • A deeper dropout-regularized MLP
  • A Mel spectrogram-based model
• Analyze performance on 2-second clips for real-time suitability
• (Optional/future) Explore multimodal features like video or lip-sync

Dataset: Fake-or-Real

We used a curated version of the publicly available Fake-or-Real dataset, which contains:

• Real human voice recordings (e.g., speeches, narrations)
• AI-generated audio from various TTS and cloning models
• Diverse speakers and accents

Preprocessing steps included:

• Segmenting all audio into uniform 2-second clips
• Extracting Wav2Vec2 embeddings from each segment
• Splitting data into training, validation, and test sets
• Organizing samples by folders: fake_audio/REAL/ and fake_audio/FAKE/

Project Structure

ai-audio-detector/ ├── data/ # Raw audio (not pushed), optional .zip for reuse ├── models/ # Trained .pth model weights ├── notebooks/ # Jupyter notebooks for training + experiments ├── scripts/ # (Optional) Automation/processing scripts ├── training/ # (Planned) PyTorch training scripts ├── utils/ # (Planned) Helper functions, loaders ├── .gitignore ├── README.md └── requirements.txt

Technologies

• Hugging Face Transformers — for Wav2Vec2 embeddings
• PyTorch + Torchaudio — for deep learning and audio I/O
• Librosa / Matplotlib — for spectrograms and visualization
• (Optional future) OpenCV / Dlib / VoxCeleb — for video/audio fusion

Setup

git clone https://github.com/sharlobo/ai-audio-detector.git
cd ai-audio-detector

python -m venv venv
source venv/bin/activate

pip install -r requirements.txt

Getting Started

1. Add your real and AI-generated .wav files into data/raw/
2. Run scripts/extract_audio_feats.py to extract embeddings
3. Use training/train.py to train the classifier
4. Evaluate with training/eval.py

Future Work

• Multiclass detection of specific synthesis models
• Lip-sync mismatch detection
• Cross-lingual AI audio detection
• Robustness testing (noise, pitch, reverb)

Authors

Created by @sharlobo @carlierherrera @talasoolyn
University project for Deep Learning and Multimodal Systems

License

MIT License EOF

This project is for educational and research use only. All datasets belong to their respective owners.