DASH ABR Client with Reinforcement Learning

A Dynamic Adaptive Streaming over HTTP (DASH) client implementing Proximal Policy Optimization (PPO) for intelligent bitrate adaptation to maximize Quality of Experience (QoE).

Overview

This project implements an adaptive bitrate (ABR) algorithm using reinforcement learning to optimize video streaming quality. The system dynamically selects optimal bitrates based on network conditions, buffer levels, and user experience metrics while being tested in realistic network environments using Mahimahi network simulator.

Architecture

The project consists of several key components:

• Reinforcement Learning Agent: PPO-based algorithm for bitrate selection
• DASH Client: HTTP-based streaming client with chunk-based video delivery
• Network Simulation: Mahimahi integration for realistic network testing
• QoE Optimization: Comprehensive reward system balancing quality, rebuffering, and smoothness

Project Structure

├── mahimahi.up              # Mahimahi uplink network trace
├── mahimahi.down            # Mahimahi downlink network trace
├── flask_server             # Video streaming server implementation
├── video_size               # Video chunk size configurations
├── streaming_environment.py # RL environment for DASH streaming
├── network_architecture.py  # Neural network model definitions
├── model_evaluation.py      # Model performance evaluation scripts
├── experience_buffer.py     # Experience replay buffer for RL
├── adaptive_bitrate_trainer.py # PPO training implementation
├── trace_loader.py          # Network trace loading utilities
├── test_environment.py      # Testing and validation scripts
├── traces/                  # Directory containing network traces
└── test_traces/            # Additional test network traces

Features

Reinforcement Learning

• PPO Algorithm: Stable policy optimization for bitrate selection
• State Space: Buffer levels, bandwidth estimation, previous bitrates
• Action Space: Multiple bitrate options (300kbps, 600kbps, 1200kbps, etc.)
• Reward Function: Balances video quality, rebuffering, and bitrate switching

DASH Streaming

• Chunk-based Delivery: Efficient video segment streaming
• Multiple Bitrates: Adaptive quality selection
• Buffer Management: Intelligent prebuffering strategies
• Real-time Adaptation: Dynamic quality adjustment

Network Simulation

• Mahimahi Integration: Realistic network condition simulation
• Variable Bandwidth: Testing under different network scenarios
• Latency Simulation: RTT and packet loss modeling
• Trace Replay: Real-world network trace analysis

Installation

Prerequisites

• Python 3.8+
• PyTorch
• OpenAI Gym
• Flask
• Mahimahi Network Simulator

Setup

# Clone the repository
git clone <repository-url>
cd dash-abr-client

# Install Python dependencies
pip install torch torchvision
pip install gym flask numpy matplotlib
pip install opencv-python requests

# Install Mahimahi (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install mahimahi

# For other systems, visit: http://mahimahi.mit.edu/

Usage

Training the Model

# Start training with default parameters
python adaptive_bitrate_trainer.py

# Train with custom network traces
python adaptive_bitrate_trainer.py --traces ./traces/ --episodes 1000

Running the DASH Server

# Start the video server
python flask_server

# Server will be available at http://localhost:5000

Testing with Mahimahi

# Run client with network simulation
mm-delay 20 mm-link mahimahi.up mahimahi.down python test_environment.py

# Test with custom traces
mm-delay 50 mm-link traces/trace1.up traces/trace1.down python streaming_environment.py

Evaluation

# Evaluate trained model performance
python model_evaluation.py --model ./models/ppo_model.pth

# Generate performance plots and metrics
python model_evaluation.py --plot --output ./results/

Performance Metrics

The system optimizes for:

• Video Quality: Average bitrate delivered
• Rebuffering: Minimize stalling events
• Smoothness: Reduce bitrate switching frequency
• QoE Score: Composite user experience metric

Experimental Setup

State Space

• Current buffer level (seconds)
• Estimated bandwidth (kbps)
• Previous chunk bitrate
• Chunk download time
• Remaining video duration

Network Traces

• Real-world network traces from various conditions
• Synthetic traces for controlled experiments
• Variable bandwidth scenarios (3G, 4G, WiFi)

Training Parameters

# adaptive_bitrate_trainer.py
LEARNING_RATE = 3e-4
BATCH_SIZE = 64
EPOCHS = 1000
CLIP_RATIO = 0.2
GAMMA = 0.99

Network Settings

# streaming_environment.py
BITRATES = [300, 600, 1200, 2400]  # kbps
CHUNK_DURATION = 4  # seconds
BUFFER_TARGET = 10  # seconds

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request