For the optimal discriminator:
If:
then:
The GAN minimax game at equilibrium becomes:
Where the Jensen-Shannon divergence is defined as:
Gradient with respect to the discriminator parameters:
Gradient with respect to the generator parameters:
This repository implements the Deep Convolutional Generative Adversarial Network (DCGAN) architecture proposed in the paper:
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
Paper: https://arxiv.org/pdf/1511.06434
The project focuses on implementing the complete GAN training pipeline entirely from scratch using PyTorch while following the architectural principles introduced in the original DCGAN paper.
I also documented the complete implementation journey, debugging process, architecture decisions, and GAN training insights here:
Building DCGAN From Scratch While Reading the Paper
Initially, the work started with a fully connected MLP-GAN implementation to understand adversarial optimization dynamics and generator-discriminator interaction at a lower abstraction level. The implementation was later extended toward convolutional adversarial architectures using DCGAN.
The repository contains:
- End-to-end PyTorch implementation of DCGAN
- Modular Generator and Discriminator blocks
- Stable adversarial training pipeline
- CelebA dataset training support
- Convolutional transpose based image synthesis
- Batch normalization based stabilization
- Experiment-ready training configuration
- Reproducible architecture definitions
You can directly run the training script and the model will use predefined configuration settings.
The primary objective of this implementation is to study:
- Adversarial optimization dynamics
- Deep convolutional latent representations
- Stability of GAN training
- Generator-discriminator equilibrium behavior
- Convolutional feature hierarchy emergence
- High-dimensional image manifold approximation
The implementation is intentionally modular to make experimentation with:
- Loss functions
- Architectural depth
- Latent dimensions
- Feature map scaling
- Training heuristics
- Regularization techniques
more convenient for future research extensions.
Training was performed on the CelebA dataset.
Dataset characteristics:
| Property | Value |
|---|---|
| Dataset | CelebA |
| Image Type | Human Faces |
| Channels | RGB |
| Resolution | 64×64 |
| Preprocessing | Resize + Normalize |
| Normalization Range | [-1, 1] |
The generator output layer uses Tanh() activation to align generated image distribution with normalized dataset statistics.
DCGAN trained for 180 epochs on CelebA:
The generated samples demonstrate:
- Coherent facial structure formation
- Semantic consistency in latent generations
- Stable texture synthesis
- Emergence of facial symmetry
- Diverse identity generation
- Structured feature composition
Artifacts are still visible in some generations, which is expected under standard adversarial optimization without additional stabilization techniques such as:
- Spectral normalization
- Wasserstein objectives
- Gradient penalty
- Progressive growing
- Self-attention mechanisms
The optimization paths of the non-cooperative game exhibit typical adversarial oscillation dynamics before reaching a relative equilibrium:
| Generator Loss | Discriminator Loss |
|---|---|
 |
 |
Tracking the prediction outputs
| Real Scores |
Fake Scores |
|---|---|
 |
 |
The generator progressively upsamples a latent vector sampled from a Gaussian prior into a structured RGB image using transposed convolutions.
Generator(
(generator): Sequential(
(0): GeneratorBlock(
(block): Sequential(
(0): ConvTranspose2d(128, 512, kernel_size=4, stride=1, bias=False)
(1): BatchNorm2d(512)
(2): ReLU(inplace=True)
)
)
(1): GeneratorBlock(
(block): Sequential(
(0): ConvTranspose2d(512, 256, kernel_size=4, stride=2, padding=1, bias=False)
(1): BatchNorm2d(256)
(2): ReLU(inplace=True)
)
)
(2): GeneratorBlock(
(block): Sequential(
(0): ConvTranspose2d(256, 128, kernel_size=4, stride=2, padding=1, bias=False)
(1): BatchNorm2d(128)
(2): ReLU(inplace=True)
)
)
(3): GeneratorBlock(
(block): Sequential(
(0): ConvTranspose2d(128, 64, kernel_size=4, stride=2, padding=1, bias=False)
(1): BatchNorm2d(64)
(2): ReLU(inplace=True)
)
)
(4): Sequential(
(0): ConvTranspose2d(64, 3, kernel_size=4, stride=2, padding=1, bias=False)
(1): Tanh()
)
)
)The discriminator maps an RGB image into a scalar probability indicating whether the image belongs to the true data distribution.
Discriminator(
(discriminator): Sequential(
(0): Conv2d(
3, 64,
kernel_size=4,
stride=2,
padding=1,
bias=False
)
(1): LeakyReLU(0.2)
(2): DiscriminatorBlock(
(block): Sequential(
(0): Conv2d(
64, 128,
kernel_size=4,
stride=2,
padding=1,
bias=False
)
(1): BatchNorm2d(128)
(2): LeakyReLU(0.2, inplace=True)
)
)
(3): DiscriminatorBlock(
(block): Sequential(
(0): Conv2d(
128, 256,
kernel_size=4,
stride=2,
padding=1,
bias=False
)
(1): BatchNorm2d(256)
(2): LeakyReLU(0.2, inplace=True)
)
)
(4): DiscriminatorBlock(
(block): Sequential(
(0): Conv2d(
256, 512,
kernel_size=4,
stride=2,
padding=1,
bias=False
)
(1): BatchNorm2d(512)
(2): LeakyReLU(0.2, inplace=True)
)
)
(5): Sequential(
(0): Conv2d(
512, 1,
kernel_size=4,
stride=1,
bias=False
)
)
)
)The implementation follows the original DCGAN paper recommendations:
| Technique | Purpose |
|---|---|
| Strided Convolutions | Learnable downsampling |
| Transposed Convolutions | Learnable upsampling |
| Batch Normalization | Stabilized gradient flow |
| ReLU in Generator | Strong gradient propagation |
| LeakyReLU in Discriminator | Prevent dead activations |
| No Fully Connected Layers | Spatial hierarchy preservation |
| Tanh Output | Distribution alignment with normalized images |
The training process alternates between:
The discriminator maximizes:
where:
- Real images should maximize discriminator confidence
- Fake images should minimize discriminator confidence
The generator attempts to fool the discriminator by minimizing:
which implicitly minimizes the divergence between:
At equilibrium:
and the discriminator becomes incapable of distinguishing real and generated samples.
| Hyperparameter | Value |
|---|---|
| Latent Dimension | 128 |
| Image Resolution | 64×64 |
| Optimizer | Adam |
| Epochs | 180 |
| Dataset | CelebA |
| Activation (Generator) | ReLU |
| Activation (Discriminator) | LeakyReLU |
| Output Activation | Tanh |
.
├── data/ # Dataset directory (CelebA images)
├── output/ # Generated sample outputs during training
├── saved_models/ # Saved generator/discriminator checkpoints
│
├── src/
│ ├── blocks.py # Reusable neural network blocks
│ ├── config.py # Training and model configuration
│ ├── discriminator.py # Discriminator architecture
│ ├── generator.py # Generator architecture
│ ├── model.py # Combined DCGAN model utilities
│ ├── sample.py # Generate samples using trained model
│ ├── train.py # Main training pipeline
│ ├── trained_config.py # Configuration for inference/sampling
│ └── utils.py # Helper and utility functions
│
├── .gitignore # Ignored files/folders
└── README.md # Project documentation
Clone the repository:
git clone https://github.com/Himanshu7921/DCGAN-PyTorch-Implementation-From-Scratch.git
cd DCGAN-PyTorch-Implementation-From-ScratchInstall dependencies:
pip install -r requirements.txtRun training:
python train.pyThe training script automatically loads the predefined configuration and starts adversarial training.
-
Ian Goodfellow et al. Generative Adversarial Nets https://arxiv.org/abs/1406.2661
-
Alec Radford et al. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks https://arxiv.org/abs/1511.06434
Himanshu Singh