Welcome to JEPA Lab, a clean-room educational and experimental laboratory designed for understanding, implementing, and probing Joint Embedding Predictive Architectures (JEPAs)!
Joint Embedding Predictive Architectures, pioneered and advocated by Yann LeCun, represent a major paradigm shift in self-supervised learning (SSL). Unlike generative architectures that predict pixels directly in the input space, JEPAs predict representations in a latent (embedding) space—ignoring chaotic, unpredictable high-frequency noise and focusing entirely on high-level semantic content!
This repository is built for you—the AI enthusiast, the YouTube learner, the curious student, and the seasoned researcher. We have structured this lab to demystify advanced representation learning concepts through concrete, hands-on experiments you can run on standard consumer hardware.
If you are new to JEPAs or want to quickly get up to speed with how self-supervised models learn features, check out our newly launched: 👉 JEPA Starter Guide: Zero to Hero in Latent Representation Learning 👈
The laboratory is laid out to support structured, step-by-step learning:
jepa-lab/
├── README.md # You are here! Main landing page & overview
├── pyproject.toml # Project configuration & dependencies
├── docs/ # Conceptual guides and educational deep-dives
│ ├── 01_jepa_basics.md # Foundations of Joint Embedding Predictive Architectures
│ ├── 02_collapse.md # Representation collapse and how to prevent it
│ ├── 03_sigreg.md # Similarity, Information, Variance, and Covariance regularization
│ ├── 04_masked_prediction.md # Masking strategy, target encoder, and predictor networks
│ ├── 05_world_models.md # World models, actions, and predicting future latent states
│ └── 06_starter_guide.md # 🚀 MUST READ: Zero-to-Hero educational guide!
├── scripts/ # Interactive, high-speed visualization and testing scripts
│ ├── train.py # High-performance AMP training with gradient clipping & checkpoints
│ ├── inspect_data.py # Verify and visualize multi-view data loading
│ ├── inspect_latent.py # Probes for measuring standard deviations and similarity
│ ├── plot_spectrum.py # Compute and plot singular-value spectrum curves
│ ├── evaluate.py # Standard k-NN and Linear Probing evaluators
│ └── evaluate_ood.py # Out-Of-Distribution spatial & lighting robustness evaluator
├── experiments/ # Automated parametersweeping and ablation studies
│ └── run_suite.py # Run weight and dimensionality ablate sweeps
├── src/ # Underlying source code (data loader, ResNet-18 model, SIGReg)
│ ├── data.py # TwoView dataset loader
│ └── model.py # Modified ResNet-18 and Predictor MLP models
Clone the repository and install the developer dependencies:
git clone https://github.com/your-username/jepa-lab.git
cd jepa-lab
pip install -e ".[dev]"See the model learn and stabilize its representations in real-time under extreme regularization:
python3 scripts/train.py --epochs 20 --lr 0.0001 --sigreg_weight 20.0 --gradient_clip_norm 1.0Generate a gorgeous singular-value spectrum plot of your model's representations and test its robustness against spatial and lighting distortions:
# Plot singular-value spectrum
python3 scripts/plot_spectrum.py
# Test OOD robustness
python3 scripts/evaluate_ood.pyWe encourage you to experiment with your own theories and contribute back! Whether you want to ablate new regularizers, implement custom MLP architectures, or optimize data pipelines, we welcome your pull requests!
Let's build the future of latent space world models together! 🚀