Machine Learning Labs — Télécom Paris

Practical work (travaux pratiques) from the machine learning courses at Télécom Paris, covering the path from classical statistical classifiers to the large-scale, high-cardinality setting of online advertising.

Every notebook is self-contained: it loads (or downloads) its own data, implements the models, and answers the questions of the lab statement inline.

Contents

Lab	Topic	Data	Highlights
01 — Logistic regression & LDA	Generative vs discriminative binary classification	Synthetic Gaussian mixtures, two moons	Bayes rule in closed form, LDA from scratch, polynomial features, ROC/AUC, gradient descent from scratch
02 — k-Nearest Neighbors	Instance-based learning and hyperparameter tuning	Breast Cancer Wisconsin	kNN from scratch in NumPy, train/inference cost vs sample size, bias–variance through k
03 — Principal Component Analysis	Dimensionality reduction and robustness	Simulated multivariate Student-t	Student-t sampler, low-rank embedding, PCA via SVD, breakdown under Cauchy contamination, MCD robust estimator
04 — Avazu CTR prediction	Click-through-rate prediction at scale	Avazu, 1M ad impressions	One-hot encoding of high-cardinality features, GBDT leaf features, hashing trick, XGBoost, FFM, blending

Running the notebooks

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
jupyter lab

Labs 01, 02 and 03 use synthetic or bundled data, run in a few seconds and are committed with their outputs. Lab 04 downloads a 1M-row dataset and fits several ensembles, so its outputs are stripped — see its README for the reference results and the extra dependencies.

Repository layout

01-logistic-regression-lda/     logistic_regression_lda.ipynb + README
02-knn-hyperparameter-tuning/   knn_from_scratch.ipynb       + README
03-pca/                         pca_robust_analysis.ipynb    + lab statement (PDF)
04-avazu-ctr-prediction/        avazu_ctr_prediction.ipynb   + README

The first commit of this repository contains the notebooks exactly as they were submitted; the following commits fix the bugs and unfinished cells found while reviewing them. git log -p shows what changed.

Credits

Lab statements by the teaching staff of Télécom Paris (Igor Colin, Pavlo Mozharovskyi, Pascal Bianchi et al.). Solutions and write-ups are mine.