A study in learning from class-imbalanced data, using the classic Kaggle credit-card fraud dataset (284,807 transactions, 492 frauds — a 0.17% fraud rate, ~1:577 imbalance). The notebook walks through EDA, imbalance-handling strategies (undersampling, SMOTE, and a from-scratch Borderline-SMOTE), and compares Logistic Regression, SVM, and Decision Tree models.
The data is not fully committed here (it's ~150 MB). Download
creditcard.csv from Kaggle and place it in dataset/:
dataset/creditcard.csv
Features V1–V28 are anonymized PCA components; Time and Amount are raw
and get standardized in the notebook. Class is the label (1 = fraud).
pip install -r requirements.txt
jupyter notebook creditcard.ipynb # or: jupyter labTo re-run the whole pipeline non-interactively:
jupyter nbconvert --to notebook --execute --inplace creditcard.ipynbIt executes top-to-bottom in well under a minute on a laptop.
- Data overview — class distribution and the scale of the imbalance.
- Hold out a realistic test set first — a stratified 30% split is taken from the full, imbalanced data before any preprocessing or sampling. The scaler is fit on the training portion only, and every sampling method is applied to training data only. This avoids data leakage and the inflated scores you get from evaluating on a balanced sample.
- EDA on a balanced training subset — correlation heatmaps, boxplots, feature distributions, fraud-only IQR outlier removal, and SVD/PCA/t-SNE projections (a balanced subset is appropriate for visualization).
- Models — Logistic Regression (CV over
C), SVM (grid search), and Decision Tree (grid search), all tuned for recall. - Honest evaluation — every model is scored on the same untouched imbalanced test set, reporting Precision / Recall / F1 / ROC-AUC / PR-AUC plus ROC and Precision–Recall curves (PR is more informative under heavy imbalance).
- Sampling comparison — Undersampling vs SMOTE vs the custom Borderline-SMOTE, oversampling the full training set.
| Model | ROC-AUC | PR-AUC | Recall | Precision | F1 |
|---|---|---|---|---|---|
| Logistic Regression (undersample) | 0.974 | 0.671 | 0.878 | 0.037 | 0.071 |
| SVM (undersample) | 0.953 | 0.033 | 0.912 | 0.017 | 0.033 |
| Decision Tree (undersample) | 0.918 | 0.078 | 0.831 | 0.040 | 0.077 |
| Logistic Regression (SMOTE) | 0.967 | 0.700 | 0.878 | 0.065 | 0.121 |
| Logistic Regression (custom Borderline-SMOTE) | 0.942 | 0.638 | 0.838 | 0.142 | 0.243 |
Takeaways
- Evaluated honestly on the real distribution, precision is low — high recall comes at the cost of many false positives. This is the central difficulty of fraud detection, and it's invisible if you test on a balanced sample (where the same LR model shows ~0.9 "precision").
- PR-AUC separates the models far more meaningfully than ROC-AUC, which looks uniformly high.
- Among sampling strategies, Borderline-SMOTE gives the best precision/F1 trade-off here.
CustomBorderlineSMOTE is implemented from scratch (danger-zone detection via
m-neighbors, borderline-1/-2 variants, linear interpolation), and is
vectorized so it balances the full ~200k-row training set in ~2 seconds. It
is deliberately named to avoid being shadowed by
imblearn.over_sampling.BorderlineSMOTE.
creditcard.ipynb # the analysis — committed WITH outputs, so it renders on GitHub
requirements.txt # pinned dependencies (see header for tested versions)
.gitignore # excludes the data files and notebook scratch
dataset/ # creditcard.csv lives here locally (gitignored — download from Kaggle)
paper/ # reference papers on imbalanced learning
The dataset is not tracked in git (it's ~150 MB) — clone the repo, then
download creditcard.csv into dataset/ as described above. The notebook is
committed with its rendered figures and metrics, so you can read the full
analysis on GitHub without running anything.
See paper/ for the imbalanced-learning literature this project draws on,
including the original Borderline-SMOTE paper.