The goal of this repository is to develop a recommender system for Deezer and improve their collaborative filtering pipeline. This project was part of a Kaggle challenge, but the focus here is not just on improving metrics—it's about building a flexible, scalable, and interpretable recommender system.
-
Kaggle Dataset: DSG17 Online Phase
-
Deezer API for metadata: Deezer for Developers
Recommendation: Use Google Colab for GPU access when working with large datasets
- Data Preprocessing : 📓Implicit_BPR_Model.ipynb
-
Preprocess the Kaggle dataset.
-
Enhance the data using Deezer API metadata.
- Modeling Methods
-
Bayesian Personalized Ranking (BPR) implemented using the implicit package.📓Implicit_BPR_Model.ipynb
-
LightFM with metadata and BERTopic reranking.📓BERTopic_LightFM.ipynb
-
LightFM with metadata.📓BERTopic_LightFM.ipynb
| Model | Precision@10 | AUC | Notes |
|---|---|---|---|
| BPR (Implicit) | 0.177 | 0.540 | Predicts above chance but not ideal for large-scale or cold-start scenarios. |
| LightFM with metadata (Baseline) | 0.53578 | 0.9585 | Strong LightFM performance with metadata enrichment, recommended for production. |
| BERTopic Reranking (LightFM) | 0.0177 | 0.8835 | Reranking did not improve performance. |
Metrics were calculated on a sample subset, as the full Deezer metadata contains ~7 million rows
The scripts are made as tutorials, so if you want to learn more about the modeling approaches you can follow them and interpret them for your own data.
1- Create a virtual environment with Python.
2- Download the Kaggle dataset.
3- Place the data in a folder within the virtual environment.
4- Install dependencies: pip install -r requirements.txt
5- Run the scripts:
-
Start with the BPR script (includes metadata enrichment).
-
Then run the LightFM script.
1- Open a Colab notebook and connect Google Drive.
2- Download the Kaggle dataset and upload it to Google Drive.
3- Connect Google Drive to Colab.
4- Install dependencies: !pip install -r requirements.txt
5- Upload scripts to Colab:
-
Run the BPR script first (save metadata to Google Drive for faster reuse).
-
Run the LightFM script next.