The main goal of reading paper is not just understanding it. Try to understand the key concept, but we need to get new ideas and research directions from the paper.
Paper information
- title: Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
- authors: Zhou, Chang, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang.
- venue: arXiv preprint arXiv:2005.12964 (2020).
- pdf link: pdf link
- github: Not released yet
- abstract:
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning is essential to large-scale recommender systems [11]. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe unfairness of exposure with a vocabulary several orders of magnitude larger than that of natural language, implying that (1) MLE will preserve and even exacerbate the exposure bias in the long run in order to faithfully fit the observed samples, and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the unfairly ignored items. In this paper, we introduce CLRec, a Contrastive Learning paradigm that has been successfully deployed in a real-world massive RECommender system, to alleviate exposure bias in DCG. We theoretically prove that a popular choice of contrastive loss is equivalently reducing the exposure bias via inverse propensity scoring, which provides a new perspective on the effectiveness of contrastive learning. We further employ a fixed-size queue to store the items’ representations computed in previously processed batches, and use the queue to serve as an effective sampler of negative examples. This queue-based design provides great efficiency in incorporating rich features of the thousand negative items per batch thanks to computation reuse. Extensive offline analyses and four-month online A/B tests demonstrate substantial improvement, including a dramatic reduction in the Matthew effect.
Summary: problems to address, key ideas, quick results
presentation link
Questions about the paper?
What do you like?
What you don't like?
How to improve?
Any new ideas?
Reproducing results (if any)
The main goal of reading paper is not just understanding it. Try to understand the key concept, but we need to get new ideas and research directions from the paper.
Paper information
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning is essential to large-scale recommender systems [11]. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe unfairness of exposure with a vocabulary several orders of magnitude larger than that of natural language, implying that (1) MLE will preserve and even exacerbate the exposure bias in the long run in order to faithfully fit the observed samples, and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the unfairly ignored items. In this paper, we introduce CLRec, a Contrastive Learning paradigm that has been successfully deployed in a real-world massive RECommender system, to alleviate exposure bias in DCG. We theoretically prove that a popular choice of contrastive loss is equivalently reducing the exposure bias via inverse propensity scoring, which provides a new perspective on the effectiveness of contrastive learning. We further employ a fixed-size queue to store the items’ representations computed in previously processed batches, and use the queue to serve as an effective sampler of negative examples. This queue-based design provides great efficiency in incorporating rich features of the thousand negative items per batch thanks to computation reuse. Extensive offline analyses and four-month online A/B tests demonstrate substantial improvement, including a dramatic reduction in the Matthew effect.
Summary: problems to address, key ideas, quick results
presentation link
Questions about the paper?
What do you like?
What you don't like?
How to improve?
Any new ideas?
Reproducing results (if any)