AutoDebias: Learning to Debias for Recommendation (2105.04170v5)

Published 10 May 2021 in cs.LG and cs.IR

Abstract: Recommender systems rely on user behavior data like ratings and clicks to build personalization model. However, the collected data is observational rather than experimental, causing various biases in the data which significantly affect the learned model. Most existing work for recommendation debiasing, such as the inverse propensity scoring and imputation approaches, focuses on one or two specific biases, lacking the universal capacity that can account for mixed or even unknown biases in the data. Towards this research gap, we first analyze the origin of biases from the perspective of \textit{risk discrepancy} that represents the difference between the expectation empirical risk and the true risk. Remarkably, we derive a general learning framework that well summarizes most existing debiasing strategies by specifying some parameters of the general framework. This provides a valuable opportunity to develop a universal solution for debiasing, e.g., by learning the debiasing parameters from data. However, the training data lacks important signal of how the data is biased and what the unbiased data looks like. To move this idea forward, we propose \textit{AotoDebias} that leverages another (small) set of uniform data to optimize the debiasing parameters by solving the bi-level optimization problem with meta-learning. Through theoretical analyses, we derive the generalization bound for AutoDebias and prove its ability to acquire the appropriate debiasing strategy. Extensive experiments on two real datasets and a simulated dataset demonstrated effectiveness of AutoDebias. The code is available at \url{https://github.com/DongHande/AutoDebias}.

Authors (8)

Jiawei Chen (161 papers)
Hande Dong (9 papers)
Yang Qiu (34 papers)
Xiangnan He (200 papers)
Xin Xin (49 papers)
Liang Chen (360 papers)
Guli Lin (4 papers)
Keping Yang (14 papers)

Citations (172)

View on Semantic Scholar

Summary

AutoDebias: Learning to Debias for Recommendation

This paper introduces AutoDebias, a novel method for addressing biases in recommender systems through the use of meta-learning. The authors tackle the persistent issue of various biases that arise due to the observational nature of data collection in most recommender systems. These biases, such as selection bias, conformity bias, exposure bias, and position bias, can significantly degrade model performance by skewing the data distribution used for training relative to the one used in unbiased testing scenarios.

The cornerstone of their approach is a general debiasing framework that views these biases through the lens of risk discrepancy—the difference between empirical risk based on biased training data and true risk. The framework is characterized by a parameterized empirical risk function designed to mitigate the skewness of the data and thus close the gap between empirical and true risk. Specifically, the authors propose decomposing the debiasing task into learning three sets of parameters: weights for observed data, weights for imputing missing data, and imputed values themselves.

To optimize this framework, the authors utilize meta-learning, leveraging a small subset of unbiased data to guide the learning of parameters that best mitigate the biases present in the larger, biased training set. This bi-level optimization problem is crafted to ensure that the debiasing parameters are informed by the unbiased data, essentially acting as hyper-parameters for the base recommender model. They use a meta-learning strategy, which updates these hyper-parameters through feedback from the unbiased data subset.

The paper demonstrates the efficacy of AutoDebias through empirical evaluations on various datasets—that cover explicit feedback, implicit feedback, and simulated recommendation lists—showing robust performance across settings affected by different types of biases. For example, on the Yahoo!R3 dataset, AutoDebias yielded a 5.6% improvement in Negative Log Likelihood and an 11.2% improvement in NDCG@5 compared to other debiasing methods. Such results illustrate its superiority over existing methods like Inverse Propensity Scoring, Doubly Robust estimation, and knowledge-distillation-based techniques.

Furthermore, the flexibility of AutoDebias allows it to adapt to a broad range of bias scenarios. The framework is versatile enough to incorporate new kinds of biases and update its debiasing strategy accordingly without manual intervention. This adaptability is particularly crucial in dynamic environments where the data distribution and biases can evolve over time.

The theoretical contributions of the paper include a proof that AutoDebias can achieve approximately optimal generalization error bounds, even in the face of inductive biases introduced by a restricted meta-model hypothesis space. This ensures that even with a constrained model, the system can still benefit from the debiasing strategy employed, thus providing a degree of robustness against the limitations of meta-model capacity.

The implications of this work are twofold: practically, it provides a scalable and adaptable solution for real-world recommendation systems where multiple and changing biases are the norm; theoretically, it enriches the understanding of how meta-learning can be harnessed to automatically deduce optimal configurations for bias mitigation in machine learning models. Future work may explore extending the meta model's capacity to capture more complex patterns and addressing the challenge of dynamic biases in real-time recommendation systems.

PDF Markdown

Related Papers

Find Related Papers

GitHub

GitHub - DongHande/AutoDebias: This is the official pytorch implementation of AutoDebias, an automatic debiasing method for recommendation. (103 stars)