Debiasing Sequential Recommenders through Distributionally Robust Optimization over System Exposure

Introduction

Recommender systems have become integral to how we discover content online, from products and videos to articles and music. Sequential recommenders, which predict user preferences based on their historical interaction sequences, are particularly effective. However, these systems suffer from a critical issue known as exposure bias – a self-reinforcing cycle where recommendations are disproportionately influenced by what the system has previously shown to users.

Figure 1: Illustration of the discrepancy between biased preference distribution (learned from biased data) and true preference distribution.

This paper presents a novel approach to mitigating exposure bias in sequential recommender systems through Distributionally Robust Optimization (DRO). Unlike previous methods, this approach leverages system exposure data to create a more accurate model of bias and develop more robust debiasing techniques.

The Exposure Bias Problem

Exposure bias occurs when a recommender system learns from user interaction data that is inherently influenced by what the system has previously exposed to users. This creates a feedback loop: the system recommends items, users interact with some of these items, and then the system learns from these interactions to make future recommendations. Items that were not exposed have no chance of being interacted with, regardless of user preference.

This bias manifests as a discrepancy between:

• The observed preference distribution (what users interact with from what they're shown)
• The true preference distribution (what users would interact with if shown all items)

The problem is particularly challenging because we only observe user feedback on items that were exposed to them, creating incomplete and biased training data.

Limitations of Current Approaches

Current debiasing approaches often rely on Inverse Propensity Scoring (IPS), which attempts to correct for the probability of exposure. However, these methods have several limitations:

1. High variance: IPS-based methods can suffer from instability when propensity scores are small.
2. Underutilization of exposure data: Many methods don't fully leverage available system exposure data.
3. Estimation challenges: It's difficult to accurately estimate the propensity of exposure for all items.

Furthermore, traditional evaluation metrics fail to account for exposure bias in test sets, making it challenging to accurately assess debiasing methods.

The DRO Approach

The proposed approach introduces a framework based on Distributionally Robust Optimization that:

1. Explicitly models the system exposure distribution using a mixture of recommendation models
2. Defines an uncertainty set around this distribution to capture possible variations in true user preferences
3. Optimizes for the worst-case scenario within this uncertainty set

Figure 2: The DRO approach defines an uncertainty set around the nominal distribution derived from system exposure data to capture possible variations in true user preferences.

This approach is more robust to distributional shifts between training and deployment environments compared to existing methods.

Exposure Simulator

A key innovation in this paper is the introduction of an exposure simulator to model the system's exposure distribution. This simulator:

• Is trained on system exposure data from a previous recommender system
• Uses a mixture of three recommendation models:
  • SASRec (a Transformer-based model)
  • GRU4Rec (a recurrent neural network model)
  • A popularity-based model

This mixture approach captures different aspects of the exposure distribution more effectively than using a single model.

The exposure simulator outputs exposure probabilities for items, which serve as the nominal distribution in the DRO framework.

Figure 3: Overview of the DRO-based debiasing approach for sequential recommenders.

Distributionally Robust Optimization Framework

The DRO framework formalizes the uncertainty in user preferences by defining an uncertainty set around the system exposure distribution. The mathematical formulation involves:

1. Nominal Distribution: The system exposure distribution $P_0$ estimated by the exposure simulator.

2. Uncertainty Set: A set of distributions defined using Kullback-Leibler (KL) divergence:

   $\mathcal{U}(P_0, \delta) = \{P : D_{KL}(P||P_0) \leq \delta\}$

   where $\delta$ controls the size of the uncertainty set.

3. DRO Objective: Minimizing the worst-case expected loss:

   $\min_{\theta} \max_{P \in \mathcal{U}(P_0, \delta)} \mathbb{E}_{P}[\mathcal{L}(x; \theta)]$

4. Penalty Term: To mitigate preference overestimation on biased samples, a penalty is introduced for items with high exposure probability:

   $\mathcal{L}_{DRO}(x_u; \theta) = \mathcal{L}_{BCE}(x_u; \theta) + a \cdot \sum_{i \in \mathcal{I}} P_0(i|x_u) \cdot \hat{y}_{ui}$

   where $a$ is a hyperparameter controlling the strength of the penalty.

The final loss function combines the standard binary cross-entropy loss with the DRO penalty term, allowing the model to learn from biased data while correcting for exposure bias.

Experimental Results

The proposed DRO-based approach was evaluated on two real-world datasets:

• ZhihuRec: Question-answering interactions from Zhihu
• Tenrec: Video-watching behavior from Tencent

For evaluation, the paper introduces a Self-Normalized Inverse Propensity Score (SNIPS) evaluator to account for bias in the test sets. The results show:

Figure 4: Coverage comparison between DRO and baselines on ZhihuRec dataset. Higher coverage indicates more diverse recommendations.

Figure 5: Coverage comparison between DRO and baselines on Tenrec dataset.

The experimental results demonstrate that:

1. Superior performance: The DRO-based method outperforms existing debiasing methods (IPS, IPS-C, RelMF) and vanilla backbone models (GRU4Rec, SASRec) in most metrics.

2. Effective exposure simulation: The mixture-based exposure simulator achieves better performance than single-model alternatives in modeling the exposure distribution.

3. Balanced recommendations: The approach improves the trade-off between accuracy and diversity, making recommendations both relevant and diverse.

Impact on Recommendation Diversity

One significant benefit of the proposed approach is its ability to improve recommendation diversity. By mitigating exposure bias, the system is less likely to focus exclusively on popular items or items similar to those a user has already seen.

The experimental results show improved coverage metrics compared to baseline methods, indicating that the DRO approach recommends a wider variety of items to users. This can help address the "filter bubble" problem in recommender systems, where users are only exposed to a narrow range of content.

Hyperparameter Sensitivity

The performance of the DRO approach is sensitive to the hyperparameter $a$, which controls the strength of the penalty for high-exposure items:

Figure 6: Sensitivity of Recall@20 to hyperparameter a on ZhihuRec dataset.

Figure 7: Sensitivity of NDCG@20 to hyperparameter a on ZhihuRec dataset.

Figure 8: Sensitivity of Recall@20 to hyperparameter a on Tenrec dataset.

Figure 9: Sensitivity of NDCG@20 to hyperparameter a on Tenrec dataset.

The analysis shows that:

1. Very small values of $a$ make the approach similar to the vanilla model without debiasing.
2. Very large values of $a$ overly penalize high-exposure items, degrading performance.
3. Moderate values (between 0.1 and 10) typically yield the best results.

This highlights the importance of proper hyperparameter tuning when implementing the DRO approach.

Practical Applications and Significance

The DRO-based debiasing approach has several significant practical implications:

1. Model-agnostic application: The approach can be applied to various sequential recommendation models, making it broadly applicable.

2. Improved user experience: By mitigating exposure bias, the method can provide more diverse and fair recommendations to users.

3. Balanced optimization: The approach balances accuracy and diversity, avoiding the common trade-off where improving one degrades the other.

4. System-level perspective: By incorporating system exposure data, the method takes a more holistic view of the recommendation ecosystem.

5. Robust evaluation: The proposed SNIPS evaluator addresses the challenge of evaluating debiasing methods on biased test sets.

The DRO approach represents a significant advancement in debiasing recommender systems by addressing limitations of existing methods and providing a more robust framework for handling exposure bias. Its ability to improve both recommendation quality and diversity makes it particularly valuable for real-world recommendation systems where both accuracy and fairness are important concerns.

By leveraging system exposure data and applying distributionally robust optimization, this method provides a promising direction for developing more unbiased, diverse, and effective recommender systems.

Relevant Citations

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. Ininternational conference on machine learning. PMLR, 1670–1679.

• This paper introduces the idea of treating recommendations as treatments in a causal inference framework, and proposes inverse propensity scoring (IPS) for debiasing both learning and evaluation of recommender systems.

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback. InWSDM. ACM, 501–509.

• This work proposes an unbiased estimator for learning from missing-not-at-random (MNAR) implicit feedback. It particularly focuses on addressing selection bias and position bias.

Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018. Unbiased offline recommender evaluation for missing-not-at-random implicit feedback. InProceedings of the 12th ACM conference on recommender systems. 279–287.

• This paper addresses the issue of unbiased offline evaluation for recommender systems when dealing with MNAR implicit feedback. They utilize self-normalized IPS (SNIPS) to estimate performance metrics and propose a method to reduce the variance of SNIPS.

Adith Swaminathan and Thorsten Joachims. 2015. The self-normalized estimator for counterfactual learning.NIPS28 (2015).

• This work introduces the self-normalized Inverse Propensity Scoring (SNIPS) estimator, which offers improved robustness and reduced variance compared to traditional IPS, particularly in situations with large or infinite importance weights.