Mitigating Selection Bias with Node Pruning and Auxiliary Options (2409.18857v1)

Abstract: LLMs often show unwarranted preference for certain choice options when responding to multiple-choice questions, posing significant reliability concerns in LLM-automated systems. To mitigate this selection bias problem, previous solutions utilized debiasing methods to adjust the model's input and/or output. Our work, in contrast, investigates the model's internal representation of the selection bias. Specifically, we introduce a novel debiasing approach, Bias Node Pruning (BNP), which eliminates the linear layer parameters that contribute to the bias. Furthermore, we present Auxiliary Option Injection (AOI), a simple yet effective input modification technique for debiasing, which is compatible even with black-box LLMs. To provide a more systematic evaluation of selection bias, we review existing metrics and introduce Choice Kullback-Leibler Divergence (CKLD), which addresses the insensitivity of the commonly used metrics to label imbalance. Experiments show that our methods are robust and adaptable across various datasets when applied to three LLMs.

Mitigating Selection Bias in LLMs using Node Pruning and Auxiliary Options

The paper "Mitigating Selection Bias with Node Pruning and Auxiliary Options" addresses a critical problem in LLMs: selection bias. Selection bias, the model's unwarranted preference for certain options in multiple-choice questions (MCQs), undermines the reliability of LLMs in tasks requiring unbiased decision-making. This research introduces two novel debiasing techniques—Bias Node Pruning (BNP) and Auxiliary Option Injection (AOI)—to effectively mitigate selection bias and improve MCQ performance.

Main Contributions

Bias Node Pruning (BNP):

BNP identifies and eliminates specific nodes in the LLM's final linear layer contributing to selection bias. This method, driven by its unique approach to scrutinize embedding-level discrepancies, reduces bias without extensively modifying the model's structure. By doing so, BNP leverages the intrinsic properties of the model's parameters, enhancing model reliability and performance.

Auxiliary Option Injection (AOI):

AOI introduces an auxiliary "I don't know" option in the MCQ input, aiming to reduce bias induced by uncertainty. This technique is simple, yet effective, and can be applied to black-box models as well. It ensures the LLM provides responses that better reflect the inherent uncertainty in situations where the model might otherwise exhibit biased preferences.

Evaluation Metrics:

The research critiques existing metrics like Standard Deviation of Recalls (RStd) and Relative Standard Deviation (RSD) for their insensitivity to label imbalance. It introduces Choice Kullback-Leibler Divergence (CKLD) as a novel metric sensitive to label distribution imbalance, providing a more rigorous assessment of selection bias.

Experimental Results

The proposed methods were evaluated on various datasets including ARC-Challenge, MMLU-Redux, and CommonsenseQA using different LLMs like Llama-3, Mistral, and Bloomz. The results demonstrated significant improvements in both bias mitigation and performance metrics:

• Llama-3 Performance:
  • BNP and AOI improved accuracy on ARC-Challenge from 52.3% to 65.3%, marking a substantial enhancement of 24.9%.
  • CKLD decreased from 0.494 to 0.124, indicating reduced selection bias.
• Bloomz and Mistral:
  • Both models exhibited performance improvements when BNP and AOI were applied, with AOI showing particularly strong adaptability across black-box scenarios.
  • CKLD values improved significantly in almost all scenarios, reaffirming the effectiveness of the proposed methods.

The experiments further validated the robustness of BNP and AOI through various ablation studies, underscoring their applicability across different model architectures and dataset distributions.

Practical and Theoretical Implications

From a practical standpoint, reducing selection bias enhances the reliability of LLM applications in automated systems, including data annotation and decision-making tasks. The proposed methods not only improve accuracy but also ensure that the model's predictions are more consistent and unbiased.

Theoretically, this research provides new insights into the internal mechanisms leading to selection bias, shifting the focus from superficial bias correction to addressing root causes at the parameter level. This paradigm shift opens new avenues for future research to explore internal model behaviors and their impacts on output biases.

Future Directions

This work paves the way for further research into the internal sources of biases in LLMs. Future studies could investigate the underlying factors contributing to selection bias across different model families and data types. Additionally, the methodologies introduced could be extended to explore other types of biases, providing a holistic approach to improving LLM reliability.

In conclusion, the introduction of Bias Node Pruning and Auxiliary Option Injection represents a significant step towards mitigating selection bias in LLMs. By addressing the problem at the parameter level and incorporating auxiliary options to manage uncertainty, this research offers practical and theoretical advancements essential for the development of unbiased, reliable AI systems.