Why Don't Prompt-Based Fairness Metrics Correlate? (2406.05918v1)

Abstract: The widespread use of LLMs has brought up essential questions about the potential biases these models might learn. This led to the development of several metrics aimed at evaluating and mitigating these biases. In this paper, we first demonstrate that prompt-based fairness metrics exhibit poor agreement, as measured by correlation, raising important questions about the reliability of fairness assessment using prompts. Then, we outline six relevant reasons why such a low correlation is observed across existing metrics. Based on these insights, we propose a method called Correlated Fairness Output (CAIRO) to enhance the correlation between fairness metrics. CAIRO augments the original prompts of a given fairness metric by using several pre-trained LLMs and then selects the combination of the augmented prompts that achieves the highest correlation across metrics. We show a significant improvement in Pearson correlation from 0.3 and 0.18 to 0.90 and 0.98 across metrics for gender and religion biases, respectively. Our code is available at https://github.com/chandar-lab/CAIRO.

• The paper introduces CAIRO, a novel method that boosts fairness metric correlation from as low as 0.3 to as high as 0.9.
• The study systematically analyzes how factors like prompt structure, verbalization, and data distribution affect bias measurements.
• The approach offers practical insights for mitigating biases in LLMs, enhancing the consistency and reliability of bias evaluations.

Analysis of Prompt-Based Fairness Metrics and Correlation Enhancement

Recent developments in LLMs have highlighted the importance of understanding and mitigating biases inherent in these models. The paper "Why Don’t Prompt-Based Fairness Metrics Correlate?" explores the inconsistencies observed across various prompt-based fairness metrics, raising questions about the reliability of these metrics when used to assess social biases in LLMs.

Overview of the Paper

The authors present a systematic examination of prompt-based fairness metrics, particularly focusing on their poor correlation. They introduce a novel method called Correlated Fairness Output (CAIRO) to improve metric correlation and thus the reliability of bias assessments. This paper involves evaluating fairness dimensions such as gender and religion, using metrics including BOLD, HolisticBias, and HONEST across multiple LLMs.

Factors Contributing to Poor Correlation

The paper identifies six key factors contributing to the lack of correlation across fairness metrics:

1. Prompt Sentence Structure: Variations in grammatical structure can lead to differing outputs, affecting the consistency of bias measurement.
2. Prompt Verbalization: The specific wording of prompts can influence model responses, leading to variability in assessed bias.
3. Prompt Distribution: The source and nature of the data used to generate prompts can affect the overlap with a model's pre-trained data distribution, influencing bias assessments.
4. Bias Quantification Methods: Different metrics might employ varying quantification methods, such as toxicity or hurtfulness, which inherently produce differing results.
5. Prompt Lexical Semantics: Variability in the intent and semantic content of prompts can lead to divergent bias measurements.
6. Targeted Subgroups: Differences in the subgroups targeted by each metric can result in varying bias assessments.

Introduction of CAIRO

The CAIRO framework effectively addresses these discrepancies by augmenting and carefully selecting prompts to maximize correlation across metrics. This approach involves:

• Data Augmentation: Generating paraphrases of original prompts using various large-scale LLMs to cover a range of sentence structures and semantic variations.
• Prompt Combination and Selection: Utilizing different combinations of these augmented prompts to identify those that enhance correlation across metrics.

By utilizing these methods, CAIRO significantly improves the Pearson correlation from values as low as 0.3 to as high as 0.9 for gender bias, demonstrating its efficacy in reconciling metrics.

Implications and Future Directions

The authors emphasize that achieving high correlation across fairness metrics doesn't merely improve reliability but also reflects on the consistency and robustness of bias evaluations. The application of CAIRO suggests the potential of similar augmentation and selection strategies in enhancing the evaluation of LLMs across various empirical tasks.

Theoretical implications of this work involve deeper understanding of bias in LLMs and its measurement, contributing to developing more consistent and just models. Practically, this work facilitates a more reliable application of LLMs in sensitive environments, where unbiased language outputs are crucial.

In future developments, exploring further into diverse datasets and augmentations could broaden the applicability of CAIRO. Additionally, examining how this approach could be adapted or extended to other forms of bias in machine learning systems stands as a promising avenue of research.

In conclusion, this paper forwards a pivotal shift in how fairness metrics can be utilized and interpreted, offering a robust pathway towards enhanced fairness in AI systems through improved evaluative consistency.