Linking reasoning behaviours to fairness outcomes across diverse tasks

Establish robust links between observable reasoning behaviours in large language models and fairness outcomes across diverse tasks.

Background

BIASTRACE is developed and validated primarily on the BBQ structured question-answering benchmark, with an initial demonstration on the COMPAS fairness task. Although the framework associates reasoning behaviours with biased outputs and provides some evidence of relevance to group-fairness measures, the paper does not establish that these relationships persist across a broad range of tasks. Determining whether reasoning-level signals reliably predict or explain fairness outcomes in diverse application settings remains an unresolved challenge.

References

With transparent recognition and ties both at their maximum, the link between them cannot be tested here.

— The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits  (2609.09048 - Vohra et al., 8 Sep 2026) in Limitations

While we present an initial demonstration of linking reasoning behaviour to group fairness metrics for the COMPAS task, we recognise that robustly linking reasoning behaviours to fairness outcomes across diverse tasks remains an open challenge.

— BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs  (2608.14161 - Ramineni et al., 14 Aug 2026) in Limitations section