Stereotype alignment of noise-induced directional judgments

Establish whether the directional judgments induced by surface noise in LLM-as-a-judge bias measurement favor the stereotype-aligned or counter-stereotypical option.

Background

The paper defines fabricated bias operationally as a transition from a neutral judgment to a directional judgment after surface noise is introduced. This establishes instability and directional commitment, but it does not establish that the selected option is actually stereotype-aligned.

The benchmark primarily distinguishes candidates by names and option positions, and demographic associations are not auditable for several categories, including disability. Consequently, the relationship between noise-induced directional commitments and actual stereotype direction remains unresolved.

References

Whether that commitment favours the stereotyped direction is a separate question, and how far it could be answered after the fact depends on the axis.

— When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text  (2609.11067 - Ryu et al., 10 Sep 2026) in Limitations, paragraph 'What a transition establishes'

We do not test whether judges recognise the demographic associations the names carry.

— When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text  (2609.11067 - Ryu et al., 10 Sep 2026) in Limitations, paragraph 'Judge comprehension of names'