Accuracy versus recognitional justice in generative AI

Determine when generative AI systems should accurately represent existing social reality and when they should instead produce deliberately idealized representations to challenge status subordination, including how to distinguish justified departures from reality from counterproductive distortions and assess their trade-offs against values such as reliability and trust.

Background

The paper argues that accurate representations may reproduce unjust social hierarchies when they reflect existing patterns of gender, racial, or other social inequality. Consequently, optimizing generative AI for descriptive accuracy may conflict with the goal of recognitional justice, understood through Nancy Fraser’s concept of parity of participation. The unresolved issue is how to determine when intentionally inaccurate or idealized outputs are normatively justified, and how such interventions should be evaluated against other socially important values.

References

This observation raises deeper normative questions that accuracy-oriented remedies can't answer: Should generative AI accurately represent social reality when that reality is structured by injustice, or should it deliberately produce a more idealized but descriptively inaccurate representation? When does departing from existing social reality constitute a justified challenge to status subordination, and at what point does it become an idealized and counter-productive distortion? These questions illustrate that whether a representation is accurate is not by itself the right normative question. What matters is whether, in any particular case, reflecting or departing from real existing conditions contributes to parity of participation. And of course, that's not the only relevant question---the case for inaccurate but recognitionally reparative generative AI outputs would have to contend with possible trade-offs against other socially important values, such as reliability and trust.

— From Fair Representation to Just Recognition in Generative AI  (2608.12669 - Engelmann et al., 13 Aug 2026) in Section “From Fair Representation to Just Recognition,” paragraph beginning “Put another way, accurate representations can be harmful”