Faithfulness of Large Language Model Decision Rationales

Establish whether the natural-language reasons provided by a single, monolithic large language model (e.g., GPT or Claude) for its recommended regulatory decisions are causally faithful to the model’s internal decision-making process or are merely post-hoc rationalizations presented after the fact.

Background

The paper argues that relying on a single LLM to make regulatory decisions would worsen explainability and accountability. Even when such a model outputs textual justifications, it is not evident that these justifications reflect the computations that produced the decision.

The authors explicitly note uncertainty about whether model-provided reasons actually determine the model’s decision versus being post-hoc rationalizations. Resolving this uncertainty is central to assessing the legitimacy, transparency, and suitability of LLMs for regulatory decision-making, and it motivates their proposal for a distributed multi-agent approach grounded in social choice theory.