Post-hoc Rationalization

• Post-hoc rationalization is the process of generating a secondary, human-readable explanation after a model or decision is made, offering simplified justifications.
• It employs surrogate models, feature-attribution maps, and counterfactual techniques to approximate and explain opaque behaviors in systems such as medical AI.
• Despite inherent approximation errors and fidelity critiques, it remains a useful strategy in enhancing predictability, calibration, and justification across various applications.

Searching arXiv for papers on post-hoc rationalization and explainability. Searching arXiv for "post-hoc rationalization explainability medical AI" and related terms. Post-hoc rationalization denotes the production of an explanatory or justificatory artifact after a decision, prediction, or verdict has already been reached. In explainable machine learning, it is typically formalized as a secondary procedure that takes an opaque predictor and an input and returns a simpler object intended to explain the output: if a black-box medical AI model is $f:X\to Y$, a post-hoc explanation is $E:(f,x)\mapsto \Delta$, where $\Delta$ may define a surrogate $g_\Delta:X\to Y$, a saliency overlay, or a feature-attribution decomposition that is easier to inspect than $f$ itself (Hatherley et al., 29 Apr 2025). In adjacent literatures, the same expression refers to generated justifications that follow an already selected answer or verdict in LLMs, and to retrospective self-justification in decision theory, where agents adopt ex post “rationales” to make past choices appear less regrettable (Devasier, 1 Mar 2026, Eyster et al., 2021). Across these settings, the central issue is the same: whether an after-the-fact rationale is merely a plausible story, or whether it can nevertheless support prediction, understanding, calibration, or justified action.

1. Definitions, notation, and scope

A standard formalization treats post-hoc explanation as a mapping from an opaque model and an instance to a human-readable rationalization. One version writes $E:(M,x)\to r\in R$, where $M:X\to Y$ is a trained predictive model, $\hat y=M(x)$, and $r$ may be a rule set, feature weights, a counterfactual, or a heat-map (Mitros et al., 2019). A more explicit formulation used in medical AI sets $\Delta=E(f,x)$ and associates $E:(f,x)\mapsto \Delta$0 with a simpler surrogate $E:(f,x)\mapsto \Delta$1, together with local and global fidelity criteria such as

$E:(f,x)\mapsto \Delta$2

and

$E:(f,x)\mapsto \Delta$3

When the explanation artifact is a feature-attribution vector $E:(f,x)\mapsto \Delta$4, consistency may also be expressed through a rank-correlation metric $E:(f,x)\mapsto \Delta$5 (Hatherley et al., 29 Apr 2025).

The same family of methods is often organized along two axes: scope and model dependence. Scope distinguishes global explanations, which ask what a model has learned overall, from local explanations, which ask why the model produced a particular output for a given $E:(f,x)\mapsto \Delta$6. Model dependence distinguishes model-agnostic procedures, which require only query access, from model-specific procedures, which exploit internal structure. This yields four standard categories: global model-agnostic, global model-specific, local model-agnostic, and local model-specific; representative methods include surrogate models, rule induction, feature-importance or sensitivity analyses, and counterfactual explanations (Mitros et al., 2019).

A related but philosophically distinct formalization appears in scientific machine learning. There, $E:(f,x)\mapsto \Delta$7 denotes the true function governing a phenomenon, $E:(f,x)\mapsto \Delta$8 is a trained model that is “reliable enough,” $E:(f,x)\mapsto \Delta$9 is a post-hoc explainer, and $\Delta$0 is an explanation that is “faithful enough.” The resulting explanatory chain is written

$\Delta$1

or equivalently $\Delta$2, with separate approximation steps for model–world reliability and explanation–model faithfulness (Oh et al., 28 Jun 2026).

A further extension generalizes post-hoc rationalization beyond explanation of predictors. In Computational Interpretabilism, a trained black-box model $\Delta$3 is paired with an interpretable approximation

$\Delta$4

with fidelity and scope bounded in advance by conditions such as $\Delta$5 on a declared domain $\Delta$6 (Oh, 2024). This broadens post-hoc rationalization from a purely local explanatory device to an epistemic interface between opaque computation, domain knowledge, and empirical testing.

2. Faithfulness critiques and epistemic limits

The most persistent critique is that post-hoc rationalizations are approximations of model behavior rather than reconstructions of actual reasoning. In the medical AI formulation, this criticism appears as approximation error

$\Delta$7

which may be large or unstable under small perturbations of $\Delta$8; even trivial or untrained models can yield deceptively plausible saliency maps, and the expected approximation error $\Delta$9 can therefore be non-negligible (Hatherley et al., 29 Apr 2025). Related empirical concerns include “placebic explanations,” which can induce as much trust as genuine ones, trust gaps $g_\Delta:X\to Y$0, and an automation bias index

$g_\Delta:X\to Y$1

with studies reporting scores that are not significantly positive or are paradoxically negative (Hatherley et al., 29 Apr 2025).

A stronger objection targets not just fidelity to a model, but claims about the world. The reliability–faithfulness chain $g_\Delta:X\to Y$2 does not, on its own, establish that $g_\Delta:X\to Y$3 reveals how the phenomenon $g_\Delta:X\to Y$4 is actually structured (Oh et al., 28 Jun 2026). The reason is structural: reliability checks whether $g_\Delta:X\to Y$5 tracks outcomes, and faithfulness checks whether $g_\Delta:X\to Y$6 tracks $g_\Delta:X\to Y$7, but neither accesses the inner workings of $g_\Delta:X\to Y$8. Oh and Jin state this as a general “No” to the question of whether reliability plus faithfulness suffices for structure discovery, and they argue that the failure persists even in the ideal limit of perfect reliability and perfect faithfulness (Oh et al., 28 Jun 2026).

Three failure cases make this limit precise. In the first, a model achieves high held-out accuracy on chest X-rays by exploiting laterality markers rather than lung pathology; saliency methods may faithfully highlight the markers, but only an external intervention reveals the confound. In the second, a proxy feature and the true causal feature co-vary in every available dataset, so no amount of additional held-out data can distinguish the wrong mechanism from the right one. In the third, two models can compute exactly the same function on all inputs while implementing completely different internal computations; a faithful explanation reports whichever internal mechanism the model uses, but reliability and faithfulness cannot determine which mechanism matches the real phenomenon (Oh et al., 28 Jun 2026).

This yields an important conceptual distinction. Absent external corroboration, post-hoc rationalization can support a “how-possibly” explanation or “objectual understanding,” but not a “how-actually” explanation of real structure (Oh et al., 28 Jun 2026). A plausible implication is that disputes over post-hoc rationalization often conflate two targets: explanation of a model’s behavior and explanation of the world the model is trained on.

3. Functional value in medical and scientific AI

Recent defenses do not deny approximation error; they instead argue that imperfect explanations can still be useful for human–AI practice. In medical AI, the key positive claim is functional rather than mechanistic: post-hoc explanations can improve users’ functional understanding of a model in the sense of predictability, increase clinician–AI team accuracy, and assist clinicians in justifying AI-informed decisions (Hatherley et al., 29 Apr 2025). One formalization defines a predictability score

$g_\Delta:X\to Y$9

where $f$0 is a small interpretable perturbation indicated by the explanation. The cited defense also reports that when radiologists use saliency masks $f$1, overall diagnostic accuracy rises by $f$2 percentage points relative to no explanations, i.e.,

$f$3

(Hatherley et al., 29 Apr 2025).

This defense is explicitly non-absolutist. Post-hoc explanations are not presented as a “silver bullet” for the black-box problem, and the paper concludes only that they remain “a useful strategy” for addressing it in medical AI (Hatherley et al., 29 Apr 2025). The crucial move is to combine local rationalizations with institutional explanations $f$4, including metadata about model training, bias-mitigation processes, performance curves, and regulatory provenance. The clinician’s final justification is then represented as

$f$5

so that the post-hoc artifact contributes to a broader justification narrative rather than serving as a stand-alone warrant (Hatherley et al., 29 Apr 2025).

The same bounded defense appears in Computational Interpretabilism. There, scientific understanding is characterized as mediated understanding arising through a four-way dialogue among model behavior, post-hoc methods, domain knowledge, and empirical validation, while bounded factivity holds that complete factual fidelity is neither attainable nor necessary (Oh, 2024). Explanatory claims are acceptable only within declared scope conditions such as a domain $f$6 and fidelity bounds $f$7, and are then embedded in a hypothesis-testing loop: interpretability suggests a hypothesis $f$8 about the phenomenon, empirical tests evaluate $f$9, and the approximation $E:(M,x)\to r\in R$0 or domain $E:(M,x)\to r\in R$1 is refined accordingly (Oh, 2024).

These two defenses converge on a common position. Post-hoc rationalization is epistemically weakest when it is treated as direct access to internal mechanism, and strongest when it is treated as a bounded, testable, and institutionally supplemented interface between opaque models and downstream scientific or clinical action (Oh, 2024, Hatherley et al., 29 Apr 2025).

4. Rationalization in language-model justifications

In language-model research, post-hoc rationalization has become a term of art for answer-conditioned or verdict-conditioned justifications. In masked diffusion LLMs for fact verification, it is defined as the situation in which the model settles on a global verdict $E:(M,x)\to r\in R$2 early in iterative refinement and subsequently generates a justification $E:(M,x)\to r\in R$3 that explains the already chosen $E:(M,x)\to r\in R$4, yielding the causal asymmetry $E:(M,x)\to r\in R$5 rather than $E:(M,x)\to r\in R$6 (Devasier, 1 Mar 2026). On AVeriTeC dev examples, LLaDA-8B fixes its verdict within the first few diffusion steps, and forcing “reasoning-first” behavior by delaying verdict unmasking degrades performance from $E:(M,x)\to r\in R$7 to $E:(M,x)\to r\in R$8 as the threshold rises to $E:(M,x)\to r\in R$9 (Devasier, 1 Mar 2026). The same study reports that when an incorrect verdict is forced, the model rationalizes it in $M:X\to Y$0 of cases, and that verdicts are strongly dependent on justification quality: $M:X\to Y$1 accuracy with corrupted justifications versus $M:X\to Y$2 with ground-truth justifications (Devasier, 1 Mar 2026).

An analogous issue appears in reverse chain-of-thought generation, where a model is given a query $M:X\to Y$3 and a pre-committed answer $M:X\to Y$4 and must synthesize a reasoning trace $M:X\to Y$5. Here post-hoc rationalization is answer anchoring: the chain becomes a backward justification from $M:X\to Y$6 rather than a forward derivation to $M:X\to Y$7 (Peng et al., 16 Feb 2026). To quantify this, the paper introduces a three-level hierarchy of lexical, entropic, and probabilistic anchoring. The probabilistic quantity

$M:X\to Y$8

measures how much the trace reduces uncertainty about the known answer, while $M:X\to Y$9 and $\hat y=M(x)$0 capture surface overlap and entropy dynamics (Peng et al., 16 Feb 2026). A notable empirical result is that semantic suppression—prompting the model not to reveal the answer until the end—reduces lexical anchoring but increases entropic and probabilistic anchoring, which the authors attribute to Ironic Process Theory (Peng et al., 16 Feb 2026).

Chain-of-thought work on soft-reasoning tasks sharpens the distinction between causal guidance and explanatory faithfulness. One metric measures influence,

$\hat y=M(x)$1

the fraction of cases in which CoT flips the model’s answer, while a separate faithfulness score

$\hat y=M(x)$2

tracks whether the CoT explicitly verbalizes a known causal cue that changed the answer (Lewis-Lim et al., 27 Aug 2025). The key finding is misalignment: distilled-reasoning models change their initial answer on $\hat y=M(x)$3 of cases on average, instruction-tuned models on only $\hat y=M(x)$4, and multi-step reasoning models on $\hat y=M(x)$5, yet cue verbalization can remain very low even when the chain is causally active (Lewis-Lim et al., 27 Aug 2025). A chain can therefore guide an answer without truthfully reporting its cause, or can report a cue without materially influencing the answer.

This literature redefines post-hoc rationalization from a problem of model interpretation to a problem of generated justifications. The common failure mode is not only low fidelity to internal computation, but also inversion of deliberative order: the answer is selected first, and the rationale is synthesized afterward.

5. Measurement, mitigation, and constructive uses

A growing methodological literature treats post-hoc rationalization as something to quantify and actively mitigate. The most direct framework is explanatory inversion, in which an attribution vector $\hat y=M(x)$6 depends on the output $\hat y=M(x)$7 rather than the forward $\hat y=M(x)$8 relationship (Tan et al., 11 Apr 2025). Inversion Quantification introduces a reliance-on-outputs score $\hat y=M(x)$9, an explanation-faithfulness score $r$0, and an inversion score

$r$1

so that $r$2 corresponds to low output reliance and high faithfulness (Tan et al., 11 Apr 2025). On synthetic tabular, image, and text data with injected spurious features, LIME, SHAP, Integrated Gradients, and Occlusion all show increased inversion under spurious correlations. The proposed Reproduce-by-Poking wrapper then uses forward perturbation checks to penalize unstable attributions, with theoretical claims $r$3, $r$4, and $r$5, and an empirical average inversion reduction of approximately $r$6 across methods and domains (Tan et al., 11 Apr 2025).

Evaluation can also proceed by comparison against intrinsically interpretable models. In histopathology, ProtoPNet was adapted to PatchCamelyon and used as a benchmark for post-hoc saliency methods via ten saliency metrics from the saliency-model literature (Amorim et al., 2023). On this setup, SmoothGrad and Occlusion were found to have a statistically bigger overlap with ProtoPNet, while Deconvolution and Lime had the least (Amorim et al., 2023). This does not prove ground-truth faithfulness, but it establishes an automatic quantitative protocol for comparing post-hoc rationalizers against a native attribution mechanism.

Post-hoc rationalization has also been used constructively to redesign explanatory representations. Post-hoc Part-prototype Networks decompose a trained classification head $r$7 into interpretable part-prototypes $r$8 satisfying $r$9, so that class logits can be exactly recovered from pooled prototype activation maps while simultaneously answering both “where” and “what” the model uses (Tan et al., 2024). On CUB-200-2011 with $\Delta=E(f,x)$0, the method reports consistency and stability gains across several backbones while preserving backbone accuracy, and on ImageNet it keeps the original $\Delta=E(f,x)$1 top-1 of ResNet-34 and $\Delta=E(f,x)$2 of ViT-Base (Tan et al., 2024).

Related work uses rationalization itself as a downstream control signal. AMPLIFY derives automated natural-language rationales from proxy-model attributions and reports prediction accuracy improvements of about $\Delta=E(f,x)$3 over a wide range of tasks (Krishna et al., 2023). Collaborative Calibration uses multi-agent deliberation to produce post-hoc confidence rationales and improve calibration without training (Yang et al., 2024). Class-wise Selective Rationalization, in turn, learns factual and counterfactual rationales for each class via a three-player adversarial game, so that evidence supporting alternative conclusions becomes explicit rather than collapsed into a single explanation (Chang et al., 2019). Taken together, these approaches suggest that post-hoc rationalization is not only an object of critique but also a design space for measurement, correction, and structured human–model interaction.

6. Ex post rationalization in decision theory

Outside machine learning, ex post rationalization is formalized as a motive in dynamic choice. In the two-period model of Dean, Kıbrıs, and Masatlioglu, a decision problem $\Delta=E(f,x)$4 consists of a first-period menu $\Delta=E(f,x)$5, a second-period menu correspondence $\Delta=E(f,x)$6, and a prior $\Delta=E(f,x)$7 over states $\Delta=E(f,x)$8. The agent has a material utility $\Delta=E(f,x)$9 and a closed, convex set of rationales $E:(f,x)\mapsto \Delta$00, with $E:(f,x)\mapsto \Delta$01, from which a rationale $E:(f,x)\mapsto \Delta$02 may be selected ex post to justify the past action $E:(f,x)\mapsto \Delta$03 (Eyster et al., 2021).

At date $E:(f,x)\mapsto \Delta$04, after choosing $E:(f,x)\mapsto \Delta$05 and observing $E:(f,x)\mapsto \Delta$06, the agent simultaneously chooses $E:(f,x)\mapsto \Delta$07 and $E:(f,x)\mapsto \Delta$08 to maximize

$E:(f,x)\mapsto \Delta$09

where $E:(f,x)\mapsto \Delta$10 is the weight on rationalization utility (Eyster et al., 2021). The bracketed term is non-positive, so the agent trades off material payoffs against retrospective self-justification. The framework then distinguishes a naïf, a sophisticate, and an empathetic sophisticate according to how first-period choice anticipates later rationalization (Eyster et al., 2021).

The model yields identified primitives and tractable comparative statics. Theorem 7 provides a representation result under axioms including Linearity, Existence, Rationalization, Monotonicity, Quasiconvexity, and Continuity; Theorem 8 identifies $E:(f,x)\mapsto \Delta$11 up to the usual positive-affine normalization (Eyster et al., 2021). Under lattice-theoretic complementarity assumptions, Theorem 3 implies distortion–sunk-cost effects: if the realized first-period choice $E:(f,x)\mapsto \Delta$12 is ex post too large, the second-period choice is distorted upward relative to the material optimum, and the chosen rationale parameter $E:(f,x)\mapsto \Delta$13 is likewise shifted upward (Eyster et al., 2021). The examples include a snowstorm ticket, repeated experimental choices, belief elicitation under quadratic scoring, and two-part tariffs (Eyster et al., 2021).

This decision-theoretic literature changes the meaning of rationalization. The issue is no longer whether an explanation faithfully reports a model’s mechanism, but whether agents reshape subsequent choices and beliefs to retrospectively defend earlier commitments. Even so, the structural similarity is clear: in both machine and human settings, rationalization is generated after a choice has been made and can causally redirect later judgment, confidence, or action.

7. Conceptual synthesis

Across explainable AI, scientific modeling, language-model reasoning, and decision theory, post-hoc rationalization names a family of after-the-fact justificatory processes rather than a single technique. In one form, it produces a surrogate $E:(f,x)\mapsto \Delta$14, saliency map, or attribution vector for an opaque predictor. In another, it produces a generated justification for an already chosen answer or verdict. In a third, it is itself part of the agent’s utility function and helps explain sunk-cost and stickiness effects (Hatherley et al., 29 Apr 2025, Devasier, 1 Mar 2026, Eyster et al., 2021).

The major controversy is whether these artifacts should be treated as explanations of actual mechanism. The strongest negative result is that no pure combination of model reliability and explanation faithfulness suffices to establish real-world structure, because both links terminate at the model rather than the phenomenon (Oh et al., 28 Jun 2026). The strongest positive result is more modest: when bounded by declared scope, combined with institutional or domain knowledge, and subjected to empirical validation, post-hoc rationalizations can improve predictability, team accuracy, calibration, or hypothesis generation even when $E:(f,x)\mapsto \Delta$15 and $E:(f,x)\mapsto \Delta$16 (Hatherley et al., 29 Apr 2025, Oh, 2024).

A plausible implication is that the most defensible use of post-hoc rationalization is neither eliminativist nor credulous. It is weakest as a warrant for unobserved causal structure and strongest as a controlled interface for diagnosis, comparison, counterfactual analysis, calibration, and follow-up testing. Under that interpretation, post-hoc rationalization is not equivalent to faithful mechanism recovery, but it remains a technically important and methodologically diverse component of contemporary interpretability and reasoning research.