Rashomon Effect in Machine Learning

• Rashomon Effect is the phenomenon where multiple distinct, nearly optimal models deliver similar predictive performance despite differences in their internal structure and feature emphasis.
• It arises from statistical, structural, and procedural sources, influencing how models are interpreted, validated, and deployed across various settings.
• Research leverages the Rashomon set to enhance interpretability, fairness, and robustness, guiding model selection beyond mere accuracy metrics.

The Rashomon Effect denotes the existence of multiple distinct models, descriptions, or representations that perform nearly equally well on the same task while differing in internal structure, feature reliance, or explanatory content. In machine learning, it is usually operationalized through a Rashomon set of near-optimal models; recent work treats this multiplicity as central to interpretability, robustness, scientific inference, AutoML, sequential decision-making, dimensionality reduction, and clinical deployment rather than as a marginal artifact of model selection (Parikh, 11 Jan 2026, Rudin, 5 Jul 2025).

1. Historical framing and conceptual scope

Leo Breiman introduced the term in the context of his contrast between “data models” and “algorithmic models.” In the modern restatement emphasized by Cynthia Rudin, Breiman’s formulation was that there is often “a multitude of different descriptions [equations $f(x)$] in a class of functions giving about the same minimum error rate,” and he used that observation to argue that good fit alone does not identify the true data-generating mechanism (Rudin, 5 Jul 2025). In this original framing, the Rashomon Effect is an epistemic warning: if many different models fit similarly well, then a single fitted model cannot automatically be interpreted as nature’s mechanism.

Subsequent work broadened the concept from a critique of parametric “data models” into a general account of model multiplicity. One contemporary synthesis defines the Rashomon Effect as “the existence of multiple, distinct models that achieve nearly equivalent predictive performance,” and distinguishes a strong Rashomon effect from mere local redundancy by requiring structural or functional diversity within the near-optimal set (Parikh, 11 Jan 2026). This broader framing makes the phenomenon relevant wherever model choice is underdetermined by performance alone.

A major reinterpretation in recent literature is that the Rashomon Effect is not only a threat to causal or scientific overreach; it is also a resource. Rudin argues that if there are many different-but-good models, some of them are likely to be interpretable, and that a large Rashomon Effect often undercuts the supposed simplicity–accuracy tradeoff sometimes called the “Occam Dilemma” (Rudin, 5 Jul 2025). This does not eliminate the original caution about causal claims, but it changes the practical lesson: multiplicity can be exploited for interpretability, fairness, robustness, or user fit, provided it is analyzed explicitly rather than ignored.

2. Formalizations of Rashomon sets and ratios

There is no single universal formalization. The common structure is a near-optimal sublevel set of a loss or objective, but papers differ on whether the threshold is additive or multiplicative, whether the set is empirical or population-level, and what counts as the relevant notion of equivalence.

A standard empirical definition is

$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$

where $L^*=L(f^*,\mathcal{D})$ is the empirical risk of the empirical risk minimizer. The associated Rashomon ratio is

$$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$

with $|\cdot|$ interpreted as an appropriate cardinality or volume measure (Parikh, 11 Jan 2026).

For infinite hypothesis classes, the ratio is defined with respect to a probability measure $\rho$ on hypothesis space rather than a simple count. The true Rashomon ratio becomes the $\rho$-probability that a random draw from the class lies in the near-optimal set, which allows the idea to extend beyond finite model libraries (Coupkova et al., 2024).

In AutoML and related empirical workflows, multiplicative tolerances are common. One formulation defines the best model $M^*$ by minimum loss over the model library $\mathcal{M}$, and then sets

$$\mathcal{R_\varepsilon} = \left\{ M_k \in \mathcal{M} \mid \phi(M_k) \leq \phi(M^*) * (1 + \varepsilon) \right\}.$$

This treats near-optimality as relative rather than additive and is particularly natural when candidate models come from a bounded AutoML search (Cavus et al., 19 Jul 2025).

In domains beyond static prediction, the equivalence criterion changes. In sequential decision-making, the relevant object is not one-step prediction but the policy-induced stochastic behavior in an environment. There the Rashomon effect is defined by policies trained on the same dataset that induce identical discrete-time Markov chains with respect to a property $$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$0 while differing under an internal metric $$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$1, such as saliency-based feature rankings (Gross et al., 19 Dec 2025). In chaotic forecasting, the Rashomon set is explicitly horizon-indexed,

$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$2

because predictive equivalence changes with lead time (Kale et al., 17 Apr 2026).

Setting	Formal object	Equivalence criterion
Supervised learning	$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$3	empirical loss within $$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$4
Infinite hypothesis classes	measure-based Rashomon ratio	$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$5-mass of the near-optimal set
AutoML	$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$6	loss within $$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$7 of best model
Sequential decision-making	Rashomon set of policies	identical induced DTMCs but different internal structure
Chaotic forecasting	horizon-constrained Rashomon set	horizon-$$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$8 loss within $$\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D}) = \{f\in\mathcal{F}: L(f,\mathcal{D}) \le L^*+\epsilon\},$$9

This variety suggests that “Rashomon set” is best understood as a family of task-specific near-equivalence constructions rather than a single canonical object.

3. Why many equally good models exist

A recent synthesis organizes the causes of the Rashomon Effect into three categories: statistical, structural, and procedural (Parikh, 11 Jan 2026). This taxonomy is important because the categories behave differently under more data, stronger assumptions, or altered optimization procedures.

Statistical sources arise from finite samples and noisy outcomes. If population risks differ by less than the empirical uncertainty scale, many models become empirically indistinguishable. Uniform-convergence intuition implies an empirical–population gap on the order of $L^*=L(f^*,\mathcal{D})$0 for classes of VC dimension $L^*=L(f^*,\mathcal{D})$1, so limited $L^*=L(f^*,\mathcal{D})$2 can flatten the effective loss landscape and enlarge the near-optimal region (Parikh, 11 Jan 2026). The same paper argues that outcome noise can further enlarge the effect by making finer distinctions non-generalizable and by pushing practice toward simpler classes that often have larger Rashomon ratios.

Structural sources persist even with infinite data. Non-convex objectives can contain multiple local minima, flat low-loss manifolds, or symmetry-related parameterizations. In neural networks, hidden-unit permutations and ReLU rescaling symmetries produce many equivalent or near-equivalent parameter settings; in causal or partially identified problems, multiple observationally equivalent structures can remain consistent with the same observed distribution (Parikh, 11 Jan 2026). One particularly sharp statement in that paper is that the identification region in partial identification is exactly a population-level Rashomon set with $L^*=L(f^*,\mathcal{D})$3: all models in the region fit the observables equally well.

Procedural sources reflect optimizer limitations and modeling choices. Initialization, mini-batch noise, early stopping, hyperparameter budgets, greedy tree construction, and deliberate restrictions to suboptimal or constrained model classes can all create or reveal multiplicity (Parikh, 11 Jan 2026). This type is not fundamental in the same way as non-identifiability, but it is consequential because actual pipelines are finite, approximate, and value-laden.

Work on variable importance with unobserved confounding sharpens the interaction between structural and interpretive multiplicity. UNIVERSE treats omitted-variable uncertainty and model multiplicity jointly, enlarging empirical Rashomon sets so that they can bound the true variable importance of the full-data conditional mean $L^*=L(f^*,\mathcal{D})$4 even when some predictors are unobserved (Donnelly et al., 14 Oct 2025). This suggests that Rashomon uncertainty is not exhausted by “many good observed-data models”; it can also reflect what the available variables fail to identify.

4. Explanatory multiplicity and explainable machine learning

The Rashomon Effect has become especially consequential in explainable machine learning because predictive equivalence does not imply explanatory agreement. A systematic empirical study framed the issue through three explanation-comparison scenarios: Numerical Stability (111) for the same model, sample, and attribution method; Solution Diversity (011) for different models, same sample, same method; and the classic Disagreement Problem (110) for different attribution methods applied to the same model and sample (Müller et al., 2023). Using AG News, Dry Bean, Breast Cancer, and Ionosphere, that study defined the Rashomon set as all models within $L^*=L(f^*,\mathcal{D})$5 accuracy of the dataset-best model and showed that models could be highly similar in predictive accuracy and Jensen–Shannon distance over outputs while still differing substantially in explanations.

The same work formalized attribution disagreement by

$L^*=L(f^*,\mathcal{D})$6

with attribution methods from $L^*=L(f^*,\mathcal{D})$7 and dissimilarity measures from $L^*=L(f^*,\mathcal{D})$8 (Müller et al., 2023). Two findings are especially important. First, explanation hyperparameters must be tuned per model instance, not just per method or dataset; Captum defaults for KernelSHAP and LIME were reported as insufficient for many models. Second, the apparent magnitude of the Rashomon effect depends strongly on the comparison metric: disagreement-based and Euclidean-based measures can produce qualitatively different pictures of the same model family.

Later work has shifted from merely documenting disagreement to designing methods that work over Rashomon sets. One proposal introduced two practical axioms for Rashomon-set exploration in XAI—generalizability and implementation sparsity—and used them to motivate an $L^*=L(f^*,\mathcal{D})$9-subgradient-based sampling framework intended to reveal attributionally distinct models without wasting search effort on redundant implementations (Li et al., 2024). Another proposal for AutoML defined the Rashomon Partial Dependence Profile as the average PDP over all near-optimal models and added bootstrap confidence intervals, along with the coverage rate and mean width of confidence intervals, to quantify whether the best model’s PDP is representative of the near-optimal family (Cavus et al., 19 Jul 2025). On 35 OpenML-CTR23 regression datasets, that study reported that in most cases the Rashomon PDP covered less than $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$0 of the best model’s PDP, indicating that single-model PDPs were often unrepresentative.

Faithfulness evaluation has also been reframed around the Rashomon set. A metamorphic-testing framework introduced five metamorphic relations linking feature attributions to perturbation-based model behavior, thereby assessing explanation faithfulness without ground-truth explanation labels (Spieker et al., 4 Jun 2026). The same paper showed that SHAP preserved more cross-model explanation diversity than LIME on California Housing and Wine Quality, but that much of this divergence was not validated by actual sensitivity differences. In scene understanding for autonomous driving, explanation ambiguity was quantified within Rashomon sets of interpretable pair-based gradient boosting models and graph-based GNNs using Qualitative Explainable Graphs; explanation disagreement was substantial in both classes, and especially severe for the GNNs, suggesting that explanation ambiguity was not merely a modeling artifact (Spieker et al., 3 Sep 2025).

Taken together, these results establish a general distinction: predictive multiplicity becomes explanatory multiplicity whenever explanations depend on the particular near-optimal model chosen.

5. Extensions beyond static supervised prediction

The Rashomon Effect has been extended to sequential decision-making by replacing “same predictions” with “same induced stochastic behavior.” In that setting, a policy $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$1 induces a discrete-time Markov chain when plugged into an MDP, and two policies exhibit the Rashomon Effect if they induce identical DTMCs with respect to a property $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$2 while differing internally under a metric $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$3, such as saliency-based feature rankings (Gross et al., 19 Dec 2025). In a taxi domain, 100 behavioral-cloning policies trained on the same expert dataset produced 10 behavioral equivalence classes, with 82 policies in the largest class satisfying the target property. Those 82 policies were behaviorally identical with respect to the verification target yet displayed different feature-importance rankings. The same study further reported that a majority-vote Rashomon ensemble improved robustness under distribution shift, and that a permissive policy derived from the Rashomon set preserved optimal performance while greatly reducing verification cost.

In dimensionality reduction, the Rashomon idea has been reformulated around embeddings rather than predictors. A loss-based Rashomon set was defined as the set of embedding parameters whose DR loss stays within $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$4 of optimal, and a graph-based Rashomon set was defined through similarity of weighted $$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$5-nearest-neighbor graphs across embeddings (Sun et al., 1 Apr 2026). That work treats the non-uniqueness of 2D embeddings as a resource rather than a defect: PCA-informed alignment makes axes interpretable, concept-alignment regularization aligns one embedding dimension with external labels or user-defined concepts, and stable nearest-neighbor relations across many good embeddings can be aggregated into refined embeddings with improved local structure.

In chaotic forecasting, multiplicity becomes explicitly time-dependent. Horizon-constrained Rashomon sets

$$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$6

capture the fact that models that are near-optimal at short lead times may cease to be near-optimal at longer horizons (Kale et al., 17 Apr 2026). That paper proves an exponential upper bound

$$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$7

linking contraction of the effective Rashomon set to the maximum Lyapunov exponent. This suggests that predictive equivalence itself can be horizon-specific rather than static.

The term has also been extended beyond machine learning. In quantum foundations, the Quantum Rashomon Effect denotes the impossibility of integrating multiple locally coherent observer-dependent quantum accounts into a single global narrative (Szangolies, 2020). A later note recast this phenomenon as a failure of gluing, in the same local-to-global sense used in sheaf-theoretic contextuality: locally consistent descriptions over different contexts exist, but no global section extends all of them at once (Ghose, 29 Dec 2025). This is a broader and more abstract use of the term, but it preserves the same structural motif of incompatible yet internally coherent perspectives.

6. Applications, trustworthiness, and practical consequences

Applied work increasingly treats the Rashomon Effect as a deployment issue rather than a purely theoretical one. In educational data mining, tree-based AutoML over the Open University Learning Analytics Dataset produced very large empirical Rashomon sets, and permutation variable importance was more stable in binary Pass/Fail prediction than in the harder Distinction/Pass/Fail multiclass setting; course DDD was especially unstable, illustrating how variable-importance conclusions can depend on which equally accurate model is examined (Kuzilek et al., 2024). In public mental health, a bootstrap-generated Rashomon set of random forests combined with PDPs was used to analyze PHQ-2 and GAD-2 risk, yielding stable age, sex, and education effects together with diurnal and circaseptan patterns, including early-morning risk peaks, while explicitly marking uncertainty through the spread of PDPs across the set (Cavus et al., 18 Jan 2026).

The effect also interacts with preprocessing and workflow design. In imbalanced classification, balancing methods such as random oversampling, SMOTE, random undersampling, and NearMiss were found to inflate predictive multiplicity, significantly increasing ambiguity and discrepancy even when AUC improved (Cavus et al., 2024). In interpretable modeling, the existence of many similarly accurate GAM/EBM configurations has been used as a basis for personalization: one study reported 92 GAMs with comparable performance ($$\rho(\epsilon,\mathcal{F},\mathcal{D}) = \frac{|\mathcal{R}(\epsilon,\mathcal{F},\mathcal{D})|}{|\mathcal{F}|},$$8) and found that 53 treatment users receiving bandit-based personalization ended up with 44 distinct final personalized configurations, although this did not translate into significant improvements in measured interpretability outcomes (Rosenberger et al., 11 May 2025).

Recent trustworthiness work makes the double-edged nature of multiplicity explicit. One paper argues that the Rashomon set can improve reactive robustness and remains stable under small distribution shifts, but that releasing more near-optimal models also increases information leakage because multiple models provide progressively richer views of the training data (Hsu et al., 26 Nov 2025). Another clinical paper addresses model choice under the Rashomon Effect by introducing Intervention Efficiency (IE), a capacity-aware metric for limited interventions, and the Perturbation Validation Framework (PVF), which selects models whose validation performance is most stable under structured perturbations of the validation data (Zhang et al., 18 Nov 2025). These approaches shift model selection away from tiny differences in F1 or accuracy and toward resource constraints, perturbation stability, and operational robustness.

Across these applications, a common conclusion recurs. Explanations, variable importance rankings, fairness properties, privacy risks, and intervention priorities can all vary across models that are nearly indistinguishable under conventional performance criteria. This implies that model selection, validation, and reporting cannot be reduced to “pick the best score and explain that model.” Rashomon-aware analysis instead treats the near-optimal family as the relevant unit: it asks what is stable across the family, what varies across it, and which additional criteria justify choosing one member over another.