Rashomon Capacity: Model Multiplicity

• Rashomon Capacity is a concept that quantifies the effective breadth of near-optimal model sets by measuring predictive score spread and decision variability.
• It is defined and operationalized across probabilistic, binary, and sequential settings using information-theoretic metrics, asymptotic growth laws, and optimization intervals.
• The metric reveals trade-offs among prediction stability, fairness, and explainability, offering practical insights for model selection and risk analysis.

Searching arXiv for papers on Rashomon Capacity and closely related Rashomon-set formulations. Rashomon Capacity denotes the extent, richness, or effective breadth of a Rashomon set: a family of models that are near-optimal under a specified performance criterion yet remain meaningfully different in predictions, scores, explanations, structures, or downstream decisions. Across the literature, the term does not have a single universal formalization. In probabilistic classification, it is introduced as an information-theoretic per-sample multiplicity metric derived from the score vectors attainable within a Rashomon set (Hsu et al., 2022). In binary classification under a Bayes-optimal flip representation, it is analyzed through the size and asymptotic growth of the largest possible Rashomon set of approximately equally accurate classifiers (Dai et al., 26 Jan 2025). In sequential decision-making, quantum foundations, federated learning, interpretable model classes, chaotic forecasting, and allocation under scarcity, the literature uses closely related constructs—behavioral equivalence classes, no-global-section obstructions, optimization ranges, empirical set counts, and horizon-dependent contractions—that do not always carry the name “Rashomon Capacity” but function as capacity notions in practice (Gross et al., 19 Dec 2025, Ghose, 29 Dec 2025, Heilmann et al., 10 Feb 2026, Kale et al., 17 Apr 2026).

1. Probabilistic classification and the original information-theoretic metric

In probabilistic classification, Rashomon Capacity is defined at the level of an individual sample $x_i$. The setup begins with a hypothesis space

$$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$

population loss

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$

and Rashomon set

$$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$

For a fixed sample $x_i$, the $\epsilon$-multiplicity set is

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$

The metric first defines a spread functional

$$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$

then maximizes over distributions on the multiplicity set,

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$

With $d=D_{KL}$, the paper defines Rashomon Capacity as

$$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$0

(Hsu et al., 2022).

This construction makes Rashomon Capacity a per-sample measure of predictive multiplicity in the simplex rather than a count of classifiers. The paper states that $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$1 is exactly the channel capacity of a channel whose rows are the score vectors in $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$2, which gives the metric its name (Hsu et al., 2022). The induced scale satisfies $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$3, with $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$4 when all score vectors are identical and $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$5 when the multiplicity set contains all simplex vertices. The same work proves that for each sample there exists a subset $$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$6 with

$$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$7

such that

$$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$8

linking the metric to a finite support representation even when the Rashomon set itself is large (Hsu et al., 2022).

The metric was proposed partly because earlier multiplicity measures such as ambiguity and discrepancy operate on thresholded labels rather than score vectors. This makes Rashomon Capacity a score-domain notion. The same paper recommends reporting multiplicity in both score and decision domains, because thresholding can convert nearly identical score vectors into opposite hard decisions (Hsu et al., 2022). This suggests that one important axis of Rashomon Capacity is the choice of predictive representation: score-level capacity and decision-level capacity need not coincide.

2. Combinatorial and asymptotic capacity in binary classification

A second, more explicitly set-size-oriented formulation appears in the study of the largest possible Rashomon set of binary classifiers around a Bayes-optimal predictor (Dai et al., 26 Jan 2025). There, a model is encoded by a flip vector

$$H \triangleq \{h_\theta: X \to \Delta_c:\theta\in\Theta\},$$9

where $$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$0 indicates that the classifier flips the Bayes-optimal prediction on record $$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$1. If

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$2

then the expected accuracy of $$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$3 is

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$4

The Rashomon set is defined as

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$5

equivalently,

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$6

(Dai et al., 26 Jan 2025).

In this framework, Rashomon Capacity is not a single scalar but a bundle of linked notions. The paper’s main size theorem gives

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$7

with

$$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$8

Hence, for large $$L(h_\theta)\triangleq \mathbb{E}_{X,Y}[\ell(h_\theta(X),Y)],$$9,

$$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$0

(Dai et al., 26 Jan 2025). This is the paper’s core notion of capacity: an asymptotic exponential growth law for the number of approximately equally accurate classifiers.

The same paper derives a behavioral counterpart to set size. For a record with weight $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$1, the asymptotic flip probability is

$$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$2

Because $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$3, this can also be written as

$$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$4

This quantifies how much an individual’s prediction varies across the Rashomon set (Dai et al., 26 Jan 2025). The paper also proves that the average model in the Rashomon set uses essentially all available error tolerance: $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$5 (Dai et al., 26 Jan 2025).

The combination of $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$6, the flip function $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$7, and the concentration of sampled models near the boundary of the error budget yields a distinctly combinatorial interpretation of Rashomon Capacity. It is simultaneously a count of admissible models, a distribution over individual-level outcome instability, and a geometric weighted-knapsack region inside $$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$8 (Dai et al., 26 Jan 2025). This differs from the per-sample information-theoretic metric of probabilistic classification, but the two are compatible: one measures spread in output space, the other measures the abundance and structure of near-optimal binary prediction patterns.

3. Capacity as fairness, sparsity, and actionable variability

Several papers use Rashomon-set structure to characterize how much secondary behavior remains attainable under near-optimal performance. They do not define Rashomon Capacity by name, but they operationalize it through ranges, extrema, and search procedures over Rashomon sets (Langlade et al., 7 Feb 2025, Dai et al., 26 Jan 2025).

An enumeration-free mathematical-programming framework defines the Rashomon set as

$$R(H,\epsilon)\triangleq \{h_\theta\in H:\; L(h_\theta)\le \epsilon\}.$$9

and studies the attainable interval of a secondary property $x_i$0 subject to a sparsity cap $x_i$1: $x_i$2 For scoring systems and decision diagrams, this interval is computed exactly using mixed-integer formulations, with $x_i$3 instantiated as statistical parity or equal opportunity (Langlade et al., 7 Feb 2025). The broader this interval, the more behavioral freedom the near-optimal set contains. This suggests a range-based notion of capacity: not how many models exist, but how widely consequential properties can vary while predictive performance remains nearly unchanged.

The fairness-focused study of multiplicity in binary classification reaches a parallel conclusion from the combinatorial side. It shows that fairness varies widely across equally accurate models in $x_i$4, that the average or uniformly sampled model is not close to the fairest model, and that large capacity enlarges the feasible fairness frontier (Dai et al., 26 Jan 2025). It also gives efficient search procedures for statistical parity and approximate procedures for FPR and TPR disparity, turning Rashomon Capacity into a usable reservoir of alternative models rather than a passive count.

This line of work also exposes a tension. Large capacity creates room for substantially fairer models, but the same multiplicity creates arbitrariness and unequal exposure to prediction instability (Dai et al., 26 Jan 2025). A closely related healthcare-allocation study argues that model-space multiplicity may fail to transfer to allocation-space multiplicity. It distinguishes the model Rashomon set

$x_i$5

from the set of $x_i$6-equal-utility allocations

$x_i$7

with cardinality

$x_i$8

The paper’s central claim is that the theoretical richness of the Rashomon set in model space often does not transfer to comparable richness in allocation space (Jain et al., 20 Mar 2025). This suggests that Rashomon Capacity can be decision-rule dependent: a large model set does not imply a large set of materially distinct downstream actions.

4. Sequential, federated, and chaotic generalizations

Outside static supervised classification, the literature develops task-specific analogues of capacity. In sequential decision-making, the Rashomon effect is defined using exact behavioral equivalence in a finite Markov decision process. Policies are memoryless deterministic,

$x_i$9

and induce a DTMC with transition probabilities

$\epsilon$0

The paper defines the Rashomon effect to occur when there exist policies $\epsilon$1 trained on the same expert dataset such that: first, they induce identical DTMCs on $\epsilon$2 with respect to a property $\epsilon$3; second, $\epsilon$4, where $\epsilon$5 is an internal-structure metric such as saliency rankings (Gross et al., 19 Dec 2025). It does not define Rashomon Capacity explicitly, but it identifies several natural proxies: the number of policies in a behavioral equivalence class, the number of distinct attribution patterns within that class, and the breadth of action unions encoded by a permissive policy. In the reported taxi-domain experiment, 100 behavioral-cloning policies produced 10 behavioral equivalence classes, with the largest class containing 82 policies that all satisfy the 5-job reachability property and induce identical DTMCs (Gross et al., 19 Dec 2025). This is a sequential analogue of large capacity, but one grounded in exact model checking rather than approximate return similarity.

In federated learning, the paper formalizes three different Rashomon sets: a global Rashomon set,

$\epsilon$6

a $\epsilon$7-agreement Rashomon set,

$\epsilon$8

and individual client Rashomon sets,

$\epsilon$9

It then adapts score-based multiplicity metrics, including Rashomon Capacity,

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$0

to federated settings (Heilmann et al., 10 Feb 2026). This makes capacity perspective-dependent: globally aggregated capacity, agreement-based capacity, and client-specific capacity need not coincide. The paper also uses Rashomon ratio as a set-size proxy, distinguishing score spread from the relative volume of the federated Rashomon set (Heilmann et al., 10 Feb 2026).

Chaotic forecasting pushes the concept further by making capacity horizon-dependent. The paper defines a horizon-constrained Rashomon set

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$1

and proves exponential contraction: $$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$2 for appropriately chosen $$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$3 (Kale et al., 17 Apr 2026). It also introduces a Lyapunov-weighted Rashomon ratio,

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$4

and shows that effective ambiguity is bounded by $$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$5 up to an exponentially small remainder (Kale et al., 17 Apr 2026). This is a dynamic conception of capacity: in chaotic systems, multiplicity is not fixed but contracts with forecast horizon at a rate determined by the maximum Lyapunov exponent.

5. Structural, geometric, and representational viewpoints

A complementary line of work studies Rashomon Capacity as a structural property of the whole near-optimal model set rather than as a scalar. In sparse decision trees, the Rashomon set is defined by

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$6

with

$$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$7

SORTD enumerates trees in nondecreasing objective-value order, providing anytime lower bounds on $$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$8 and making capacity empirically observable as a function of $$M_\epsilon(x_i)\triangleq \{h(x_i): h\in R(H,\epsilon)\}\subseteq \Delta_c.$$9, depth, regularization, and feature space (Arslan et al., 5 Nov 2025). The earlier TREEFARMS framework completely enumerates the Rashomon set of sparse decision trees and stores it in a compressed Model Set representation (Xin et al., 2022). The resulting literature does not define capacity by a closed-form formula, but it turns capacity into an exact countable object for a highly nonlinear discrete model class.

For rule sets, the Rashomon set is

$$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$0

where

$$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$1

The paper develops exact branch-and-bound enumeration, approximate counting, and almost-uniform sampling. It identifies $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$2 as the main representation-dependent proxy for capacity and shows empirically that slight increases in $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$3 can cause the Rashomon set to grow exponentially fast (Ciaperoni et al., 2024). This suggests that in interpretable combinatorial model classes, capacity is often most usefully studied through exact counts, approximate counts, and sample-based summaries of feature reliance or fairness variation.

For infinite hypothesis classes, the infinite-family Rashomon-ratio paper provides a measure-theoretic generalization. Given a probability space $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$4, the true Rashomon set is

$$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$5

and the Rashomon ratio is

$$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$6

This extends finite-family proportion to probability mass. The paper proves that the ratio can be estimated by random sampling from $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$7, and that if the Rashomon ratio of a large family is large, then a random subset of the family is likely to contain a near-optimal classifier (Coupkova et al., 2024). This is a genuine infinite-family notion of capacity: not cardinality, but measure under a specified sampling distribution.

A very different structural interpretation appears in quantum foundations. The “Quantum Rashomon Effect” is described as the existence of locally coherent accounts $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$8 over contexts $$\rho(M_\epsilon(x_i),P_M)\triangleq \inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q),$$9 that do not admit a global section $$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$0 satisfying

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$1

The paper identifies this as a failure of gluing or nonexistence of a global section, and it interprets Rashomon-like multiplicity as a local-to-global obstruction rather than a count of models (Ghose, 29 Dec 2025). It does not define a numerical capacity, but it suggests several ingredients for one: number of contexts, size of compatible local-section families, overlap structure, and cohomological obstruction classes (Ghose, 29 Dec 2025). This suggests that in some domains Rashomon Capacity may be better understood as the maximal extent of mutually coherent but globally ungluable descriptions.

6. Practical interpretation, explanation uncertainty, and open questions

The practical literature increasingly treats Rashomon Capacity as a governance-relevant property. In explainability, several papers argue that the number of near-optimal models is less informative than the amount of non-redundant explanatory diversity they support. A practical sampling paper defines a generalized Rashomon set around a reference model

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$2

and uses proxies such as searching efficiency ratio and the functional efficiency range

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$3

to characterize explanation-space breadth (Li et al., 2024). This is not a formal capacity theory, but it makes explicit that hypothesis multiplicity and attribution multiplicity need not coincide.

An AutoML paper similarly defines a finite candidate-pool Rashomon set

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$4

with

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$5

It then aggregates partial dependence profiles across the set and finds that, on 35 regression datasets, the Rashomon PDP covers less than 70% of the best model’s PDP in most cases (Cavus et al., 19 Jul 2025). This is an empirical proxy for explanatory capacity: a larger near-optimal set often implies greater explanation variability.

The trustworthiness literature further emphasizes that a large Rashomon set is double-edged. One paper defines empirical and population Rashomon sets in the standard near-optimality form,

$$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$6

and studies raw multiplicity, predictive diversity, geometric diversity, set stability, and aggregate informativeness as operational components of capacity (Hsu et al., 26 Nov 2025). Its results suggest that a richer Rashomon set can support reactive robustness and shift resilience, but also increases distributional leakage when many near-optimal models are disclosed (Hsu et al., 26 Nov 2025). This suggests that capacity is not merely an epistemic resource; it can also be an exposure surface.

Across these domains, several limitations recur. The value of any capacity measure depends on the chosen tolerance $$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$7 or $$C_d(M_\epsilon(x_i)) \triangleq \sup_{P_M}\inf_{q\in\Delta_c}\mathbb{E}_{h\sim P_M} d(h(x_i)\|q).$$8, the performance criterion, the internal-difference metric, and, in infinite or sampled settings, the measure or search distribution over hypotheses (Hsu et al., 2022, Coupkova et al., 2024, Dai et al., 26 Jan 2025, Gross et al., 19 Dec 2025). Many papers study empirical approximations to the Rashomon set rather than the full set, so reported capacities are often lower bounds or representation-dependent surrogates (Ciaperoni et al., 2024, Cavus et al., 19 Jul 2025, Heilmann et al., 10 Feb 2026). In sequential and chaotic settings, exact equivalence is strong and clean but restrictive, which suggests the need for approximate variants based on occupancy measures, bounded probabilistic bisimulation distances, or horizon-aware tolerances (Gross et al., 19 Dec 2025, Kale et al., 17 Apr 2026). In explanation-focused work, observed disagreement can reflect either genuine model multiplicity or artifacts of the explainer, so explanation-space capacity should not be conflated automatically with function-space capacity (Spieker et al., 4 Jun 2026).

Taken together, the literature supports a plural view. Rashomon Capacity can mean an information-theoretic spread in predictive scores, an asymptotic exponential growth law for near-optimal classifiers, a fairness or sparsity range over the Rashomon set, a count or measure of near-optimal interpretable models, a horizon-dependent multiplicity curve in chaotic forecasting, or a local-to-global obstruction in contextual settings. The common thread is not a single formula, but a common question: how much room remains for materially different models, policies, explanations, or decisions once predictive performance is held essentially fixed?