Model Multiplicity in Machine Learning

• Model multiplicity is the phenomenon where different models achieve near-equivalent predictive performance while exhibiting distinct behaviors and internal representations.
• Researchers quantify multiplicity using metrics like Δmax, self-consistency, and SVCCA to reveal differences in fairness, robustness, and individual-level predictions.
• Practical implications include designing systems that expose a council of valid models, improving transparency, accountability, and trust in AI deployments.

Model multiplicity is the phenomenon that multiple models can achieve similar predictive performance while exhibiting materially different underlying behaviors, predictions, or other deployment-relevant properties. In contemporary machine learning, it is closely associated with the Rashomon effect and the Rashomon set: under specified performance constraints, many models may be equally acceptable, yet differ on individual cases, internal representations, fairness, robustness, privacy, explanations, or recourse (Ganesh et al., 2024, Ganesh, 2023, Heljakka et al., 2022). Recent work treats this plurality not merely as a backend nuisance but as a substantive issue for trustworthiness, accountability, and system design, including user-facing interfaces that expose a council of equally accurate models rather than collapsing them into a single answer (Eerlings et al., 2024).

1. Conceptual foundations

In the current literature, model multiplicity is usually defined at the level of near-equivalent predictive performance. One formulation describes it as the phenomenon that multiple models can achieve similar predictive performance on the nominal task while exhibiting materially different underlying behaviors (Ganesh, 2023). Another broad formulation defines it through two layers: first, models belong to the same Rashomon set when they satisfy specified performance constraints; second, multiplicity is present when those models still differ enough on some deployment-relevant behavior (Ganesh et al., 2024). In interactive-system work, the same idea is expressed more operationally as “using slightly different AI models yielding equally valid outcomes or predictions for the same task,” with the models recast as “many simultaneous expert advisors” (Eerlings et al., 2024).

This literature distinguishes multiplicity from several neighboring ideas. It is not identical to conventional ensemble learning, whose purpose is often to collapse multiple outputs into a vote or average; model multiplicity instead emphasizes retaining and exposing the plurality of valid models (Eerlings et al., 2024). It is also not exhausted by average-performance comparisons, because two models can look equivalent by conventional accuracy and still differ in which examples they misclassify, how they generalize, or how they behave under shift (Heljakka et al., 2022). A recurring explanation is that modern learning problems are under-specified and models are over-parameterized, so standard objectives do not pin down a unique acceptable solution (Ganesh, 2023).

A further distinction concerns the level at which multiplicity is observed. “Predictive multiplicity” refers to disagreement in outputs across risk-equivalent models, whereas “representational multiplicity” refers to differences in internal representations even when outputs agree on the currently observed sample (Heljakka et al., 2022). This asymmetry is central: observed predictive disagreement implies internal divergence, but the absence of observed prediction disagreement on a finite sample does not imply the absence of latent representational divergence.

2. Formalizations and metrics

A common formal starting point is the Rashomon set. In one general definition, if $\mathbf{\Delta}^P$ is a set of performance-difference functions with thresholds $\mathcal{E}^P$, then two models $h_1,h_2$ are in the same Rashomon set when

$$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$

Multiplicity is then defined by adding a second constraint: despite satisfying all performance conditions, the models differ enough on some deployment-relevant metric $\delta^M$, namely $\delta^M(h_1,h_2)>\epsilon^M$ (Ganesh et al., 2024). This formulation is deliberately broad: it does not restrict multiplicity to one hypothesis class, one data distribution, or final predictions alone.

Other papers define multiplicity more directly over a set of risk-equivalent models. For a family $H=\{h_k\}_{k=1}^K$ with near-equal empirical risk, predictive multiplicity over a sample $S$ is

$$PM(S,H)=\mathbb{E}_{x\in S}\,\mathrm{Var}_{h\in H}\{h(x)\},$$

while representational multiplicity is measured through SVCCA-based activation similarity, with higher average correlation implying lower representational multiplicity (Heljakka et al., 2022). For benchmarking trustworthy behavior across settings, another line of work translates heterogeneous trustworthy metrics into a common language of “accuracy under appropriate interventions,” and summarizes multiplicity by $\Delta_{max}$ within a slice and $\mathcal{E}^P$0 across a whole multiplicity sheet (Ganesh, 2023).

Instance-level metrics are especially prominent. The legal and fairness literature uses self-consistency: if $\mathcal{E}^P$1 models are in the Rashomon set, $\mathcal{E}^P$2 output $\mathcal{E}^P$3, and $\mathcal{E}^P$4 output $\mathcal{E}^P$5 for a particular individual, then self-consistency is defined as

$\mathcal{E}^P$6

Lower self-consistency indicates greater arbitrariness for that individual (Ganesh et al., 2024). Other papers use ambiguity, discrepancy, and related metrics to quantify whether at least one near-optimal model disagrees on a point, how much a single alternative model can disagree with a reference, or how widely disagreement spreads through the Rashomon set (Baltz et al., 13 Jun 2026, Cavus, 16 Apr 2025). The literature therefore has no single dominant metric; rather, it offers a family of measures targeted at prediction, score dispersion, internal representation, or individual burden.

3. Sources and empirical patterns

Multiplicity can arise from many small changes in the learning pipeline. Recent work explicitly broadens its sources beyond random seed variation to include dataset selection, feature selection, train/validation splits, hyperparameters, optimizer choice, and evaluation criteria (Ganesh et al., 2024). In image-classification benchmarking, multiplicity was induced through variation in random seed, learning rate, batch size, augmentation method, optimizer, and architecture (Ganesh, 2023). In tabular LLM fine-tuning, fine-tuning multiplicity is linked to seed changes, random weight initialization, retraining on additional samples, and retraining after deleting samples (Hamman et al., 2024).

Empirical studies show that multiplicity is often much larger on deployment-relevant metrics than on nominal accuracy. In the UTKFace benchmark, standard accuracy had $\mathcal{E}^P$7, while fairness on the Asian subgroup had $\mathcal{E}^P$8, OOD robustness on FairFace had $\mathcal{E}^P$9, perturbation accuracy for privacy with $h_1,h_2$0 had $h_1,h_2$1, and PGD accuracy for security with $h_1,h_2$2 had $h_1,h_2$3 (Ganesh, 2023). The same study reports that overall accuracy across benchmarked models varied only from about $h_1,h_2$4 to $h_1,h_2$5, whereas accuracy for older Asian individuals ranged from $h_1,h_2$6 to $h_1,h_2$7.

Fine-tuned tabular LLMs show the same pattern. In T-Few experiments with 40 models, arbitrariness was about $h_1,h_2$8 on Adult, $h_1,h_2$9 on German Credit, and $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$0 on Diabetes, despite comparable average accuracies across the alternative fine-tunings (Hamman et al., 2024). By contrast, recidivism-risk modeling provides an important counterexample to the simplistic view that many near-optimal models always imply severe predictive arbitrariness: a pool of 78 similarly accurate interpretable models with $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$1 still showed about $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$2 average self-consistency on the test set, and nearly $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$3 of instances exhibited perfect predictive agreement across all models (Singh et al., 1 Jun 2026). This suggests that structural diversity and predictive multiplicity are distinct empirical questions.

4. Individual-level, fairness, and legal implications

A major contemporary shift is from population-level performance to the burden borne by particular individuals. One line of work argues that multiplicity is a source of arbitrariness that can affect not only final predictions but also unpredictability, planning burden, robustness, privacy, explanations, and recourse (Ganesh et al., 2024). The central point is that average performance can remain stable while the outcome for a particular person changes substantially depending on which equally acceptable model is selected.

This burden can be unequally distributed across groups. The same legal-technical study reports that adding fairness constraints does not necessarily reduce individual arbitrariness: on the South German Credit dataset, selecting by accuracy within a $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$4 threshold yielded 121 models, while selecting by both accuracy and group fairness within $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$5 yielded only 6 models, yet the latter set showed lower self-consistency for more individuals (Ganesh et al., 2024). On UTKFace, the Asian group had roughly twice the fraction of low-self-consistency individuals compared to others, and threshold optimization to improve accuracy parity further increased multiplicity for all groups while maintaining disparity (Ganesh et al., 2024).

These findings support a legal claim developed in the Canadian anti-discrimination context: heightened arbitrariness for protected groups can constitute an adverse impact under the Moore test, because the burden is not merely a worse expected outcome but a higher exposure to randomness, instability, and inability to plan (Ganesh et al., 2024). At the same time, recidivism-risk work shows that the relationship between multiplicity and arbitrariness is not uniform. In that setting, many similarly accurate models with comparable error-rate disparities did not necessarily translate into severe predictive multiplicity, and a lowest-risk policy—assigning each individual the lowest risk among the admissible models—was reported as an effective way to address residual arbitrariness (Singh et al., 1 Jun 2026).

5. Visualization, benchmarking, and decision support

Recent work has turned multiplicity into an interface-design problem. “AI-Spectra” frames deployment as the concurrent use of all models and their predictions in an interactive system rather than the expert-only comparison of candidate models (Eerlings et al., 2024). Its dashboard uses a bar chart of custom Chernoff faces, called Chernoff Bots. For a given input, the x-axis shows all possible class labels; every model is represented by one Chernoff Bot and is stacked into the bar corresponding to its predicted label, so each bar’s height equals the number of models predicting that label (Eerlings et al., 2024). Consensus appears as the tallest bar, disagreement as dispersion across bars, and repeated glyph features allow users to see what kinds of models contribute to each prediction.

This deployment framing is explicitly distinct from conventional ensembling. The goal is not to aggregate outputs into a single vote, but to expose a council of equally accurate experts and the model properties associated with their disagreements (Eerlings et al., 2024). The dashboard visualizes model metadata rather than internal learned parameters: training-data composition and augmentation settings, hidden-layer count, dropout, batch size, validation-set usage, and related properties are encoded in the Chernoff Bot design, while optimizer is included in metadata but not given a visual encoding (Eerlings et al., 2024).

Benchmarking work addresses a complementary problem: how to compare multiplicity across fairness, robustness, privacy, and security in a common language. “Multiplicity sheets” record raw scores, axis-wise variability through $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$6, and overall variability through $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$7, after translating trustworthy metrics into “accuracy under appropriate interventions” (Ganesh, 2023). This makes it possible to report that trustworthy multiplicity can be three to five times larger than standard-accuracy multiplicity in the same benchmark, and to identify whether variation is driven primarily by seed, architecture, batch size, or another design axis (Ganesh, 2023).

6. Extensions across domains and system settings

Multiplicity has been extended beyond conventional centralized classification. In survival analysis for predictive maintenance, the prediction object is a time-indexed event-risk quantity $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$8, and multiplicity is defined through conflicting risk estimates within a survival-model Rashomon set (Cavus, 16 Apr 2025). That work introduces ambiguity, discrepancy, and obscurity for survival models, and reports that ambiguity steadily increases, reaching up to $$\delta^P_i(h_1,h_2)\leq \epsilon^P_i \qquad \forall (\delta^P_i,\epsilon^P_i)\in(\mathbf{\Delta}^P,\mathcal{E}^P).$$9–$\delta^M$0 of observations, while discrepancy is lower and obscurity remains mild and concentrated in a few models (Cavus, 16 Apr 2025).

In federated learning, centralized Rashomon notions do not transfer directly because raw data are not shared and client distributions are heterogeneous. Recent work therefore distinguishes three federated Rashomon perspectives: a global Rashomon set defined over aggregated statistics across clients, a $\delta^M$1-agreement Rashomon set requiring local agreement for at least a specified fraction of clients, and individual Rashomon sets defined on each client’s local distribution (Heilmann et al., 10 Feb 2026). Empirically, strict client consensus can fail altogether: on ACS Income, the $\delta^M$2-agreement setting produced no Rashomon sets within the tested $\delta^M$3 range (Heilmann et al., 10 Feb 2026).

Security work has adapted multiplicity in two different ways. In model stealing, the surrogate is treated not as a unique clone but as part of an extraction-induced Rashomon set: high fidelity to the target can coexist with substantial ambiguity, discrepancy, Rashomon Capacity, and fairness variation across equally faithful surrogates (Baltz et al., 13 Jun 2026). In edge training of small LLMs, multiplicity becomes a defense mechanism rather than a reliability concern: multiple lightweight model replicas are trained on independently sampled client subsets, and divergence across trajectories is used as a signal of poisoning, with earlier and more reliable detection than classical single-model defenses such as Flanders and Robust (Behfar et al., 5 Jun 2026).

7. Limitations, controversies, and open directions

The literature is explicit that multiplicity is easier to diagnose conceptually than to manage operationally. A repeated limitation is computational cost. Exposing or benchmarking multiplicity often requires retraining many models; in federated learning this makes candidate generation expensive enough that current empirical studies use simple architectures and cross-silo settings, while privacy-preserving estimation of score-based multiplicity metrics through differentially private histograms was reported as poor enough to leave trusted-server assumptions in place for the main experiments (Heilmann et al., 10 Feb 2026). Interactive deployment work likewise notes unresolved scalability, increased maintenance burden, and a larger carbon footprint as the number of models grows (Eerlings et al., 2024).

Another open issue is evaluation itself. There is no consensus on how arbitrariness should be measured as a fairness issue, and many metrics—self-consistency, ambiguity, Rashomon capacity, representational measures—have not been systematically consolidated into a single framework (Ganesh et al., 2024). Additional specifications during model selection help only partially: in the UTKFace benchmark, requiring models to rank in the top $\delta^M$4 or top $\delta^M$5 on fairness, robustness, privacy, and security reduced some unforeseen fairness and security multiplicity, but unforeseen robustness and privacy multiplicity remained essentially unchanged (Ganesh, 2023). This supports the view that one cannot eliminate multiplicity simply by checking a finite list of desired properties.

A newer data-centric direction argues that multiplicity is also shaped by small dataset changes. Under a neighbouring-datasets framework, two datasets differing by only one point can induce different Rashomon sets, and, under a shared Rashomon threshold $\delta^M$6, greater inter-class overlap can yield lower multiplicity because the harder dataset admits fewer models below the fixed loss threshold (Ganesh et al., 24 Oct 2025). The same work extends this perspective to active learning and data imputation, proposing multiplicity-aware acquisition and imputation strategies that can steer ambiguity without large changes in predictive accuracy (Ganesh et al., 24 Oct 2025). This suggests that future multiplicity research will increasingly connect model plurality to upstream data curation, not only to downstream model selection.

Across these strands, a stable conclusion has emerged. Model multiplicity is not merely the existence of many good models; it is the practical fact that near-equivalent models can disagree on people, groups, explanations, risks, or downstream actions. Whether that plurality is treated as hidden arbitrariness, a benchmarked form of uncertainty, an end-user transparency mechanism, or a systems-level design resource depends on the application. What unifies the literature is the rejection of the assumption that one acceptable loss value determines one acceptable model.