UnpredictaBench: Benchmarking LLM Unpredictability

• UnpredictaBench is a benchmark that measures LLM unpredictability by evaluating distributional calibration and capability prediction using both stochastic generation and performance forecasting.
• It employs rigorous statistical tests such as the two-sample Kolmogorov–Smirnov test along with debiased Wasserstein-1 Z-score and Jensen–Shannon divergence to assess output fidelity.
• The framework also outlines a design blueprint for capability predictability, highlighting challenges like mode collapse, shot instability, and transfer failures across model families.

UnpredictaBench is the name used in recent arXiv literature for benchmarking forms of LLM unpredictability that are not captured by average accuracy alone. In its most explicit formulation, it is a 2026 evaluation suite for distributional randomness, asking whether a model can repeatedly generate samples that match a specified target distribution rather than merely produce varied or plausible outputs (Abaskohi et al., 4 Jun 2026). In a related but distinct line of work, the same label is used for a design blueprint for benchmarking unpredictability in LLM capability prediction from historical BIG-bench records, with emphasis on residual error, transfer across model families, shot instability, and extrapolation to larger scales (Ye et al., 2023). Taken together, these uses of the term frame unpredictability as a measurable property of both stochastic generation and performance forecasting.

1. Meaning and scope

In the 2026 benchmark, UnpredictaBench is designed to answer a narrow question: when prompted to “be stochastic,” can an LLM actually sample from a target distribution, or does it collapse toward a few plausible answers? The benchmark is motivated by settings in which LLMs are treated as stand-ins for humans or complex systems, including economic simulations, epidemiological counterfactuals, mixed-strategy games, and other noisy processes. In those settings, the relevant requirement is not output diversity in the colloquial sense, but distributional calibration: the empirical distribution of repeated outputs should match the target distribution in support, shape, tails, and mass allocation (Abaskohi et al., 4 Jun 2026).

This formulation explicitly distinguishes output diversity from distributional calibration. A model may emit many different strings and still misallocate probability mass, overconcentrate on a mode, truncate tails, or fail on multimodal support. UnpredictaBench therefore evaluates repeated-sample fidelity to a black-box target distribution rather than one-off plausibility or judge-model preference (Abaskohi et al., 4 Jun 2026).

A separate antecedent uses “UnpredictaBench” in a different sense. There, the concern is not whether an LLM can sample from a prescribed stochastic process, but whether one can predict LLM task performance from metadata such as model family, parameter count, task identity, and shot count. The motivating study asks whether a regression function can recover normalized BIG-bench performance from historical experiment records, and then uses those findings to outline a benchmark of capability unpredictability centered on residuals, transfer failures, and hard-to-predict task regions (Ye et al., 2023).

2. Benchmark composition for distributional randomness

The 2026 UnpredictaBench contains 448 problem instances in English. These span 40 probability distributions selected from Wikipedia, covering absolutely continuous, discrete, joint or multivariate, mixed, and non-numeric families. The benchmark is organized into seven task categories: Text Explicit, Text Implicit, Code Explicit, Code Implicit, Multimodal, Shuffling, and Real-World Scenario (Abaskohi et al., 4 Jun 2026).

The task construction is deliberately heterogeneous. Canonical families include Normal, Beta, Exponential, Poisson, Negative Binomial, Dirichlet, Multinomial, Logistic, Chi-square, Fréchet, Student’s $t$, Weibull, and Pareto. Parameters are chosen to probe both concentrated and spread-out regimes. In the code-based tasks, models see Python or NumPy sampling logic; in explicit tasks the code directly calls a sampler, whereas in implicit tasks the target distribution arises from transformations such as ratios, sums, or nonlinear maps. Multimodal tasks include mixtures and additive constructions intended to expose mode collapse. Shuffling tasks require a uniformly random permutation of lists of up to five items. Real-world scenarios encode stochasticity through OS concurrency, garbage collection, network simulations, distributed systems, hashing, or MCMC processes (Abaskohi et al., 4 Jun 2026).

The prompt inventory is itself structured. The released composition includes 398 GPT-5.4–authored prompts—specifically 159 Text Explicit, 79 Text Implicit, 80 Code Explicit, and 80 Code Implicit, with 80 multimodal and 318 unimodal—plus 50 human-authored prompts, comprising 30 Real-World and 20 Shuffling tasks. The Real-World subset is divided into OS concurrency (6), garbage collection (6), network simulations (5), distributed systems (5), hashing (4), and MCMC (4) (Abaskohi et al., 4 Jun 2026).

Ground truth is generated per instance. For each task, a reference sample $B$ of size $M = 10{,}000$ is drawn from the target generator, whether that generator is a standard sampler, a stochastic program, or a canonical construction for the task. The benchmark reports that resampling the ground-truth reference shows stable results, indicating low sensitivity to the particular draw of $B$ (Abaskohi et al., 4 Jun 2026).

3. Statistical machinery and evaluation protocol

The core metric is KS@N, built from the two-sample Kolmogorov–Smirnov test. For each task, the evaluator collects $N$ model samples and compares their empirical CDF to the empirical CDF of the ground-truth sample. The underlying statistic is

$D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$

where $F_n$ and $G_m$ are the empirical CDFs of the model and ground-truth samples. The null hypothesis is that both samples come from the same distribution. UnpredictaBench defines a task as a pass when the two-sample KS $p$-value satisfies $p_{ks,t} \ge 10^{-4}$, and then aggregates over tasks as

$B$0

Larger $B$1 makes the test stricter, because increasingly small distributional mismatches become detectable as more samples are observed (Abaskohi et al., 4 Jun 2026).

The benchmark supplements KS@N with two additional distances. The first is a debiased Wasserstein-1 Z-score (WDZ), where $B$2 is computed from the sorted samples and then standardized using a permutation null over pooled-sample random splits. The second is Jensen–Shannon divergence (JSD), computed from KDEs on a common grid. These auxiliary measures are intended to capture aspects of mismatch not fully summarized by KS alone (Abaskohi et al., 4 Jun 2026).

Sequence-valued tasks require a scalar reduction. For shuffling, each permutation $B$3 is mapped via its Lehmer code $B$4, each coordinate is normalized to $B$5, and the benchmark scores only the first normalized coordinate $B$6. The rationale given is that $B$7 is well-spread under uniform permutations and therefore serves as a stringent one-dimensional proxy, although the evaluation remains primarily univariate (Abaskohi et al., 4 Jun 2026).

The experimental protocol uses $B$8 samples per task by default, temperature $B$9 by default, and reasoning off by default. A dedicated reasoning setting, denoted “xhigh,” allows up to 4096 tokens for chain-of-thought. Prompts request exactly one value, or a list in list-prompting ablations, in a structured format to simplify parsing. Outputs are parsed using a strict pattern; when parsing fails, a GPT-4o-mini extractor is used, with up to 5 retries per call, and invalid outputs are discarded and resampled. Infrastructure is primarily via OpenRouter, with some local runs on an RTX 3090, 24GB VRAM, and the reported total API cost is approximately $300 (Abaskohi et al., 4 Jun 2026).

4. Empirical findings on calibrated stochastic generation

The headline empirical result is that no evaluated model exceeds 40% at KS@100. The strongest reported system, Nemotron-3 Super 120B, attains approximately 32.6–35.4% depending on run or setting. Many models lie between 0–20%, with examples including GPT-4o at about 24%, DeepSeek V3.2 at about 22%, GPT-5.4 at about 15%, and GPT-4o-mini at about 10%. A small open-weight model, Qwen-3.5-2B, reaches roughly 17–18%, while Claude Sonnet 4.6 is reported at about 5% and OLMo-3 7B at about 3% (Abaskohi et al., 4 Jun 2026).

Category-level behavior is uneven. Shuffling and Code are the most demanding categories. Some strong proprietary systems reportedly collapse to 0% on shuffling, whereas other models maintain approximately 37% there. Real-World tasks can be relatively high for some models—for example, Llama-3.2-1B instruct is reported near 59%—but very low for others, including Nemotron-3 Super 120B at about 3.3%. Explicit prompting generally helps, but several models do better on implicit tasks, which the benchmark interprets as evidence that explicit distribution names can overanchor generation toward memorized prototypes rather than the specified parameterization (Abaskohi et al., 4 Jun 2026).

Difficulty also varies sharply by target family. At KS@100, the easiest distributions are Bernoulli at about 43%, Categorical at about 35%, and Discrete Uniform at about 17%. The hardest include Fréchet and Dirichlet at about 1.7%, Negative Binomial at about 5.2%, Negative Multinomial at about 6.1%, and Compound Poisson, Erlang, Inverse Gaussian, and Pareto, all below 9%. Heavy-tailed and multivariate targets are therefore especially problematic (Abaskohi et al., 4 Jun 2026).

The metric’s dependence on sample size is central to interpretation. All models score approximately 100% at $M = 10{,}000$0, because a single draw is rarely rejectable, but performance declines monotonically as $M = 10{,}000$1 grows, with the steepest drops occurring between $M = 10{,}000$2 and $M = 10{,}000$3. A cited example is Claude Sonnet 4.6, which falls from about 98% at KS@2 to about 5% at KS@100. This behavior underscores that apparent one-shot plausibility is not evidence of calibrated repeated sampling (Abaskohi et al., 4 Jun 2026).

Several intervention studies refine this picture. Reasoning can improve final-answer KS modestly; for example, Nemotron-3 Super 120B gains about +3.3 at KS@20. Yet numbers extracted from reasoning traces can be dramatically worse, including a reported −33.2 at KS@20 for the same model, which the benchmark interprets as evidence that deliberation often revisits a narrow candidate set rather than sampling the support broadly. Temperature helps many strong models: Nemotron-3 Super 120B peaks near $M = 10{,}000$4 with average KS@100 ≈ 39.6 across Text/Code, whereas $M = 10{,}000$5 is severely harmful. List prompting can also help substantially, with reported gains such as +17.1 for Nemotron-3 Super 120B, +14.3 for Ministral-3 3B, and +17.6 for OLMo-3 7B when asking for lists of ten values per call (Abaskohi et al., 4 Jun 2026).

The benchmark identifies several recurring failure modes: mode collapse, tail truncation or miscalibration, discrete miscalibration, and shuffling non-uniformity. It further reports evidence from logits indicating that, for some systems, collapse is already present in the model’s answer distribution before decoding, suggesting that decoding is not the sole source of the problem. Cross-benchmark analyses show KS@100 correlating positively with CREATE and NoveltyBench Utility10, but negatively with Distinct10, reinforcing the benchmark’s claim that distributional fidelity is different from lexical variety (Abaskohi et al., 4 Jun 2026).

5. Antecedent work on capability predictability

The 2023 BIG-bench study investigates a different problem: whether one can predict normalized LLM performance on a BIG-bench subtask from an experimental configuration $M = 10{,}000$6, where $M = 10{,}000$7 is model family, $M = 10{,}000$8 is size, $M = 10{,}000$9 is task identifier, and $B$0. The target $B$1 is normalized performance on the task’s preferred metric—exact_str_match, multiple_choice_grade, or rougeLsum—scaled to $B$2. The dataset contains 56,143 experiment records, filtered from BIG-bench to retain 313 subtasks across 134 tasks, 6 model families, and tasks with preferred metrics in the allowed set; programmatic tasks, all-zero tasks, and subtasks with fewer than 100 examples are removed (Ye et al., 2023).

A range of predictors is compared, including matrix completion baselines, kNN variants, tree ensembles, and a multi-layer perceptron (MLP). For trees and the MLP, the feature set includes one-hots for model family, model identity, task, and preferred metric, plus continuous size features such as total parameters, non-embedding parameters, FLOP-matched non-embedding parameters, their natural logs, and $B$3. The MLP is trained with MSE, with dropout, weight decay, and early stopping, and its first-layer task vectors serve as learned task representations (Ye et al., 2023).

On a random 10-fold CV split over $B$4 tuples, the MLP reaches RMSE ≈ 0.0500 and $B$5, while XGBoost is very similar at $B$6. Harder generalization protocols reduce performance but still yield high $B$7: L2.1 ≈ 0.9251, L2.2 ≈ 0.9391, L3 ≈ 0.8922, and L3 Composition ≈ 0.8597. The paper characterizes this as evidence of “highly learnable structure” in the historical records under interpolation, but it also finds that extrapolation is much harder: holding out GPT-3 200B yields $B$8 between about 0.70 and 0.88 depending on leakage setting, while PaLM 535B ranges from about 0.15 to 0.68 (Ye et al., 2023).

Predictability is not uniform across tasks. Highly predictable tasks include modified_arithmetic: three_digit_addition_control, two_digit_multiplication_control, linguistic_mappings: sentence_negation_json, and qa_wikidata. Less predictable tasks include linguistics_puzzles, checkmate_in_one, gender_inclusive_sentences_german, paragraph_segmentation, and authorship_verification:swapped. The diagnostics attribute low predictability to surprising or non-smooth scaling curves, overestimation, and emergent-like jumps. The study also finds that zero-shot transitions are harder, larger models are intrinsically harder to predict, and sparse families such as Gopher benefit strongly from multi-group learning (Ye et al., 2023).

6. UnpredictaBench as a capability-predictability design blueprint

Building on those BIG-bench results, the same source material outlines an UnpredictaBench-style benchmark for measuring where capability prediction fails. The proposed core quantities are per-task MSE and $B$9 under specified splits, a per-task unpredictability score

$N$0

with clipping to $N$1 when $N$2, family-transfer $N$3 obtained by holding out model families, shot-instability defined through

$N$4

and an aggregate instability index

$N$5

It also proposes size-transfer RMSE/$N$6 and cross-task generalization metrics based on L3 and L3 Composition splits (Ye et al., 2023).

The benchmark-composition principles are similarly explicit. Candidate tasks should include those with high residuals or low $N$7, those showing non-smooth scaling or frequent overestimation “false positives,” those that fail in family-transfer or size-transfer, and those with high $N$8, especially strong 0→1-shot discontinuities. To avoid redundancy, the design uses MLP-derived task embeddings $N$9, clusters them with $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$0-means in Euclidean space, and chooses tasks near cluster centroids. A value-aware variant filters candidates using a global task value $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$1, computed from Best-of-5000 search logs. The blueprint also recommends balancing emergent and non-emergent tasks and offering budget-aware tiers at $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$2 (Ye et al., 2023).

A key empirical result supporting this design is the “small-bench” search. Using nested CV over model families, the work shows that BIG-bench Lite and BIG-bench Hard (BBH) are sub-optimal for the objective of recovering full-benchmark performance. A Best-of-5000 subset with $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$3 is reported to be as informative as BBH with $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$4, while random 16-task subsets can also rival BBH. $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$5-means subsets are often competitive with or better than search-based subsets at equal budget, and $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$6-means + Task Value improves further, supporting the view that diversity and global utility jointly matter for compact benchmark design (Ye et al., 2023).

This blueprint therefore treats unpredictability as a property of generalization gaps rather than raw benchmark difficulty. A plausible implication is that, under this conception, UnpredictaBench is less a conventional leaderboard and more a stress test for whether performance-prediction methods survive transfer across families, scales, tasks, and shot regimes (Ye et al., 2023).

7. Related frameworks, limitations, and significance

UnpredictaBench sits near, but is not identical to, PredictaBoard, a 2025 benchmark for instance-level score predictability. PredictaBoard defines an assessor $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$7 that predicts the Bernoulli success variable $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$8 for a prompt $D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,$9, and evaluates selective prediction through rejection thresholds, coverage, error on the accepted set, AUARC, and the predictably valid region

$F_n$0

It also reports AUROC, Brier score, and Winkler’s score, and studies in-distribution performance on MMLU-Pro with out-of-distribution evaluation on BBH. The paper states that it does not explicitly cite UnpredictaBench, but conceptually positions its framework as complementary to an UnpredictaBench-style agenda centered on unpredictability and safety-oriented error anticipation (Pacchiardi et al., 20 Feb 2025).

Both versions of UnpredictaBench come with important limitations. The 2026 benchmark is primarily univariate even for multivariate targets, because sequence-valued or joint outputs are reduced to scalar proxies; the KS test is more sensitive near the median than in the tails; the Lehmer $F_n$1 mapping does not fully characterize permutation distributions; prompts are English-only and code tasks are Python-only; and prompt style may reflect bias from GPT-generated templates (Abaskohi et al., 4 Jun 2026). The capability-predictability blueprint inherits limitations from BIG-bench composition, sparse metadata about pretraining or prompting, the small number of model families $F_n$2, potential overfitting under random splits, and possible mismatch between search objectives and final evaluation protocols (Ye et al., 2023).

Despite these constraints, the significance of the term across both usages is clear. In the distributional benchmark, the central claim is that calibrated stochastic generation is a distinct capability from plausibility and diversity, and that failure on simple repeated-sample tests already casts doubt on using LLMs as stand-ins for stochastic systems (Abaskohi et al., 4 Jun 2026). In the capability-prediction lineage, the central claim is that average predictability under interpolation can coexist with sharp failures under transfer or extrapolation, making unpredictability itself a benchmarkable object rather than a residual nuisance (Ye et al., 2023). Together with assessor-based frameworks such as PredictaBoard, this literature shifts evaluation from raw task performance toward a broader question: not only what LLMs can do, but where their behavior can be forecast, trusted, or safely rejected (Pacchiardi et al., 20 Feb 2025).