---
title: 'UnpredictaBench: Benchmarking LLM Unpredictability'
url: https://www.emergentmind.com/topics/unpredictabench
type: topic
---

# UnpredictaBench: Benchmarking LLM Unpredictability

UnpredictaBench is the name used in recent arXiv literature for benchmarking forms of LLM unpredictability that are not captured by average accuracy alone. In its most explicit formulation, it is a 2026 evaluation suite for **distributional randomness**, asking whether a model can repeatedly generate samples that match a specified target distribution rather than merely produce varied or plausible outputs [2606.06622]. In a related but distinct line of work, the same label is used for a **design blueprint** for benchmarking unpredictability in LLM capability prediction from historical BIG-bench records, with emphasis on residual error, transfer across model families, shot instability, and extrapolation to larger scales [2305.14947]. Taken together, these uses of the term frame unpredictability as a measurable property of both stochastic generation and performance forecasting.

## 1. Meaning and scope

In the 2026 benchmark, UnpredictaBench is designed to answer a narrow question: when prompted to “be stochastic,” can an LLM actually sample from a target distribution, or does it collapse toward a few plausible answers? The benchmark is motivated by settings in which LLMs are treated as stand-ins for humans or complex systems, including economic simulations, epidemiological counterfactuals, mixed-strategy games, and other noisy processes. In those settings, the relevant requirement is not output diversity in the colloquial sense, but **distributional calibration**: the empirical distribution of repeated outputs should match the target distribution in support, shape, tails, and mass allocation [2606.06622].

This formulation explicitly distinguishes **output diversity** from **distributional calibration**. A model may emit many different strings and still misallocate probability mass, overconcentrate on a mode, truncate tails, or fail on multimodal support. UnpredictaBench therefore evaluates repeated-sample fidelity to a black-box target distribution rather than one-off plausibility or judge-model preference [2606.06622].

A separate antecedent uses “UnpredictaBench” in a different sense. There, the concern is not whether an LLM can sample from a prescribed stochastic process, but whether one can predict LLM task performance from metadata such as model family, parameter count, task identity, and shot count. The motivating study asks whether a regression function can recover normalized BIG-bench performance from historical experiment records, and then uses those findings to outline a benchmark of **capability unpredictability** centered on residuals, transfer failures, and hard-to-predict task regions [2305.14947].

## 2. Benchmark composition for distributional randomness

The 2026 UnpredictaBench contains **448 problem instances in English**. These span **40 probability distributions selected from Wikipedia**, covering absolutely continuous, discrete, joint or multivariate, mixed, and non-numeric families. The benchmark is organized into seven task categories: **Text Explicit**, **Text Implicit**, **Code Explicit**, **Code Implicit**, **Multimodal**, **Shuffling**, and **Real-World Scenario** [2606.06622].

The task construction is deliberately heterogeneous. Canonical families include Normal, Beta, Exponential, Poisson, Negative Binomial, Dirichlet, Multinomial, Logistic, Chi-square, Fréchet, Student’s \(t\), Weibull, and Pareto. Parameters are chosen to probe both concentrated and spread-out regimes. In the code-based tasks, models see Python or NumPy sampling logic; in explicit tasks the code directly calls a sampler, whereas in implicit tasks the target distribution arises from transformations such as ratios, sums, or nonlinear maps. Multimodal tasks include mixtures and additive constructions intended to expose mode collapse. Shuffling tasks require a uniformly random permutation of lists of up to five items. Real-world scenarios encode stochasticity through OS concurrency, garbage collection, network simulations, distributed systems, hashing, or MCMC processes [2606.06622].

The prompt inventory is itself structured. The released composition includes **398 GPT-5.4–authored prompts**—specifically **159 Text Explicit, 79 Text Implicit, 80 Code Explicit, and 80 Code Implicit**, with **80 multimodal** and **318 unimodal**—plus **50 human-authored prompts**, comprising **30 Real-World** and **20 Shuffling** tasks. The Real-World subset is divided into **OS concurrency (6), garbage collection (6), network simulations (5), distributed systems (5), hashing (4), and MCMC (4)** [2606.06622].

Ground truth is generated per instance. For each task, a reference sample \(B\) of size \(M = 10{,}000\) is drawn from the target generator, whether that generator is a standard sampler, a stochastic program, or a canonical construction for the task. The benchmark reports that resampling the ground-truth reference shows stable results, indicating low sensitivity to the particular draw of \(B\) [2606.06622].

## 3. Statistical machinery and evaluation protocol

The core metric is **KS@N**, built from the **two-sample Kolmogorov–Smirnov test**. For each task, the evaluator collects \(N\) model samples and compares their empirical CDF to the empirical CDF of the ground-truth sample. The underlying statistic is

$$
D_{n,m} = \sup_x \left|F_n(x) - G_m(x)\right|,
$$

where \(F_n\) and \(G_m\) are the empirical CDFs of the model and ground-truth samples. The null hypothesis is that both samples come from the same distribution. UnpredictaBench defines a task as a pass when the two-sample KS \(p\)-value satisfies \(p_{ks,t} \ge 10^{-4}\), and then aggregates over tasks as

$$
KS@N = \frac{1}{T}\sum_{t=1}^{T}\mathbf{1}\{p_{ks,t} \ge 10^{-4}\}\times 100\%.
$$

Larger \(N\) makes the test stricter, because increasingly small distributional mismatches become detectable as more samples are observed [2606.06622].

The benchmark supplements KS@N with two additional distances. The first is a **debiased Wasserstein-1 Z-score (WDZ)**, where \(W_1\) is computed from the sorted samples and then standardized using a permutation null over pooled-sample random splits. The second is **Jensen–Shannon divergence (JSD)**, computed from KDEs on a common grid. These auxiliary measures are intended to capture aspects of mismatch not fully summarized by KS alone [2606.06622].

Sequence-valued tasks require a scalar reduction. For shuffling, each permutation \(\pi \in S_n\) is mapped via its **Lehmer code** \(L(\pi)\), each coordinate is normalized to \(Z_i = L_i/(n-i)\), and the benchmark scores only the first normalized coordinate \(Z_1\). The rationale given is that \(Z_1\) is well-spread under uniform permutations and therefore serves as a stringent one-dimensional proxy, although the evaluation remains primarily univariate [2606.06622].

The experimental protocol uses **\(N = 100\)** samples per task by default, **temperature \(T = 1.0\)** by default, and **reasoning off by default**. A dedicated reasoning setting, denoted **“xhigh,”** allows up to **4096 tokens** for chain-of-thought. Prompts request exactly one value, or a list in list-prompting ablations, in a structured format to simplify parsing. Outputs are parsed using a strict pattern; when parsing fails, a **GPT-4o-mini extractor** is used, with **up to 5 retries per call**, and invalid outputs are discarded and resampled. Infrastructure is primarily via **OpenRouter**, with some local runs on an **RTX 3090, 24GB VRAM**, and the reported total API cost is **approximately \$300** [2606.06622].

## 4. Empirical findings on calibrated stochastic generation

The headline empirical result is that **no evaluated model exceeds 40% at KS@100**. The strongest reported system, **Nemotron-3 Super 120B**, attains approximately **32.6–35.4%** depending on run or setting. Many models lie between **0–20%**, with examples including **GPT-4o at about 24%**, **DeepSeek V3.2 at about 22%**, **GPT-5.4 at about 15%**, and **GPT-4o-mini at about 10%**. A small open-weight model, **Qwen-3.5-2B**, reaches roughly **17–18%**, while **Claude Sonnet 4.6** is reported at about **5%** and **OLMo-3 7B** at about **3%** [2606.06622].

Category-level behavior is uneven. **Shuffling** and **Code** are the most demanding categories. Some strong proprietary systems reportedly collapse to **0% on shuffling**, whereas other models maintain approximately **37%** there. **Real-World** tasks can be relatively high for some models—for example, **Llama-3.2-1B instruct** is reported near **59%**—but very low for others, including **Nemotron-3 Super 120B** at about **3.3%**. Explicit prompting generally helps, but several models do better on implicit tasks, which the benchmark interprets as evidence that explicit distribution names can overanchor generation toward memorized prototypes rather than the specified parameterization [2606.06622].

Difficulty also varies sharply by target family. At KS@100, the easiest distributions are **Bernoulli** at about **43%**, **Categorical** at about **35%**, and **Discrete Uniform** at about **17%**. The hardest include **Fréchet** and **Dirichlet** at about **1.7%**, **Negative Binomial** at about **5.2%**, **Negative Multinomial** at about **6.1%**, and **Compound Poisson, Erlang, Inverse Gaussian, and Pareto**, all below **9%**. Heavy-tailed and multivariate targets are therefore especially problematic [2606.06622].

The metric’s dependence on sample size is central to interpretation. All models score approximately **100% at \(N=1\)**, because a single draw is rarely rejectable, but performance declines monotonically as \(N\) grows, with the steepest drops occurring between **\(N=20\)** and **\(N=100\)**. A cited example is **Claude Sonnet 4.6**, which falls from about **98% at KS@2** to about **5% at KS@100**. This behavior underscores that apparent one-shot plausibility is not evidence of calibrated repeated sampling [2606.06622].

Several intervention studies refine this picture. **Reasoning** can improve final-answer KS modestly; for example, **Nemotron-3 Super 120B** gains about **+3.3 at KS@20**. Yet numbers extracted from reasoning traces can be dramatically worse, including a reported **−33.2 at KS@20** for the same model, which the benchmark interprets as evidence that deliberation often revisits a narrow candidate set rather than sampling the support broadly. **Temperature** helps many strong models: Nemotron-3 Super 120B peaks near **\(T=1.2\)** with average **KS@100 ≈ 39.6 across Text/Code**, whereas **\(T=0.1\)** is severely harmful. **List prompting** can also help substantially, with reported gains such as **+17.1** for Nemotron-3 Super 120B, **+14.3** for Ministral-3 3B, and **+17.6** for OLMo-3 7B when asking for lists of ten values per call [2606.06622].

The benchmark identifies several recurring failure modes: **mode collapse**, **tail truncation or miscalibration**, **discrete miscalibration**, and **shuffling non-uniformity**. It further reports evidence from logits indicating that, for some systems, collapse is already present in the model’s answer distribution before decoding, suggesting that decoding is not the sole source of the problem. Cross-benchmark analyses show **KS@100** correlating positively with **CREATE** and **NoveltyBench Utility10**, but negatively with **Distinct10**, reinforcing the benchmark’s claim that distributional fidelity is different from lexical variety [2606.06622].

## 5. Antecedent work on capability predictability

The 2023 BIG-bench study investigates a different problem: whether one can predict normalized LLM performance on a BIG-bench subtask from an experimental configuration \(x = (l, nparam, t, nshot)\), where \(l\) is model family, \(nparam\) is size, \(t\) is task identifier, and \(nshot \in \{0,1,2,3,5\}\). The target \(y \in [0,1]\) is normalized performance on the task’s preferred metric—**exact\_str\_match**, **multiple\_choice\_grade**, or **rougeLsum**—scaled to \([0,1]\). The dataset contains **56,143 experiment records**, filtered from BIG-bench to retain **313 subtasks across 134 tasks**, **6 model families**, and tasks with preferred metrics in the allowed set; programmatic tasks, all-zero tasks, and subtasks with fewer than 100 examples are removed [2305.14947].

A range of predictors is compared, including matrix completion baselines, kNN variants, tree ensembles, and a **multi-layer perceptron (MLP)**. For trees and the MLP, the feature set includes one-hots for model family, model identity, task, and preferred metric, plus continuous size features such as total parameters, non-embedding parameters, FLOP-matched non-embedding parameters, their natural logs, and \(nshot\). The MLP is trained with **MSE**, with **dropout**, **weight decay**, and **early stopping**, and its first-layer task vectors serve as learned task representations [2305.14947].

On a **random 10-fold CV** split over \((l, nparam, t, nshot)\) tuples, the MLP reaches **RMSE ≈ 0.0500** and **\(R^2 ≈ 0.9508\)**, while **XGBoost** is very similar at **\(R^2 ≈ 0.9510\)**. Harder generalization protocols reduce performance but still yield high \(R^2\): **L2.1 ≈ 0.9251**, **L2.2 ≈ 0.9391**, **L3 ≈ 0.8922**, and **L3 Composition ≈ 0.8597**. The paper characterizes this as evidence of “highly learnable structure” in the historical records under interpolation, but it also finds that extrapolation is much harder: holding out **GPT-3 200B** yields \(R^2\) between about **0.70** and **0.88** depending on leakage setting, while **PaLM 535B** ranges from about **0.15** to **0.68** [2305.14947].

Predictability is not uniform across tasks. Highly predictable tasks include **modified_arithmetic: three_digit_addition_control**, **two_digit_multiplication_control**, **linguistic_mappings: sentence_negation_json**, and **qa_wikidata**. Less predictable tasks include **linguistics_puzzles**, **checkmate_in_one**, **gender_inclusive_sentences_german**, **paragraph_segmentation**, and **authorship_verification:swapped**. The diagnostics attribute low predictability to surprising or non-smooth scaling curves, overestimation, and emergent-like jumps. The study also finds that **zero-shot** transitions are harder, **larger models** are intrinsically harder to predict, and sparse families such as **Gopher** benefit strongly from multi-group learning [2305.14947].

## 6. UnpredictaBench as a capability-predictability design blueprint

Building on those BIG-bench results, the same source material outlines an UnpredictaBench-style benchmark for measuring where capability prediction fails. The proposed core quantities are **per-task MSE and \(R^2\)** under specified splits, a per-task unpredictability score

$$
U_{task} = 1 - R^2_{task},
$$

with clipping to \([0,1]\) when \(R^2 < 0\), **family-transfer \(R^2\)** obtained by holding out model families, **shot-instability** defined through

$$
V_{shot}(l,t)=\mathrm{Var}\{y(l,nparam,t,nshot)\},
$$

and an aggregate instability index

$$
I_{shot}(t)=E_{l,nparam}[V_{shot}(l,t)].
$$

It also proposes **size-transfer RMSE/\(R^2\)** and cross-task generalization metrics based on **L3** and **L3 Composition** splits [2305.14947].

The benchmark-composition principles are similarly explicit. Candidate tasks should include those with **high residuals or low \(R^2\)**, those showing **non-smooth scaling** or frequent overestimation “false positives,” those that fail in **family-transfer** or **size-transfer**, and those with high **\(I_{shot}\)**, especially strong **0→1-shot discontinuities**. To avoid redundancy, the design uses **MLP-derived task embeddings** \(e_t\), clusters them with **\(k\)-means** in Euclidean space, and chooses tasks near cluster centroids. A value-aware variant filters candidates using a global **task value** \(v_t\), computed from Best-of-5000 search logs. The blueprint also recommends balancing **emergent** and **non-emergent** tasks and offering **budget-aware tiers** at \(b \in \{8,16,24,32,42\}\) [2305.14947].

A key empirical result supporting this design is the **“small-bench”** search. Using nested CV over model families, the work shows that **BIG-bench Lite** and **BIG-bench Hard (BBH)** are sub-optimal for the objective of recovering full-benchmark performance. A **Best-of-5000** subset with **\(b=8\)** is reported to be **as informative as BBH with \(b=24\)**, while random **16-task** subsets can also rival BBH. **\(k\)-means** subsets are often competitive with or better than search-based subsets at equal budget, and **\(k\)-means + Task Value** improves further, supporting the view that diversity and global utility jointly matter for compact benchmark design [2305.14947].

This blueprint therefore treats unpredictability as a property of **generalization gaps** rather than raw benchmark difficulty. A plausible implication is that, under this conception, UnpredictaBench is less a conventional leaderboard and more a stress test for whether performance-prediction methods survive transfer across families, scales, tasks, and shot regimes [2305.14947].

## 7. Related frameworks, limitations, and significance

UnpredictaBench sits near, but is not identical to, **PredictaBoard**, a 2025 benchmark for **instance-level score predictability**. PredictaBoard defines an assessor \(a:\mathcal{X}\to[0,1]\) that predicts the Bernoulli success variable \(v_i \in \{0,1\}\) for a prompt \(x_i\), and evaluates selective prediction through rejection thresholds, **coverage**, **error on the accepted set**, **AUARC**, and the **predictably valid region**

$$
PVR(\varepsilon)=\max_{\tau \in [0,1]}\{C(\tau): Err(\tau)\le \varepsilon\}.
$$

It also reports **AUROC**, **Brier score**, and **Winkler’s score**, and studies in-distribution performance on **MMLU-Pro** with out-of-distribution evaluation on **BBH**. The paper states that it does not explicitly cite UnpredictaBench, but conceptually positions its framework as complementary to an UnpredictaBench-style agenda centered on unpredictability and safety-oriented error anticipation [2502.14445].

Both versions of UnpredictaBench come with important limitations. The 2026 benchmark is **primarily univariate** even for multivariate targets, because sequence-valued or joint outputs are reduced to scalar proxies; the **KS test** is more sensitive near the median than in the tails; the **Lehmer \(Z_1\)** mapping does not fully characterize permutation distributions; prompts are **English-only** and code tasks are **Python-only**; and prompt style may reflect bias from GPT-generated templates [2606.06622]. The capability-predictability blueprint inherits limitations from BIG-bench composition, sparse metadata about pretraining or prompting, the small number of model families \((|L|=6)\), potential overfitting under random splits, and possible mismatch between search objectives and final evaluation protocols [2305.14947].

Despite these constraints, the significance of the term across both usages is clear. In the distributional benchmark, the central claim is that **calibrated stochastic generation is a distinct capability from plausibility and diversity**, and that failure on simple repeated-sample tests already casts doubt on using LLMs as stand-ins for stochastic systems [2606.06622]. In the capability-prediction lineage, the central claim is that **average predictability under interpolation can coexist with sharp failures under transfer or extrapolation**, making unpredictability itself a benchmarkable object rather than a residual nuisance [2305.14947]. Together with assessor-based frameworks such as PredictaBoard, this literature shifts evaluation from raw task performance toward a broader question: not only what LLMs can do, but where their behavior can be forecast, trusted, or safely rejected [2502.14445].

Source: https://www.emergentmind.com/topics/unpredictabench