The Illusion of Stochasticity in LLMs
Abstract: In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for LLMs operating as agents. Agentic systems are frequently required to sample from distributions, often inferred from observed data, a process which needs to be emulated by the LLM. This leads to a distinct failure point: while standard RL agents rely on external sampling mechanisms, LLMs fail to map their internal probability estimates to their stochastic outputs. Through rigorous empirical analysis across multiple model families, model sizes, prompting styles, and distributions, we demonstrate the extent of this failure. Crucially, we show that while powerful frontier models can convert provided random seeds to target distributions, their ability to sample directly from specific distributions is fundamentally flawed.
Summary
- The paper exposes a fundamental flaw: LLMs do not reliably sample from prescribed probability distributions despite understanding the underlying policy.
- It employs rigorous statistical tests such as Chi-Square and Kolmogorov-Smirnov, uncovering biases like over-selection of specific numbers and positional preferences.
- LLMs perform well in deterministic distribution conversion from uniform inputs but remain unreliable for autonomous stochastic sampling tasks.
The Illusion of Stochasticity in LLMs: A Critical Analysis
Introduction and Motivation
"The Illusion of Stochasticity in LLMs" (2604.06543) rigorously investigates the capability of modern LLMs to sample reliably from specified probability distributions, a core requirement for the deployment of agentic LLMs in reinforcement learning (RL) and decision-making contexts. While earlier literature has reported knowing-doing gaps and limited exploration in LLM-driven agents, this work isolates and defines a fundamental flaw: the inability of LLMs to align their stochastic outputs with target action distributions, even when the models demonstrate an explicit understanding of the policy.
The core claim is contradictory to widespread assumptions—even powerful, frontier LLMs are empirically and statistically biased when tasked with direct stochastic sampling, failing to produce outputs that conform to canonical distributions (uniform, Gaussian, etc.) across multiple families and prompting strategies.
Empirical Failures in Sampling Distributions
The study employs a comprehensive empirical protocol. LLMs (e.g., Gemini, Qwen3, OLMO) are prompted to sample from uniform discrete, uniform continuous, and Gaussian distributions using both natural language and mathematical prompts, with each output parsed for statistical analysis.
Results demonstrate severe deviation from the prescribed target distributions. Statistical tests (Chi-Square and Kolmogorov-Smirnov) yield p-values << 0.05 for all models, indicating consistent empirical failure. Salient biases appear: over-selection of numbers such as 7, positional preference in response options, and a tendency toward median values in continuous domains.
Figure 1: LLM-generated empirical frequencies for target distributions, highlighting strong deviations from uniformity and Gaussianity.
Altering decoding hyperparameters (temperature, top-k, top-p) across a broad grid does not correct stochastic misalignment; biases persist regardless of sampling temperature or prompt verbosity.
Figure 2: Comparative empirical frequency plots across temperatures, confirming failure modes are robust to output stochasticity parameters.
The failure mode remains under both zero-shot and chain-of-thought (CoT) reasoning. In certain configurations, disabling reasoning amplifies the problem (e.g., certain models return "42" as the only sample), indicating that overthinking is not the primary driver of distributional failure.
Sequential, Batched, and Tool-Based Sampling
Experiments extend to sequential and batched sampling regimes. While sequential sampling with full history context can slightly attenuate distributional failures for low-entropy distributions, this benefit is not universally robust and brings undesirable side effects such as negative short-range autocorrelation ("repulsion") or pronounced periodicity in the output sequence.
Figure 3: Auto-correlation analysis for various sampling protocols, revealing non-trivial temporal dependencies and repulsive artifacts.
Batched generation further reveals counting failures and periodic patterns, with certain sample indices consistently misaligned with the target distribution. Across all modifications, failures persist for Gaussian targets and advanced distributions, and temporal dependencies arise that undermine independent stochasticity.
Experiments leveraging tool-use (Python code execution with PRNG calls) circumvent native LLM sampling limitations, but stateful randomness remains fragile: unless proper seeding (e.g., with system time) is enforced and tool code execution is correctly integrated, deterministic output and bias can re-emerge. Instruction-following variability across models impacts the reliability of this workaround.
Figure 4: Empirical frequency plots for tool-generated samples with and without proper random seed initialization.
Simulated PRNGs and Distribution Conversion
The authors also explore using LLMs to simulate PRNG algorithms when provided with code and explicit seeds. Sufficiently large models (≥4B parameters) can simulate uniform PRNGs with high accuracy, and the simulated outputs strongly adhere to the target distributions for discrete/continuous uniform targets. However, simulation becomes unreliable for more complex transformations (e.g., Box-Muller for Gaussian sampling), owing to LLM arithmetic limitations, especially in multiplying large numbers and maintaining PRNG state.
A major positive finding is that LLMs excel at deterministic distribution conversion—when provided independently sampled uniform random numbers (from [0, 1]), models can reliably execute the mapping to a wide range of target distributions (discrete, Gaussian, mixture models) via prompt-driven reasoning.
Figure 5: Examples of distribution conversion from uniform [0, 1] to discrete, Gaussian, non-uniform, and mixture models, confirming algorithmic competence with deterministic inputs.
This reliability is emergent, requiring sufficient scale and reasoning capacity. Failures become evident as transformation complexity increases or model scale decreases below threshold.
Comparative Results in Other Architectures
Extending experiments to other model families, such as OLMO, shows that stochasticity failures are widespread and not limited to a specific transformer variant, confirming the generality of the phenomenon.
Figure 6: Empirical distributions for OLMO-3 models, revealing sampling biases consistent with Gemini/Qwen3.
Implications for Agentic Modeling and RL
The findings have direct, significant implications for RL and agentic modeling with LLMs:
- Agentic Stochasticity: For tasks requiring genuinely random behavior from a specified policy (e.g., exploration, policy regularization, bandit algorithms), direct LLM-based sampling is fundamentally flawed; models will not achieve required entropy or policy unpredictability.
- Predictability and Security: Biases in output (e.g., MCQ placing the answer in position “C” disproportionately) render LLM-based policies predictable and exploitable in adversarial settings.
- Cost of Simulated Stochasticity: Simulating PRNGs in CoT or via deterministic mapping is computationally intensive and impractical for high-throughput, sequential sampling—further exacerbating inference cost for RL applications.
Theoretical Implications and Future Directions
The observed failures highlight intrinsic limitations of the autoregressive, stateless transformer paradigm when tasked with controlled stochasticity. These failures persist despite instruction-based prompting, model scale, and architectural variety.
Theoretical avenues for improving LLM stochasticity include:
- Stateful Tool Integration: Embedding access to external, stateful samplers within LLM tool-calling frameworks, with careful handling of seed initialization and state persistence.
- Specialized Training Objectives: Tailored augmentation or architectural modification (as considered in [von Oswald et al., 2024], [Gillman et al., 2024]) may instill robust internal randomness mechanisms, potentially via supervised or RL-based meta-learning to align output statistics with prescribed distributions.
- External Sampler Reliance: For agentic deployment, modular external sampling—decoupled from the transformer inference path—remains the most reliable and scalable method for generating controlled stochasticity.
Conclusion
This work reveals a critical and previously underexamined deficiency in current LLMs: the illusion—not reality—of stochastic sampling. While frontier models are capable of deterministic, algorithmic transformation of supplied random inputs, their direct sampling outputs are fundamentally biased and unsuitable for tasks requiring precise, controlled randomness. Addressing this shortcoming is necessary for advancing LLM-based agentic systems and aligning policy execution with theoretical and practical requirements in decision-making under uncertainty. Future progress demands architectural innovations or hybrid system designs that embed true stochasticity into the generative machinery of language-driven agents.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
This paper asks a simple but important question: can LLMs be truly “random” when they need to be? When LLMs act like agents (for example, playing games, planning, or generating quizzes), they sometimes must make choices at random but with specific probabilities—like flipping a fair coin or rolling a weighted die. The authors show that today’s LLMs often fail at this. They may look random on the surface, but their choices are biased in hidden ways—the “illusion of stochasticity” (stochasticity means randomness).
What questions did the researchers ask?
The team focused on a few easy-to-understand questions:
- If you ask an LLM to pick a random item (like a number 0–9) with equal chance, does it really do that?
- If you ask it to sample from a continuous distribution (like picking a decimal number between 0 and 1) or from a bell-shaped curve (a Gaussian), does the pattern of its picks match what it should?
- Do common settings (like temperature or top‑k/top‑p) or reasoning styles (long “chain-of-thought” vs. short answers) fix the problem?
- Can LLMs do better if:
- They keep track of history while sampling?
- They generate many samples in one go (batched sampling)?
- They use tools (like running Python code with a real pseudo‑random number generator, PRNG)?
- They are given a true random number and asked to convert it into the right distribution using a deterministic rule?
How did they test it?
Here’s the basic approach, explained with everyday analogies:
- Think of “sampling from a distribution” like drawing from a hat with known chances:
- Uniform discrete: every item (say numbers 0–9) should be equally likely.
- Uniform continuous: every decimal between two numbers (like 0 and 1) should be equally likely.
- Gaussian (bell curve): numbers near the center (the mean) should be more common, and far-away numbers rarer.
- The researchers:
- Asked various LLMs (from different families and sizes) to “pick one random sample” many times (typically 1,024 times) and collected the results.
- Compared the pattern of picks to what a correct random process should produce, using standard statistical checks (called goodness‑of‑fit tests). Think of these tests as a fairness checker: if the p‑value is bigger than 0.05, the picks look like they came from the right hat; if it’s much smaller, the picks are clearly biased.
- They also tried:
- Changing decoding parameters (temperature/top‑k/top‑p), which affect how “creative” the model’s next word is.
- Turning chain‑of‑thought on and off.
- Sequential sampling (feeding earlier picks back into the prompt) and batched sampling (asking for many picks at once).
- Tool use: letting the model write Python code and run a real PRNG.
- PRNG simulation: giving the model the code and a seed and asking it to “mentally” run the algorithm step by step.
- Distribution conversion: giving the model a true random number in [0,1] and asking it to convert that number into a sample from a target distribution (this is a deterministic procedure, like dividing the [0,1] line into buckets for a uniform list, or using an “inverse CDF” for a Gaussian).
What did they find, and why does it matter?
Here are the main results, explained with examples and why they’re important:
- LLMs often pick certain values too often, even when they should be uniform.
- Example: When asked to place the correct answer in a multiple-choice quiz at random among A, B, C, D, models showed a strong bias toward “C.” That makes quizzes predictable and easy to game.
- When choosing numbers 0–9 “at random,” some numbers like 7 or 42 appear more often than they should.
- The bias shows up across many models, sizes, and prompts—and it’s not fixed by common settings.
- Changing temperature/top‑k/top‑p didn’t make the samples truly match the target distributions.
- Turning chain‑of‑thought off didn’t solve it; sometimes it made biases worse.
- Statistical tests gave very small p‑values (much smaller than 0.05), meaning the outputs don’t match the desired distributions.
- Positional and prompt-order bias is real.
- If the options are listed as {left, right, up, down, fire} versus {up, down, left, right, fire}, the model’s “random” choices change noticeably. This means wording and order in the prompt can skew results.
- Sequential and batched sampling introduce new problems.
- Feeding all previous picks back into the prompt sometimes made uniform sampling more accurate for simple tasks, but it introduced “temporal biases,” like avoiding repeating the same number (repulsion) or flipping between certain values in a pattern.
- Asking for many samples in one answer led to counting mistakes and periodic patterns (like repeating every 10th value), which is not how real randomness behaves.
- Both approaches are slow or impractical for real-time agents.
- Using real PRNGs as tools can work—if done carefully.
- If the model writes code that uses a PRNG but forgets to set a proper seed or uses the same seed each time, outputs can become accidentally deterministic.
- When explicitly told to seed with the current time, stronger models followed instructions and produced good randomness; weaker models were less reliable.
- Even then, parallel requests can collide if they get the same time-based seed.
- LLMs can simulate a PRNG step-by-step with a provided seed and code, but this has limits.
- For uniform sampling, larger models reached high accuracy at imitating the code’s output.
- For Gaussian sampling (which needs a more complex transformation), models struggled with large intermediate calculation steps and lost accuracy.
- This approach is slow and relies on carrying state across calls, which typical LLM deployments don’t do.
- LLMs are surprisingly good at “distribution conversion” when given a true random number.
- If you provide a uniform [0,1] number and ask the model to convert it to a target distribution using a deterministic rule, larger models can do this well—even for some non‑uniform distributions or simple Gaussian mixtures.
- This skill seems to “emerge” with bigger models: small ones fail, mid/large ones succeed.
- But as the transformation gets more complex, errors reappear.
Why it matters: Many agent behaviors need controlled randomness. Without it, agents don’t explore properly, can be exploited by opponents, or accidentally “cheat” fairness (for example, always favoring certain options). The paper shows that what looks like randomness in LLM outputs is often just noise from their token predictions—not true, controlled randomness tied to a precise probability.
Simple takeaways and impact
- If you need real randomness with specific probabilities, don’t rely on an LLM’s own token sampling. It often produces biased, predictable patterns.
- Changing decoding settings or adding chain‑of‑thought doesn’t reliably fix this.
- Better approaches:
- Use a proper, stateful random number tool (a PRNG) and manage seeds carefully.
- If you already have a true random number, ask the model to convert it to your target distribution deterministically—larger models can handle many such conversions.
- For “agentic” systems that must make many random decisions over time, the safest path is to plug in an external, stateful sampler the model can call, rather than trying to “fake” randomness inside the LLM.
In short, today’s LLMs can understand probability, and they can even follow recipes to turn one kind of random number into another. But generating their own truly random, correctly distributed choices on demand—without help—is something they still don’t do reliably. This matters for building fair, robust, and trustworthy AI agents.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a focused list of what remains missing, uncertain, or unexplored in the paper, phrased to enable concrete follow-up work:
- Scope across models and training regimes:
- Do the observed failures generalize to more model families (e.g., GPT-4/4.1/4o, Llama variants, Claude, Mistral) and non‑Transformer or hybrid architectures?
- How do RLHF, safety-tuning, instruction-tuning, and post-training methods affect stochastic sampling reliability?
- Do specialized “reasoning” models (o1-like, long-context, chain-of-thought-optimized) differ materially?
- Multilingual and tokenization effects:
- How do biases vary across languages, scripts, and tokenization schemes (e.g., byte-level BPE vs. SentencePiece)?
- Does number tokenization (e.g., digit-wise vs. whole-number tokens) systematically skew numeric sampling?
- Distributional coverage and complexity:
- Performance on multivariate, correlated, and constrained distributions (e.g., Dirichlet, categorical with large support, multivariate Gaussians with nontrivial covariance, truncated/heavy-tail distributions) remains untested.
- Sampling over combinatorial or structured objects (sets, permutations, paths), or policies with continuous-action vectors, is not evaluated.
- Prompting and decoding breadth:
- The study only varies temperature/top‑p/top‑k; what about other decoders (e.g., Mirostat, typical decoding, entmax sampling, contrastive search) and repetition/penalty settings?
- Impact of alternative prompting strategies (schema-enforced function calling, JSON/formal grammars, explicit numeric format constraints, few-shot demonstrations with ground-truth RNG exemplars) is not systematically studied.
- Robustness to adversarial or innocuous prompt perturbations (ordering, padding, distractor text, varied separators/whitespace) is not quantified.
- Mechanistic understanding of bias:
- What internal representations and layers drive positional, semantic (e.g., affinity for 7/42), and temporal biases?
- To what extent do training-data frequencies vs. decoding dynamics vs. tokenization account for the observed deviations?
- Why do sequential and batched generations show repulsion and periodicity, respectively? Are these tied to attention cache behavior, positional encodings, or implicit repetition penalties?
- Harnessing native entropy:
- Can the model’s own token-level randomness be turned into unbiased bits (e.g., via von Neumann extractors or hash-based debiasing) and then mapped to target distributions? This pathway is not explored.
- Alignment of token-level and action-level sampling:
- Methods to enforce correct action probabilities despite tokenization (e.g., constrained decoding over an action grammar, logit routing/aggregation across synonymous surface forms, calibrated semantic heads) are not evaluated.
- Downstream, task-level consequences:
- Quantitative impact on exploration, regret, and exploitability in bandits/MDPs or real agent benchmarks is not measured.
- How predictable/exploitable are the biases by adversaries (e.g., in exams, markets, security settings), and what are practical risk bounds?
- Tool-use and PRNG rigor:
- Cross-runtime variability (Python
random, NumPy, JavaScript, Rust, C++), default seeding policies, and deterministic containerized execution effects remain under-characterized. - Concurrency and collision risks when time-based seeds are used (especially under high parallelism) are not analyzed.
- No standardized protocol is proposed for securely provisioning seeds, avoiding state loss between calls, and ensuring reproducibility/auditability.
- Cross-runtime variability (Python
- PRNG simulation limits:
- Complexity thresholds (code length, numerical range, precision demands) at which models fail to simulate PRNGs are not mapped.
- Alternatives to Box–Muller (e.g., Ziggurat, acceptance–rejection, inverse CDF with rational approximations) and fixed-point arithmetic to alleviate large-number errors are not tested.
- Stateful sampler integration:
- The proposed “stateful tool” concept lacks empirical validation: How to design APIs, manage state across stateless inference calls, handle multi-tenant access, ensure privacy, control versioning, and bound latency?
- Training-time interventions:
- No experiments on training objectives that encourage reliable sampling (e.g., auxiliary losses for distribution matching, dedicated RNG head, noise-conditional heads, Fourier/randomness heads, or curriculum with sampling tasks).
- Effectiveness and data requirements of fine-tuning to mitigate biases are unknown.
- Scale and statistical robustness:
- Results are primarily with N=1024; how do deviations scale with much larger N, across multiple independent runs and seeds? Provide confidence intervals, effect sizes, and variance decomposition.
- Parsing and formatting reliability:
- Systematic measurement of parsing error rates, out-of-range values, and numeric-format variability (under different temperatures and prompts) is missing; mitigation via strict schemas or constrained decoding is not evaluated.
- Latency and compute trade-offs:
- Precise cost/latency profiles for PRNG tool-use, PRNG simulation via chain-of-thought, and distribution conversion are not quantified; no budget-based recommendations are provided.
- Multimodal and embodied settings:
- Whether similar stochasticity failures occur in VLMs and embodied agents (e.g., sampling multimodal actions, camera-based policies) is untested.
- Model update drift and reproducibility:
- Stability of biases across model updates, providers, and deployments is not studied; guidelines for monitoring and regression testing are absent.
- Theoretical foundations:
- Formal analysis explaining why next-token likelihood training fails to produce calibrated stochastic behavior over semantic/action spaces is not provided; conditions under which reliable sampling is learnable remain open.
Practical Applications
Immediate Applications
Below are concrete ways to apply the paper’s findings now, with suggested sectors, tooling, and feasibility notes.
- External RNG for agent actions
- Sectors: robotics, gaming, RL, software agents, A/B testing
- What to do: Route all action sampling through an external pseudo-random number generator (PRNG) or randomness microservice. Have the LLM output the action probabilities as structured data; sample the action externally; feed the chosen action back to the agent.
- Tools/workflows: Add a “randomness” microservice with API (seed control, independent streams), logging, and audit trails; update agent orchestrators to separate “policy inference” (LLM) from “action sampling” (tool).
- Assumptions/dependencies: Agent framework must support tool-calling and structured outputs; probability estimates from LLM must be well-formed; seed management must avoid collisions in parallel runs.
- Seed-in, sample-out prompting pattern (deterministic conversion)
- Sectors: software, education content generation, simulations
- What to do: Provide a uniform random number from a trusted RNG in the prompt and instruct the LLM to convert it to the target distribution (bucketization for categorical; inverse-CDF for Gaussian).
- Tools/workflows: Wrapper that injects
u ~ U[0,1]into prompts; post-hoc goodness-of-fit (GoF) checks. - Assumptions/dependencies: Works reliably only on sufficiently capable models (≥~4B for Gaussians per paper); conversion may fail for complex transforms; still requires external RNG.
- Fix MCQ/randomized content pipelines
- Sectors: education, hiring/HR assessments, certification platforms
- What to do: Do not ask LLMs to “randomize” answer order or correct-option placement; shuffle options with external RNG either before prompting or as a post-process step.
- Tools/workflows: Assessment templating with external shuffle; logging of positions; periodic audits for uniformity.
- Assumptions/dependencies: Integration with authoring tools; downstream systems must respect the shuffled order.
- Online validation of “randomness” claims
- Sectors: MLOps across industries
- What to do: Instrument production systems with GoF tests (Chi-square, KS), autocorrelation functions (ACF), and transition-matrix checks for any LLM output that is supposed to be stochastic.
- Tools/workflows: Monitoring dashboards; batch collectors to reach sample sizes (e.g., N≈1k); alerting thresholds on p-values and ACF patterns.
- Assumptions/dependencies: Sufficient sample sizes; small additional compute/storage overhead.
- Code-tool RNG with explicit seeding
- Sectors: software engineering agents, data science assistance, analytics bots
- What to do: Permit tool-use to call language-native RNGs but force explicit, robust seeding (e.g., OS-level CSPRNG, per-request UUID-derived seeds). Disallow implicit seeds or repeated deterministic seeds.
- Tools/workflows: Static analysis or lint rules that check generated code includes seed lines; execution sandbox that exposes high-quality RNG; seed ledger.
- Assumptions/dependencies: Model’s instruction-following reliability; concurrency can cause near-identical time seeds—prefer independent streams or counter-based RNGs.
- Replace LLM-based Monte Carlo draws
- Sectors: finance (pricing, risk), energy (forecasting), logistics (simulation), research
- What to do: Keep LLMs for modeling/logic/explanation; perform all sampling via numerical libraries/services; feed draw results back to the LLM if needed.
- Tools/workflows: Python/Rust/C++ RNG libraries; vectorized samplers; secure seeding.
- Assumptions/dependencies: Clean interfaces between reasoning and sampling; performance considerations for large simulations.
- Prompt-order randomization as a guardrail
- Sectors: general product interfaces, chatbots, UI experiments
- What to do: When asking LLMs to “pick uniformly from a set,” first randomly permute the set externally to mitigate positional biases; or avoid asking the LLM to pick at all.
- Tools/workflows: Pre-prompt shufflers; schema that separates set from selection.
- Assumptions/dependencies: External RNG availability; does not fix the core sampling issue, only reduces exploitable patterns.
- PRNG state management for multi-step agents
- Sectors: conversational agents, games, tutoring systems
- What to do: Maintain per-session RNG state outside the LLM; expose a “draw” tool that returns samples and updates state.
- Tools/workflows: Session-scoped RNG streams; reproducibility via logged seeds and stream positions.
- Assumptions/dependencies: Orchestrator needs session state; privacy and persistence policies.
- Research/benchmark adoption
- Sectors: academia, model evaluation labs
- What to do: Adopt the paper’s diagnostics (GoF, ACF, transition matrices) and prompt templates to evaluate sampling reliability across models and decoding settings.
- Tools/workflows: Open-source test harness; CI for model eval.
- Assumptions/dependencies: Access to multiple model families/sizes; consistent decoding setups.
- Governance and internal policy updates
- Sectors: education, healthcare operations, HR, gambling/compliance
- What to do: Require external, auditable RNG for any randomized decision or content; ban LLM-only “randomization.” Document RNG method and validation.
- Tools/workflows: Policy checklists; audit logs; periodic fairness tests.
- Assumptions/dependencies: Organizational buy-in; legal/compliance oversight.
Long-Term Applications
These require additional research, engineering, or ecosystem standardization before deployment.
- Native stochasticity controls in LLMs
- Sectors: foundation model providers, robotics, RL platforms
- What: Architectures or heads that can faithfully map policies to samples (e.g., learned inverse-CDF heads, “Fourier head” approaches). Train with objectives that test calibrated sampling.
- Tools/products: “Stochastic compliance” benchmarks and training curricula.
- Dependencies: New training recipes; access to internals of models; compute cost.
- Seeded-decoding and distribution APIs
- Sectors: model serving, inference platforms
- What: A standard API allowing clients to provide a seed and a target distribution descriptor; the model returns a deterministic sample and proof-of-fit metrics.
- Tools/products: Inference servers supporting seeded conditional sampling; SDKs that formalize distribution specs (categorical, Gaussian, mixtures).
- Dependencies: Model retraining or auxiliary modules; ecosystem standardization.
- Structured action heads and decoding
- Sectors: robotics, games, autonomous agents
- What: Separate token generation from action selection via structured “action heads” producing probability vectors; sampling handled by a tightly integrated, verifiable sampler.
- Tools/products: Agent frameworks with native action interfaces (not text-only); schema validators.
- Dependencies: Changes in agent design; compatibility with legacy LLM APIs.
- Statefulness as a first-class tool paradigm
- Sectors: agent platforms, enterprise automation
- What: Standardize stateful tools (RNGs, counters, unique ID allocators, session memories) accessible to agents with guaranteed persistence and audit.
- Tools/products: Tool registries with state management; permissioning and auditing layers.
- Dependencies: Security models; multi-tenant isolation; governance.
- Improved numerical/algorithmic reasoning in LLMs
- Sectors: model R&D, scientific computing
- What: Enhance models’ ability to simulate algorithms (e.g., Box–Muller) and do stable arithmetic at scale; possibly via tool-augmented training or differentiable tool calling.
- Tools/products: Benchmarks on algorithm emulation under numerical stress; curriculum emphasizing precision.
- Dependencies: Training data, objectives, and evaluation suites; may increase model size or latency.
- Regulatory standards for AI-driven randomization
- Sectors: education testing, clinical trials, gambling, finance
- What: External standards mandating documented RNG methods, seeding, independence checks, and periodic GoF audits when AI systems randomize.
- Tools/products: Certification programs (“Randomness QA”); audit tooling.
- Dependencies: Cross-industry consensus; regulators’ adoption.
- Unified “Randomness QA” services
- Sectors: enterprise AI platforms, MLOps vendors
- What: SaaS tools that inject, test, and certify randomness quality in AI workflows (distribution fit, temporal independence, reproducibility controls).
- Tools/products: Dashboards, APIs, compliance reports, attack simulations for exploitability.
- Dependencies: Integration with diverse pipelines; data retention policies.
- Token-to-action mapping research
- Sectors: robotics, simulation, interactive systems
- What: New tokenization/decoding strategies that ensure semantic actions map unambiguously to sampling steps (reducing the gap between token entropy and action distributions).
- Tools/products: Action tokenizers; constrained decoders; latent action spaces.
- Dependencies: Research validation in real environments; changes to pretraining/finetuning.
- Learned distribution conversion modules
- Sectors: general AI, content generation
- What: Train compact differentiable modules that convert external seeds to complex target distributions reliably (beyond Gaussian/mixtures), callable by LLMs.
- Tools/products: Plug-in libraries with proofs of calibration; fallbacks and self-checks.
- Dependencies: Training data covering diverse distributions; verification methods.
- Security-grade randomness for high-stakes AI
- Sectors: security, finance, healthcare
- What: Integrate cryptographically secure RNGs (CSPRNG) and provable independence into AI stacks, including key management for seeds and tamper-evident logs.
- Tools/products: HSM-backed seed sources; verifiable randomness beacons; reproducible replay for audits.
- Dependencies: Security infra; compliance; latency budgets.
Notes on feasibility (global): Immediate solutions depend on tool-calling, reliable seeding, and monitoring; they trade minimal development for strong improvements. Long-term solutions depend on model-level changes, new APIs, and standardization—requiring cooperation between model providers, platform vendors, and regulators.
Glossary
- Agentic systems: Architectures where models act as autonomous agents, making decisions and taking actions in environments. "Agentic systems are frequently required to sample from distributions"
- Auto-correlation function (ACF): A measure of correlation between elements of a sequence at different time lags, used to detect temporal dependence. "we visualize the auto-correlation function (ACF)"
- Batched sampling: Generating many samples in a single inference call rather than one per call. "with batched sampling, LLMs indeed can sometimes provide reasonably accurate empirical distribution estimates for uniform distributions"
- Box-Muller transform: A deterministic method to convert uniform random numbers into Gaussian-distributed samples. "apply the Box-Muller transform."
- Bucketization algorithm: A procedure that partitions the unit interval into equal subintervals to map a uniform random number to discrete outcomes. "For uniform discrete distribution, LLMs use the bucketization algorithm"
- Chain-of-thought: An explicit reasoning style where models produce intermediate steps before the final answer. "using chain-of-thought (i.e., without tool use)"
- Chi-Square test: A statistical goodness-of-fit test for discrete distributions comparing observed and expected frequencies. "for the uniform discrete distribution we conduct a Chi-Square test"
- Decoding parameters: Settings controlling how tokens are sampled during generation, affecting randomness and diversity. "depends on the choices of decoding parameters, such as temperature, top-p, top-k"
- Emergent properties: Capabilities that appear only beyond a certain model scale, not predictable from smaller sizes. "we observe clear emergent properties w.r.t. the model size."
- Entropy (of a distribution): A measure of uncertainty or randomness in a probability distribution. "when the distribution has entropy (i.e., it is not deterministic)."
- Frontier LLMs: The most capable, cutting-edge LLMs at the time of study. "we mainly explore the behaviour of frontier LLMs"
- Gaussian mixture model (GMM): A probabilistic model representing data as a mixture of multiple Gaussian components. "a two-component Gaussian mixture model (GMM)"
- Goodness-of-fit (GoF) tests: Statistical tests that assess how well empirical data match a specified distribution. "we conduct goodness-of-fit (GoF) tests."
- Inverse transform sampling: A method to generate samples from a target distribution by applying the inverse CDF to uniform samples. "they adopt a inverse transform sampling approach."
- Kolmogorov-Smirnov tests: Nonparametric tests comparing empirical and theoretical CDFs for continuous distributions. "for the continuous uniform and Gaussian distributions we conduct Kolmogorov-Smirnov tests."
- Knowing-doing gap: When a model can infer correct actions but fails to execute them accordingly. "a phenomenon sometimes referred as the knowing-doing gap"
- Logits: Pre-softmax scores output by a model that determine token probabilities. "between the logits of the next token and the semantic action required"
- Multi-armed bandit: A class of sequential decision problems balancing exploration and exploitation across actions (arms). "solve simple multi-armed bandit problems"
- Mutator operator: A mechanism in evolutionary algorithms that introduces variation by modifying candidate solutions. "used successfully as a mutator operator in evolutionary search"
- P-value: The probability, under a null hypothesis, of obtaining results at least as extreme as those observed. "We report p-values of these GoF tests"
- Positional bias: A systematic preference influenced by the order of options presented in the prompt. "In this case the bias seems to be more positional, based on the order of the values in the prompt."
- Pseudo-random number generator (PRNG): A deterministic algorithm that produces sequences of numbers approximating randomness given a seed. "Pseudo-random number generator (PRNG) algorithms are typically stateful"
- Random seed: An initial value that determines the sequence produced by a PRNG. "convert provided random seeds to target distributions"
- Repulsive bias: A tendency for consecutive samples to avoid the current state, inducing negative short-lag correlations. "it suffers from the repulsive bias (also observed by the ACF plot)"
- Sequential sampling: Producing samples one after another while conditioning on previously generated samples. "Sequential sampling with history"
- Stateful: Maintaining internal state across calls, affecting subsequent outputs. "PRNGs are stateful"
- Stateless: Not preserving state between calls; each inference is independent. "stateless independent inference calls."
- Stochastic policy: A policy that selects actions according to specified probabilities rather than deterministically. "act according to a stochastic policy"
- Temporal biases: Systematic dependencies over time in a sequence of outputs. "we can also investigate the temporal biases when sampling from LLMs."
- Top-k sampling: A decoding method restricting sampling to the k most probable tokens. "(-1 means no top-k sampling)"
- Top-p sampling: A decoding method restricting sampling to the smallest set of tokens whose cumulative probability exceeds p (nucleus sampling). "(1.0 means no top-p sampling)"
- Transition matrices: Tables of probabilities or counts describing transitions between discrete states across steps. "transition matrices (for simplicity shown as counts) between discrete states."
- Tool use: Having the model generate and execute code or call external tools to perform tasks like sampling. "without tool use"
Collections
Sign up for free to add this paper to one or more collections.
