Arithmetic Sampling Techniques

Updated 5 July 2026

Arithmetic sampling is a family of techniques where sampling decisions arise from arithmetic operations rather than explicit state enumeration.
It finds applications in constant-probability distinguishers, parallel decoding in language models, and exact random variate generation, ensuring precise and efficient sampling.
By leveraging operations like modular multiplication, interval refinement, and digit-wise comparisons, arithmetic sampling optimizes both computational speed and accuracy.

Searching arXiv for recent and foundational uses of “Arithmetic Sampling” across domains. Arithmetic sampling denotes a family of techniques in which sampling decisions are realized directly through arithmetic structure rather than through explicit state enumeration. The phrase appears in multiple, technically distinct literatures. In theoretical computer science, it refers to the sampler $Sample(x)=[a x \bmod 2^w \le t]$ , which yields a constant-probability distinguisher for arbitrary commutative monoids (Thorup, 2014). In autoregressive sequence modeling, it refers to decoding from an implicit arithmetic code-book defined by token probabilities, so that a code point in $[0,1)$ maps deterministically to a full sequence and many samples can be produced in parallel (Vilnis et al., 2022). Related work uses quasi-random code points for multi-sample inference in reasoning and translation (Parashar et al., 2024). A separate usage applies arithmetic sampling to exact generation from the normal distribution via digit-by-digit uniform random numbers (Karney, 2013). By contrast, finite-population sampling also contains “arithmetic” constructions, but these are arithmetic-mean estimators rather than arithmetic sampling in the coding-theoretic or hashing-theoretic sense (Singh et al., 2014).

1. Scope of the term

The principal senses of arithmetic sampling differ in object, mechanism, and purpose.

Domain	Core object	Representative formulation
Distinguishers and streaming	Random subset of keys	$Sample(x)=[a x \bmod 2^w \le t]$
Autoregressive decoding	Code point in $[0,1)$ mapped to a sequence	Arithmetic code-book induced by the model
Exact random variate generation	Digit-by-digit uniform deviate	Base- $b$ u-rand construction
Finite-population sampling	Estimator of a population mean	Weighted arithmetic-mean dual-to-ratio estimator

What unifies these uses is the replacement of generic randomness handling by arithmetic structure: modular multiplication over machine words, interval refinement under cumulative probabilities, or digit-wise comparison against rational thresholds. This suggests that “arithmetic sampling” is best understood as a methodological label rather than a single algorithmic object.

2. Constant-probability distinguishers over commutative monoids

In the distinguisher framework, one begins with an arbitrary key universe $U$ and a commutative monoid $(R,+,0)$ . A random sampling function $Sample:U\to\{0,1\}$ defines a sampled sum

$E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$

where $v:U\to R$ is a fixed value assignment. The sampler is a $[0,1)$ 0-distinguisher if, for every non-zero assignment $[0,1)$ 1, the probability that $[0,1)$ 2 is at least $[0,1)$ 3 (Thorup, 2014).

The arithmetic construction fixes a word size $[0,1)$ 4, treats keys as integers in $[0,1)$ 5, chooses $[0,1)$ 6 uniformly from the odd $[0,1)$ 7-bit integers and $[0,1)$ 8 uniformly from all $[0,1)$ 9-bit integers, and defines

$Sample(x)=[a x \bmod 2^w \le t]$ 0

With standard unsigned machine arithmetic, the multiplication already occurs modulo $Sample(x)=[a x \bmod 2^w \le t]$ 1, so the sampling decision is implemented as a*x<=t in C (Thorup, 2014).

The central result is that this sampler is a $Sample(x)=[a x \bmod 2^w \le t]$ 2-distinguisher. The proof sorts the hash values $Sample(x)=[a x \bmod 2^w \le t]$ 3 of the support $Sample(x)=[a x \bmod 2^w \le t]$ 4, partitions $Sample(x)=[a x \bmod 2^w \le t]$ 5 into adjacent intervals, and marks an interval “good” when the associated partial sum is non-zero. For uniform $Sample(x)=[a x \bmod 2^w \le t]$ 6, the success probability conditioned on $Sample(x)=[a x \bmod 2^w \le t]$ 7 is the measure of the union of good intervals divided by $Sample(x)=[a x \bmod 2^w \le t]$ 8. The lower bound is obtained by relating good-interval measure to local gap lengths around each hashed key and then proving the Average Gap-Length Lemma, namely that for each fixed $Sample(x)=[a x \bmod 2^w \le t]$ 9,

$[0,1)$ 0

where $[0,1)$ 1 and $[0,1)$ 2 is the minimum of the two adjacent interval lengths around $[0,1)$ 3. Summing over keys yields the exact constant $[0,1)$ 4 (Thorup, 2014).

The construction was motivated by situations in which the full value map $[0,1)$ 5 cannot be stored, but the sampled sum can be maintained incrementally from a stream of pairs $[0,1)$ 6. In that setting,

$[0,1)$ 7

yet one can maintain only the single accumulator $[0,1)$ 8 (Thorup, 2014).

Its practical significance follows from the implementation profile. The sampler uses one multiply and one compare per key, works for any commutative monoid, and uses $[0,1)$ 9 random bits. The comparison in the source material is explicit: Naor–Naor small-bias distinguishers use a 7-independent hash function implemented via a degree-6 polynomial modulo a large prime; Dietzfelbinger’s universal hash $b$ 0 is fast but does not give a constant-probability distinguisher; AMS sketches achieve $b$ 1-distinguishing probability for real-valued $b$ 2 but require 4-independent hashing and processing squares (Thorup, 2014).

The same source states two open directions. First, amplification by $b$ 3 independent copies reduces failure to $b$ 4. Second, it remains open whether one can obtain $b$ 5 with an even faster or lower-randomness construction, or whether a matching lower bound rules out $b$ 6 with only one multiply and one compare (Thorup, 2014).

3. Arithmetic code-books for parallel decoding in LLMs

In autoregressive language modeling, arithmetic sampling is defined through the arithmetic code-book implicitly induced by the model. At each time step $b$ 7, if the model assigns probabilities $b$ 8 to vocabulary items $b$ 9, then the cumulative distribution

$U$ 0

partitions $U$ 1 into half-open intervals $U$ 2, one per token (Vilnis et al., 2022). A code point $U$ 3 selects the unique token whose interval contains $U$ 4, after which the residual is renormalized and decoding continues. Repeating to EOS yields an exact sample from the model distribution because the interval assigned to a full sequence has length exactly $U$ 5 (Vilnis et al., 2022).

For $U$ 6 parallel samples, the paper chooses a one-dimensional shifted lattice

$U$ 7

and arithmetic-decodes each code independently. The procedure is “embarrassingly parallel”: no inter-sample communication is required during decoding, CDFs can be formed by cumulative sums over the vocabulary dimension, and the only per-sample state is the current renormalized code and interval bounds (Vilnis et al., 2022).

The theoretical guarantees are unusually strong for a decoding rule. For any real-valued sequence score $U$ 8, the sample average over arithmetic samples is unbiased for $U$ 9, and as $(R,+,0)$ 0 it converges almost surely to the same expectation. The paper also gives beam-diversity bounds: if a prefix $(R,+,0)$ 1 has probability $(R,+,0)$ 2, then with spacing $(R,+,0)$ 3 it can appear at most $(R,+,0)$ 4 times and at least $(R,+,0)$ 5 times; in particular, if $(R,+,0)$ 6, that prefix never repeats (Vilnis et al., 2022).

The framework also subsumes common logit transformations. Temperature scaling replaces logits $(R,+,0)$ 7 by $(R,+,0)$ 8 before renormalization; top- $(R,+,0)$ 9, nucleus, and typical decoding are handled by truncating support and rebuilding the CDF (Vilnis et al., 2022). Arithmetic sampling is therefore not a separate model family but a decoding rule that sits downstream of the model’s conditional distributions.

Empirically, the paper evaluates WMT14 En→Fr, WMT16 En→Ro, and CNN/DailyMail with fine-tuned T5-Base. On WMT14 En→Fr with beam size $Sample:U\to\{0,1\}$ 0, oracle BLEU is reported as approximately $Sample:U\to\{0,1\}$ 1 for ancestral sampling, approximately $Sample:U\to\{0,1\}$ 2 for arithmetic sampling, and approximately $Sample:U\to\{0,1\}$ 3 for beam search, so arithmetic sampling closes about $Sample:U\to\{0,1\}$ 4 of the gap between standard sampling and beam search. For sentence-level BLEU estimation with $Sample:U\to\{0,1\}$ 5, the reported standard deviation is approximately $Sample:U\to\{0,1\}$ 6 BLEU points for standard sampling and approximately $Sample:U\to\{0,1\}$ 7 for arithmetic sampling, a reduction of about $Sample:U\to\{0,1\}$ 8. Similar $Sample:U\to\{0,1\}$ 9– $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 0 standard-deviation reductions are reported for En→Ro BLEU and CNN/DailyMail ROUGE2 (Vilnis et al., 2022).

4. Quasi-random arithmetic sampling for multi-sample inference

A later line of work studies arithmetic sampling specifically as a multi-sample inference primitive for chain-of-thought self-consistency and minimum Bayes risk decoding. The defining change is the code generator: instead of independent uniforms, the method draws a single offset $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 1 and sets

$E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 2

These codes form an evenly spaced lattice over $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 3 and are then decoded independently through the model’s arithmetic code-book (Parashar et al., 2024).

The interpretation given in the source is geometric. Each next-token distribution partitions the current live interval $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 4 into contiguous subintervals, and the fixed code $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 5 deterministically selects the subinterval that contains it. Because the codes are stratified across $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 6, they tend to branch apart early and thereby improve diversity. The asymptotic computational cost remains $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 7, the same as $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 8 ancestral samples, and there is no communication across samples, so the procedure remains trivially parallelizable (Parashar et al., 2024).

The reported gains are specific to multi-sample settings. On GSM8K with 8-shot prompting and $E_v(Sample)=\sum_{x\in U:Sample(x)=1} v(x),$ 9, arithmetic sampling improves accuracy by $v:U\to R$ 0 percentage points for Gemma-7B and $v:U\to R$ 1 percentage points for Llama-2-7B over ancestral sampling. On CommonsenseQA with 6-shot prompting and $v:U\to R$ 2, the gains are $v:U\to R$ 3 percentage points for Gemma-7B and $v:U\to R$ 4 percentage points for Llama-2-7B. For WMT19 De→En and Ru→En with Flan-T5 and MT0, arithmetic sampling improves COMET by $v:U\to R$ 5 to $v:U\to R$ 6 percentage points, including $v:U\to R$ 7 on Flan-T5 De→En at $v:U\to R$ 8 (Parashar et al., 2024). The abstract summarizes the broader pattern as a $v:U\to R$ 9– $[0,1)$ 00 point increase in GSM8K accuracy and a $[0,1)$ 01– $[0,1)$ 02 point increment in COMET score, without significant computational overhead (Parashar et al., 2024).

The limitations are also explicit. The mapping from $[0,1)$ 03 to tokens depends on a fixed vocabulary ordering, and the reported experiments use a random permutation. The construction is one-dimensional, using a single real code $[0,1)$ 04, and the method is formulated for diverse sampling rather than for finding the single most probable sequence. The paper therefore identifies semantically informed vocabulary ordering, higher-dimensional stratification, and beam-search-like or risk-averse extensions as future directions (Parashar et al., 2024).

5. Exact random variate generation via u-rands

In exact random number generation, arithmetic sampling refers to a digit-by-digit paradigm in which a continuous uniform deviate is represented as a base- $[0,1)$ 05 expansion

$[0,1)$ 06

but digits are generated only as needed. The paper calls such partially revealed quantities “u-rands” (Karney, 2013). Acceptance tests are performed by comparing prefixes against rational thresholds; once the accept/reject logic is resolved, the remaining digits can be copied directly into the output representation.

For the standard normal distribution, the algorithm samples an integer part $[0,1)$ 07, then a fractional part $[0,1)$ 08 as a u-rand, performs a rejection step based on $[0,1)$ 09, and finally assigns an independent sign to $[0,1)$ 10 (Karney, 2013). The correctness argument factors the density into a term proportional to $[0,1)$ 11 for the integer part and a term proportional to $[0,1)$ 12 for the fractional part, yielding the joint density proportional to $[0,1)$ 13 and hence the exact standard normal law after sign randomization.

The paper emphasizes implementation properties rather than asymptotic novelty. It “performs no extended precision arithmetic, calls no transcendental functions, and, indeed, uses no floating point arithmetic whatsoever; it uses only simple integer operations” (Karney, 2013). The mean cost scales linearly in the requested precision, and the method can generate values exactly rounded to any precision. It can also be adapted to the discrete normal distribution when parameters are rational (Karney, 2013).

This usage differs from both the modular-hash distinguisher and the LLM code-book decoder. The common element is again arithmetic realization: comparisons are digit-wise and integer-only, rather than mediated by floating-point approximations or table-driven transforms. A plausible implication is that the phrase “arithmetic sampling” here denotes exactness of implementation as much as it denotes a sampling law.

6. Terminological boundary with finite-population sampling

A distinct but terminologically adjacent line of work concerns estimators in simple random sampling without replacement. In that setting, the study variable is $[0,1)$ 14, the auxiliary variables are $[0,1)$ 15, the population means $[0,1)$ 16 are known, and one defines $[0,1)$ 17. The arithmetic-mean dual-to-ratio estimator is

$[0,1)$ 18

with nonnegative weights satisfying $[0,1)$ 19 (Singh et al., 2014).

Using the first-order error expansion

$[0,1)$ 20

and $[0,1)$ 21, the paper derives

$[0,1)$ 22

and

$[0,1)$ 23

It further compares arithmetic-, geometric-, and harmonic-mean estimators, stating that all three share the same MSE up to first order, whereas under a mild positivity condition the absolute biases satisfy

$[0,1)$ 24

The arithmetic-mean estimator therefore has the smallest absolute bias among the three under those conditions (Singh et al., 2014).

This literature belongs to survey sampling rather than to arithmetic sampling in the hashing, decoding, or exact-random-generation senses. The proximity is lexical rather than conceptual. Including it is nevertheless useful because it marks a common source of confusion: “arithmetic” in sampling theory may refer either to the arithmetic form of an estimator or to an arithmetic mechanism that realizes the sampling process itself.

7. Cross-cutting themes, limitations, and misconceptions

Across these literatures, arithmetic sampling repeatedly trades generic randomness management for low-level arithmetic structure. In the distinguisher setting, this is modular multiplication over $[0,1)$ 25; in LLM decoding, it is interval subdivision under a CDF; in exact normal generation, it is digit-wise comparison of u-rands; in survey sampling, it is weighted arithmetic aggregation. This suggests a family resemblance rather than a single canonical definition.

Several misconceptions are corrected directly by the cited works. Arithmetic sampling for LLMs is not beam search: beam search is sequential across beams and requires synchronization, whereas arithmetic sampling is embarrassingly parallel (Vilnis et al., 2022). It is also not merely ancestral sampling with different random seeds: the shifted lattice or quasi-random code construction is what yields diversity guarantees and variance reduction (Vilnis et al., 2022, Parashar et al., 2024). Conversely, arithmetic sampling in the normal-generator paper is not an arithmetic-coding decoder for text; it is an exact digit-by-digit sampler for a continuous distribution (Karney, 2013). The modular-hash construction is not a general-purpose approximate sketch in the AMS sense, because its guarantee is formulated as non-zero sampled sum over arbitrary commutative monoids with success probability exactly $[0,1)$ 26 (Thorup, 2014).

The limitations are domain-specific. For distinguishers, the open problem is whether one can exceed $[0,1)$ 27 with comparable speed and randomness usage (Thorup, 2014). For LLM inference, vocabulary ordering, one-dimensional codes, and the absence of a MAP objective remain active constraints (Parashar et al., 2024). For exact normal sampling, the contribution is exactness and precision scaling, not acceleration relative to conventional floating-point generators (Karney, 2013). For finite-population estimation, the arithmetic construction improves bias properties only within a first-order approximation regime and under the stated sampling-design assumptions (Singh et al., 2014).

Taken together, these strands show that arithmetic sampling is not a single algorithm but a recurring design pattern: represent randomness through arithmetic objects that are cheap to manipulate, analyzable with exact or near-exact arguments, and well matched to the computational substrate of the problem domain.