Random Float Sampling (RFS)

• Random Float Sampling (RFS) is a collection of techniques that use floating-point randomness for tasks ranging from biased integer generation to secure noise synthesis and positional encoding.
• In statistical computing, mapping finite-precision floats to integers via multiply–floor methods introduces quantization bias, which has been rigorously analyzed and quantified.
• Applications extend to Transformer models that use continuous indexing for improved extrapolation and hardware solutions that leverage s-MTJs for energy-efficient uniform Float16 sampling.

Random Float Sampling (RFS) is a polysemous term in recent technical literature. In statistical computing, it commonly denotes the practice of generating discrete random outcomes from floating-point uniform variates, especially via the map $Y = 1 + \lfloor mX \rfloor$, a construction shown to induce quantization bias under finite precision (Ottoboni et al., 2018). In differential privacy, the same phrase is used for secure floating-point sampling schemes that combine multiple independent uniforms so that sampled noise cannot be efficiently inverted back to its source randomness (Holohan et al., 2021). In Transformer research, RFS denotes a position-indexing strategy in which tokens are assigned randomly sampled continuous values during training to improve length generalization (Shimizu et al., 15 Feb 2026). A further, hardware-oriented usage concerns direct uniform Float16 sampling with stochastic magnetic tunnel junctions (s-MTJs) (Alder et al., 2024). These usages share an emphasis on float-valued randomness, but they address different problems, assumptions, and failure modes.

1. Scope and principal usages

In the cited literature, “Random Float Sampling” refers to several distinct mechanisms rather than a single canonical algorithm. The common element is that floating-point random values are treated as primary computational objects, but the objectives range from integer generation to privacy-preserving noise injection, positional indexing, and hardware-native sampling.

Usage of RFS	Core mechanism	Representative source
Statistical integer generation	Multiply a random float by $m$, take the floor, and add $1$	(Ottoboni et al., 2018)
Secure random sampling in differential privacy	Combine multiple independent uniform variates through infinitely divisible decompositions	(Holohan et al., 2021)
Transformer position indexing	Sample continuous position values in $[0,1)$, sort them, and use them as indices	(Shimizu et al., 15 Feb 2026)
Uniform Float16 generation	Configure bitwise Bernoulli probabilities for IEEE Float16 output using s-MTJs	(Alder et al., 2024)

The terminological breadth matters because the mathematical critique applicable to multiply–floor integer generation does not apply to the differential-privacy or Transformer usages. Conversely, the security arguments developed for differential privacy do not address the positional out-of-distribution problem studied in sequence models.

2. Statistical-computing RFS: multiply–floor sampling and quantization bias

A central usage of RFS arises in software that generates a random integer uniformly on $\{1,\ldots,m\}$ by drawing a pseudorandom float $X$ on $[0,1)$ and returning

$Y = 1 + \lfloor mX \rfloor.$

R, in Version 3.5.1 patched, was described as using this strategy for random integers, and because the sample function relies on generating random integers, random sampling in R is biased (Ottoboni et al., 2018).

The problem is finite-precision quantization. In practice, $X$ is not truly continuous, but takes values

$X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$

typically with $m$0. The interval $m$1 is partitioned into

$m$2

but the number of quantized float values falling into each $m$3 is generally unequal. Consequently,

$m$4

is not exactly uniform (Ottoboni et al., 2018).

The distortion can be analyzed through the ratio of the largest and smallest output probabilities. For some $m$5,

$m$6

to first order, where $m$7 and $m$8 (Ottoboni et al., 2018). The dependence on $m$9 is substantive rather than asymptotically negligible. With $1$0 and $1$1, the ratio is approximately $1$2, corresponding to a $1$3 difference between the most likely and least likely outcomes. As $1$4 approaches $1$5, the ratio approaches $1$6, so one output can be about twice as likely as another (Ottoboni et al., 2018).

The broader critique is not limited to one implementation. “Random Sampling: Practice Makes Imperfect” argues that many packages rely on the false assumption that PRNG outputs are IID $1$7 variables, whereas actual PRNG outputs are discrete and generated from limited state spaces (Stark et al., 2018). In that analysis, mapping discrete PRNG outputs to $1$8 by multiplying and rounding can never be exactly uniform unless $1$9 is a power of $[0,1)$0. When $[0,1)$1, at least $[0,1)$2 values cannot be selected at all. The paper also gives a concrete example: for $[0,1)$3, R’s sample() function generates about $[0,1)$4 even numbers and $[0,1)$5 odd numbers (Stark et al., 2018).

These results isolate a structural flaw: even if the underlying PRNG were otherwise satisfactory, constructing integers through random floats can still produce biased sampling. The bias is a property of the mapping, not merely of the generator.

3. Direct integer generation and related algorithmic alternatives

The standard fix is to construct random integers directly from random bits rather than from floating-point multiplication. One formulation computes a minimal bit length $[0,1)$6 such that $[0,1)$7, generates an integer $[0,1)$8, and returns $[0,1)$9 only if $\{1,\ldots,m\}$0; otherwise it rejects and retries (Ottoboni et al., 2018). Because every accepted bit-string corresponds to exactly one output in $\{1,\ldots,m\}$1, the resulting distribution is uniform if the input bits are independent and fair.

The method is explicitly described as the strategy taken in Python’s numpy.random.randint() function, and related descriptions identify the same rejection-based construction in C++11 std::uniform_int_distribution and the Mersenne Twister documentation (Ottoboni et al., 2018). The trade-off is that some draws are rejected; if $\{1,\ldots,m\}$2 is not a power of $\{1,\ldots,m\}$3, the rejection probability can be substantial, approaching $\{1,\ldots,m\}$4 in the worst case (Ottoboni et al., 2018).

The critique in (Stark et al., 2018) extends from integer generation to sampling workflows. It recommends avoiding methods that assume PRNG outputs are IID $\{1,\ldots,m\}$5, including generating a random sample by permuting the population and taking the first $\{1,\ldots,m\}$6 items, or generating random integers by multiplying a pseudo-random binary fraction or float by a constant and rounding the result. More accurate methods are described as available, including bit-masking with rejection sampling for integers, reservoir sampling, and Fisher–Yates or Knuth shuffle for permutations (Stark et al., 2018).

That paper also links RFS-style failures to state-space limitations. A 32-bit PRNG cannot generate all permutations of $\{1,\ldots,m\}$7 items, and even Mersenne Twister cannot generate all permutations of $\{1,\ldots,m\}$8 items (Stark et al., 2018). This suggests that multiply–floor bias is one component of a larger issue: software can simultaneously suffer from quantization artifacts, insufficient generator state, and algorithmic overuse of the available randomness.

4. Secure random float sampling in differential privacy

In differential privacy, the phrase Random Float Sampling refers to a different problem. Mechanisms such as Laplace or Gaussian noise addition are often implemented by inverse transform sampling,

$\{1,\ldots,m\}$9

with $X$0 a floating-point approximation to a uniform draw on $X$1. Because floating-point representations are finite, the mapping from $X$2 to $X$3 can be inverted, enabling attacks that recover the original value with non-negligible advantage (Holohan et al., 2021).

The proposed remedy is to make output generation non-injective by combining multiple independent uniform variates and exploiting infinite divisibility. A distribution is infinitely divisible if, for any $X$4, it can be expressed as the sum of $X$5 IID random variables with the original law recovered after aggregation. The paper states that both Laplace and Gaussian are infinitely divisible, and uses this fact to construct secure samplers whose outputs have large preimages in the space of source uniforms (Holohan et al., 2021).

For Gaussian sampling, one decomposition is

$X$6

For Laplace sampling, the paper lists several equivalent constructions, including

$X$7

$X$8

and the implementation-oriented expression

$X$9

with independent $[0,1)$0 (Holohan et al., 2021).

The security argument is combinatorial. Standard inverse transform sampling may map one output back to a unique source float from a pool of $[0,1)$1 possibilities. Under the proposed construction, each output can arise from a much larger set of tuples $[0,1)$2, with attack cost scaling as $[0,1)$3; for 53-bit floats and $[0,1)$4, the paper gives $[0,1)$5 as the attack complexity (Holohan et al., 2021). The method is described as generalisable to any infinitely divisible probability distribution and as designed to make side channel attack infeasible because brute-force attacks are inherently exponential in the size of the domain (Holohan et al., 2021).

This usage of RFS is therefore not a critique of floating-point sampling per se. Rather, it is a reconstruction of float-based sampling so that finite-precision outputs preserve the intended statistical law while becoming computationally non-invertible.

5. RFS as continuous position indexing in Transformers

A third usage appears in sequence modeling, where Random Float Sampling is introduced as a position encoding strategy for improving length generalization in Transformers (Shimizu et al., 15 Feb 2026). Instead of assigning tokens discrete indices from a predefined set, RFS assigns randomly sampled continuous values during training.

For a sequence of length $[0,1)$6, training samples

$[0,1)$7

and then sorts the sampled floats to obtain

$[0,1)$8

At inference, for context length $[0,1)$9, positions are assigned deterministically by even partitioning: $Y = 1 + \lfloor mX \rfloor.$0 All indices are additionally multiplied by a scaling factor $Y = 1 + \lfloor mX \rfloor.$1, typically $Y = 1 + \lfloor mX \rfloor.$2 (Shimizu et al., 15 Feb 2026).

These float positions can replace integer indices in absolute sinusoidal encoding, RoPE, and ALiBi. For the sinusoidal case, the paper gives

$Y = 1 + \lfloor mX \rfloor.$3

The stated motivation is to avoid out-of-distribution issues on unseen lengths by exposing the model to diverse indices during training (Shimizu et al., 15 Feb 2026).

The empirical claims are specific. On copy tasks, RFS achieves about $Y = 1 + \lfloor mX \rfloor.$4 accuracy on sequences twice as long as those seen during training, while the best alternative, NoPE, drops to about $Y = 1 + \lfloor mX \rfloor.$5 accuracy at only $Y = 1 + \lfloor mX \rfloor.$6 longer lengths (Shimizu et al., 15 Feb 2026). The paper further reports superior performance on reverse, sort, and other algorithmic sequence tasks, as well as higher out-of-distribution accuracy on zero-shot commonsense reasoning benchmarks including HellaSwag, RACE, ARC-e, ARC-c, OpenBookQA, WinoGrande, and BoolQ (Shimizu et al., 15 Feb 2026).

A notable analytical claim is that simple extension of standard positional encodings leads to a rank increase in the position encoding matrix, contaminating semantic subspaces, whereas RFS keeps the rank of the position matrix stable regardless of length (Shimizu et al., 15 Feb 2026). This suggests that the method’s benefit is not merely stochastic regularization, but a modification of the geometry of positional representations under extrapolation.

6. Hardware-native uniform float sampling

A fourth line of work addresses direct hardware generation of floating-point random samples. “Energy-Efficient Sampling Using Stochastic Magnetic Tunnel Junctions” introduces an algorithm for uniform Float16 sampling using room-temperature s-MTJ devices (Alder et al., 2024). The device-level objective is to generate truly random IEEE Float16 values by assigning Bernoulli probabilities $Y = 1 + \lfloor mX \rfloor.$7 to the output bits: $Y = 1 + \lfloor mX \rfloor.$8

For Float16, the sign and mantissa bits use $Y = 1 + \lfloor mX \rfloor.$9, while exponent-bit probabilities are chosen so that the resulting 16-bit word, when interpreted as an IEEE Float16 number, is uniform over the representable range. The desired condition is written as

$X$0

After sampling, values can be mapped to an interval $X$1 by

$X$2

with the note that the normalization should be performed in Float32 or higher for precision (Alder et al., 2024).

The paper reports concrete energy figures. Biasing the exponent bits requires a total of $X$3 at $X$4 MHz sampling, corresponding to $X$5 picojoules per sample; reading all s-MTJs costs $X$6 femtojoules per sample. For $X$7 Float16 samples, total energy is about $X$8 mJ for raw sampling and $X$9 mJ including normalization arithmetic (Alder et al., 2024). These values are compared against software generators: a minimum factor of $X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$0 improvement relative to Mersenne-Twister and a factor of $X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$1 relative to PCG (Alder et al., 2024).

Beyond uniform Float16 sampling, the same work represents an arbitrary 1D distribution as a finite mixture of non-overlapping uniform distributions,

$X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$2

and uses this representation for sampling, convolution, and prior-likelihood operations. Reported approximation errors are given as KL-divergence $X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$3 for convolutions and $X_k = \frac{k}{2^w}, \qquad k = 0,\ldots,2^w-1,$4 for prior-likelihood multiplication (Alder et al., 2024). In this sense, RFS becomes a hardware-supported primitive for broader probabilistic computation.

7. Terminological ambiguity and the unrelated “RFS” of tracking theory

A persistent source of confusion is that the acronym RFS is also entrenched in estimation and tracking as Random Finite Set. In that literature, RFS refers to finite-set-valued stochastic modeling of targets, landmarks, and measurements, not to float sampling. Examples include arithmetic average fusion for unlabeled and labeled RFS densities (Li, 2022), heterogeneous unlabeled and labeled RFS filter fusion (Li et al., 2023), and PMBM-based data-association sampling for batch SLAM (Ge et al., 2024).

The mathematical vocabulary of that domain—PHD consistency, MB, MBM, PMBM, GLMB, LMB, Gibbs sampling over association hypotheses, and GraphSLAM back-ends—is unrelated to the floating-point sampling problems discussed above. The overlap is purely acronymic. This distinction is operationally important because searches for “RFS” in arXiv-scale corpora will retrieve both families of work, but the technical content is disjoint.

A second misconception concerns the phrase “random float sampling” itself. In statistical computing, the term is often used critically, because constructing integers by multiplying floats and rounding is shown to be biased under finite precision (Ottoboni et al., 2018). In differential privacy and Transformer research, by contrast, random float sampling is a constructive technique: in one case to make floating-point noise generation non-invertible (Holohan et al., 2021), and in the other to expose a model to diverse continuous position indices for better out-of-distribution behavior (Shimizu et al., 15 Feb 2026). The same wording therefore spans both a documented failure mode and two affirmative design patterns.

Taken together, the literature presents Random Float Sampling not as a single settled method but as a cluster of domain-specific techniques centered on float-valued randomness. Its technical meaning depends entirely on context: in legacy integer generation it is a source of measurable bias; in privacy mechanisms it is a route to secure non-injective sampling; in sequence modeling it is a continuous indexing scheme for extrapolation; and in hardware sampling it is an energy-efficient way to generate floating-point randomness directly.