---
title: Linear Extractors Overview
url: https://www.emergentmind.com/topics/linear-extractors
type: topic
---

# Linear Extractors Overview

Linear extractors are randomness extractors whose extraction map is linear over a finite field or vector space. In the binary setting, the canonical form is a fixed matrix $M$ applied to a weak source $X\in\{0,1\}^n$, producing $Y=XM$ over $\mathbb{F}_2$ and aiming for $Y$ to be $\epsilon$-close to uniform in statistical distance. Within this broad class fall deterministic linear transformations for independent, bit-fixing, and hidden-Markov sources; code-based binary extractors whose performance is governed by code spectra; affine and directional affine extractors over $\mathbb{F}_2^n$; deterministic extractors for additive sources such as arithmetic progressions and generalized arithmetic progressions; and seeded lossless rank extractors over $\mathbb{F}_q$ [1209.0732] [1405.2820] [2201.10997] [1410.7253] [2604.13431].

## 1. Formal notion and parameters

A deterministic linear extractor is defined by a fixed binary matrix $M$ of size $n\times m$, applied to a weak source $X\in\{0,1\}^n$ by
$$
Y = X M
$$
with all arithmetic modulo $2$. The extraction objective is that $Y$ be $\epsilon$-close to the uniform distribution $U_m$ in statistical distance:
$$
\|Y-U_m\|_1/2 \equiv \frac12 \sum_{y\in\{0,1\}^m} |P[Y=y] - 2^{-m}| \le \epsilon.
$$
The standard parameters are $n$ for input length, $m$ for output length, $k=H_{\min}(X)$ for min-entropy, and $\epsilon$ for allowable error [1209.0732].

In the code-based formulation, a $k\times n$ binary matrix $G$ defines
$$
Y^k = G\cdot X^n,\qquad Y^k\in\mathbb{F}_2^k.
$$
Each output bit is the modulo-$2$ sum of some subset of the input bits. The quality measure is total-variation distance:
$$
\operatorname{TVD}(P_Y,U)=\frac12\|P_Y-U\|_1=\delta_Y/2,
$$
where
$$
\delta_Y=\sum_{y\in\mathbb{F}_2^k}|P[Y^k=y]-2^{-k}|.
$$
This formulation emphasizes that linear binary extractors are deterministic post-processors that compress biased input bits into fewer output bits while attempting to reduce residual bias [1405.2820].

The literature considered here treats several source models. For independent sources, $X=(x_1,\dots,x_n)$ has independent coordinates with arbitrary bias, and
$$
H_{\min}(X)=\sum_i -\log_2 \max\{P[x_i=0],P[x_i=1]\}.
$$
For bit-fixing sources, exactly $k$ positions are uniform and independent, while the remaining $n-k$ bits are arbitrary. For hidden-Markov sources, an underlying Markov chain $\theta_1\to\theta_2\to\cdots\to\theta_n$ produces $x_i=f(\theta_i)$, subject to the bounded-noise condition
$$
P[x_{i_2}=1 \mid \theta_{i_1},\theta_{i_3}] \in [1/2-e/2,\;1/2+e/2],
$$
which implies that sums of $t$ bits have bias at most $e^{(t-1)/2}/2$ [1209.0732].

## 2. Sparse linear transformations and extraction from weak binary sources

A central construction uses a random binary matrix $M$ whose entries are i.i.d. Bernoulli$(p)$ with
$$
p=\omega(\log n/n),\qquad p\le 1/2.
$$
Here “sparse” means $p\to 0$ but $p\gg (\log n)/n$. A Fourier-analytic bound gives
$$
\rho(Y)=\|Y-U_m\|_1/2 \le \sum_{u\in\{0,1\}^m\setminus\{0\}} \bigl|P[(XM)\cdot u=1]-1/2\bigr|,
$$
reducing extraction analysis to the biases of nonzero linear forms of the output [1209.0732].

For independent sources with $H_{\min}(X)=k$, the main asymptotic statement is that if $m<k$, then for such sparse random $M$,
$$
\rho(Y)\to 0
$$
in probability as $n\to\infty$. In this sense, one can extract up to nearly $k$ uniform bits. For uniformly random $M$ with $p=1/2$, the expectation bound
$$
E[\rho(Y)] \le 2^{m-H_{\min}(X)-1}
$$
is obtained directly, while the sparse case is handled by a small-weight and large-weight partition of the nonzero vectors $u$ [1209.0732].

For bit-fixing sources, the rate is again $m<k$, but the behavior is stronger: with high probability over $M$, one gets $\rho(Y)=0$ as soon as $m<k$. The reason is structural: any nonzero linear form on $X$ must involve at least one of the $k$ uniform bits, which ensures unbiasedness [1209.0732].

For hidden-Markov sources, the same sparse-linear framework remains effective but is no longer stated as asymptotically optimal. The extractable rate is
$$
m \approx n\log_2\!\bigl(2/(1+\sqrt e)\bigr),
$$
and $\rho(Y)\to 0$ in probability whenever
$$
m / \bigl[n\log_2(2/(1+\sqrt e))\bigr] < 1.
$$
The proofs again use the partition into small-weight and large-weight contributions [1209.0732].

| Source model | Condition on output length | Asymptotic guarantee |
|---|---:|---|
| Independent source | $m<k$ | $\rho(Y)\to 0$ in probability |
| Bit-fixing source | $m<k$ | With high probability, $\rho(Y)=0$ |
| Hidden-Markov source | $m/[n\log_2(2/(1+\sqrt e))] < 1$ | $\rho(Y)\to 0$ in probability |

The same work also analyzes explicit matrices. If $G$ is the generator matrix of a primitive BCH code whose row-weight distribution is asymptotically binomial, and each input bit has bias at most $e$, then for
$$
m < n\log_2\!\bigl(2/(1+e)\bigr),
$$
the output $Y=XG^\top$ satisfies $\rho(Y)\to 0$. The accompanying trade-off is computational: primitive BCH generator matrices are dense, with approximately $n/2$ ones per row, so they require more computation than sparse random matrices [1209.0732].

## 3. Code spectra, minimum distance, and entropy bounds

A linear extractor defined by a binary matrix $G$ can be studied through the linear code it generates. If $G$ generates an $[n,k,d]$ binary linear code, then a classical minimum-distance-only bound gives
$$
\delta_Y \le 2^k \epsilon_x^d,
$$
where $\epsilon_x$ is the bias of each independent input bit. This estimate depends only on the minimum distance $d$ [1405.2820].

The sharper bound introduced later uses the full weight distribution of the code. If $A_\ell$ denotes the number of codewords of Hamming weight $\ell$, then
$$
\delta_Y \le \sum_{\ell=d}^n A_\ell\,\epsilon_x^\ell.
$$
This makes the distance spectrum, not merely the minimum distance, the controlling object. A code with the same $d$ but fewer low-weight codewords has a smaller right-hand side, and when the bulk of the spectrum lies well above $d$, the improvement over the minimum-distance bound can be substantial [1405.2820].

The same analysis yields a lower bound on the Shannon-entropy rate of the output. For $\Omega=\mathbb{F}_2^k$ and $H(U)=1$ for the uniform distribution on $\Omega$,
$$
H(Y^k) \ge 1
- (\delta_Y/2)\log_{2^k}(2^k-1)
- h(\delta_Y/2),
$$
with
$$
\delta_Y = \sum_{\ell=d}^n A_\ell\,\epsilon_x^\ell.
$$
This converts a total-variation estimate into an entropy-rate guarantee [1405.2820].

The code-spectrum viewpoint also changes code-selection criteria. Classical distance-optimal codes, including BCH codes, Reed–Solomon binary images, and some LDPC ensembles, are singled out because sparse low-weight spectra improve the bound. Numerical illustrations using the Reed–Muller $[16,11]$ and $[256,163]$ codes show that the spectrum-based right-hand side can be orders of magnitude smaller than $2^{11}\epsilon_x^4$ for moderate $\epsilon_x$ [1405.2820].

Taken together with the sparse-matrix analysis, these results show two distinct mechanisms for linear extraction. Random sparse transformations rely on probabilistic structure and asymptotic bias decay, whereas fixed code-based maps are governed by algebraic invariants of the underlying code. The latter perspective is especially relevant when deterministic post-processing is required and the matrix cannot be resampled [1209.0732] [1405.2820].

## 4. Affine, additive, and directional forms of linear extraction

Affine sources and additive sources provide a different seedless regime for linear extractors. A line source in $\mathbb{F}_q^n$ is the uniform distribution on an affine line
$$
X=\{a+td:t\in\mathbb{F}_q\},\qquad d\neq 0,
$$
and has min-entropy $H_\infty(X)=\log_2 q$. For every $n$ and prime power $q$, there is an explicit deterministic extractor
$$
\operatorname{Ext}:\mathbb{F}_q^n\to\{0,1\}
$$
such that for every line source $X$,
$$
\epsilon = |\operatorname{Ext}(X)-U_1|_{\rm TV} \le 4\sqrt{n/q}.
$$
By the XOR lemma, this extends to
$$
\operatorname{Ext}:\mathbb{F}_q^n\to\{0,1\}^m
$$
with
$$
\epsilon \le 4\sqrt{(n/q)\,2^{m/2}}.
$$
The construction is based on a norm-polynomial of degree $O(\sqrt n)$ that remains nonconstant on every affine line, reducing the field-size requirement from $q=\Omega(n^2)$ in earlier work to $q=O(n)$ [1410.7253].

The same polynomial-plus-Weil-bound method extends to short arithmetic progressions and to constant-dimensional generalized arithmetic progressions. The additive-source model includes arithmetic progressions, generalized arithmetic progressions, and Bohr sets, each of which generalizes affine sources. Over both $\mathbb{Z}_p$ and $\mathbb{Z}_p^n$, explicit extractors are obtained for additive sources with linear min-entropy, although the results over $\mathbb{Z}_p^n$ require a list-decodability condition [1410.7253].

Directional affine extractors strengthen standard affine extractors by requiring pseudorandomness of every nonzero directional derivative. For a function $f:\mathbb{F}_2^n\to\mathbb{F}_2^m$ and nonzero direction $a\in\mathbb{F}_2^n$, the derivative is
$$
D_a f(x) \coloneqq f(x+a)+f(x).
$$
The function is an $(n,k,m,\epsilon)$ directional affine extractor if for every nonzero $a$ and every affine subspace $S\subseteq\mathbb{F}_2^n$ of dimension at least $k$, the distribution $D_a f|_S$ has error at most $\epsilon$ from uniform. In the one-bit case, Gryaznov, Pudlák, and Talebanfard give an explicit triple-trace construction on $n=3k$ bits:
$$
f(x,y,z)=\operatorname{Tr}_{\mathbb{F}_{2^k}/\mathbb{F}_2}\!\bigl(\phi(x)\phi(y)\phi(z)\bigr),
$$
which is a directional affine extractor with extractor dimension $2k+c$, output length $1$, and bias $\epsilon\le 2^{-c}$ [2201.10997].

A later explicit construction is stated as
$$
\operatorname{Ext}:\mathbb{F}_2^n\times\mathbb{F}_2^d\to\mathbb{F}_2^m
$$
such that for every $k$-dimensional affine source $X\subseteq\mathbb{F}_2^n$ and every nonzero $\alpha\in\mathbb{F}_2^d$,
$$
\Delta(\operatorname{Ext}(X,\alpha),U_m)\le \epsilon.
$$
Its parameters are
$$
k=o(n),\qquad d=\operatorname{poly}(\log n),\qquad m=n-o(n),\qquad \epsilon=2^{-\Omega(n)}.
$$
A key ingredient is a new linear somewhere condenser for affine sources based on dimension expanders [2304.11495].

These directional constructions have average-case complexity consequences. If $f:\mathbb{F}_2^n\to\mathbb{F}_2$ is an $(n,k,1,\epsilon)$ directional affine extractor with $\epsilon<1/2$, then any strongly read-once linear branching program $P$ of size
$$
|P|\le \epsilon\cdot 2^{n-k-1}
$$
computes $f$ with advantage at most $\sqrt{2\epsilon}$ over random guessing:
$$
\Pr_x[g(x)=f(x)] \le 1/2 + \frac12\sqrt{2\epsilon}.
$$
In the later framework, if $\operatorname{Ext}$ is a $(k,\epsilon)$-directional affine extractor, then any strongly read-once linear branching program that agrees with $\operatorname{Ext}$ on more than $1/2+\epsilon$ fraction of inputs must have size at least $2^{n-k-1}$ [2201.10997] [2304.11495].

## 5. Rank extractors and linear-algebraic pseudorandomness over finite fields

A seeded lossless rank extractor is a family of linear maps
$$
E:\mathbb{F}_q^k\times S\to\mathbb{F}_q^r
$$
such that, for each seed $s\in S$, the map $E_s$ is an $r\times k$ matrix over $\mathbb{F}_q$, and for every full-rank matrix $M\in\mathbb{F}_q^{k\times r}$ the number of bad seeds satisfying
$$
\operatorname{rank}(E_sM)<r
$$
is at most $L$. Equivalently, for every $r$-dimensional subspace $V\subseteq\mathbb{F}_q^k$,
$$
|\{s\in S:\operatorname{rank}(E_s|_V)<r\}|\le L.
$$
When only existence of one good seed is required for every $V$, the object is called a lossless rank disperser [2604.13431].

Recent work gives explicit constructions in the small-field regime, where the field size depends only on the rank parameter and is independent of the ambient dimension. Over every non-prime field $\mathbb{F}_q$ of size $q\ge \operatorname{poly}(r)$, there exist infinitely many $k$ for which one gets an explicit $(k,r,L)$ lossless extractor with
$$
L=O(r(k-r)),\qquad n\ge 4Lq^{1/4},\qquad L/n\le q^{-1/4}.
$$
Over prime fields $\mathbb{F}_q$ with $q\ge \operatorname{poly}(r)$, a similar extractor exists with
$$
L\le (2r/\delta)^{O(\log r)}\cdot r(k-r),\qquad n\ge L/\delta.
$$
Over arbitrary small $q$, including $q=2$, one can build $(k,r)$ lossless rank dispersers of size
$$
n=O\!\left(\max\{(c\log r)/(\log q),2\}^r\cdot r(k-r)\right).
$$
These are the first explicit constructions of lossless rank extractors and weak subspace designs for $r\ll k$ over fields $\mathbb{F}_q$ with $q\ge \operatorname{poly}(r)$ and $q$ non-prime [2604.13431].

The constructions are algebraic. One replaces classical polynomial-method constructions by analogues over a function field $F/F_q$ of genus $g$ with many rational places, chooses functions from suitable Riemann–Roch spaces, and evaluates them at rational places to form the extractor matrices. The losslessness argument expands $\det(EM)$ via Cauchy–Binet, uses valuations at a distinguished place $P_\infty$, and bounds the number of bad seeds by the pole order of a nonzero determinant function. For prime fields, the construction passes through a quadratic extension and then uses explicit hitting sets for symbolic determinants of the form
$$
f(x)=\det\!\left(\sum x_iA_i\right)
$$
with each $A_i$ of rank at most $1$ [2604.13431].

A complementary Fourier-analytic framework uses $\epsilon$-biased sets in $\mathbb{F}_q^k$. If $\epsilon<q^{-s}$, then such a set meets every codimension-$s$ affine subspace. If $\epsilon<(q-1)/(2q^{s+1})$, then its projectivization is a strong $s$-blocking set in $\mathrm{PG}(k-1,q)$. Through these connections, explicit rank extractors yield weak subspace designs and strong blocking sets, including constructions of size $O(s(k-s)q^s)$ for sufficiently large non-prime fields $q\ge \operatorname{poly}(s)$ and $kq^{(2+o(1))s}$ for arbitrary $q$ via the $\epsilon$-biased method [2604.13431].

## 6. Computational trade-offs and relation to seeded extraction

One motivation for linear extractors is implementation speed. Sparse random matrices are computationally fast, easy to implement using hardware like FPGAs, and attractive in high-speed random number generation. In block-processing mode, one partitions the input stream into blocks of length $n$ and computes $Y=XM$ by a matrix-vector XOR. In streaming mode, one keeps a rolling vector $V\in\{0,1\}^m$ and updates
$$
V \leftarrow V \oplus (x_i\cdot \mathrm{row}_i(M))
$$
for each new bit $x_i$. The total cost for an $n\to m$ extraction is $O(pnm)$ XORs, and with $p=\Theta(\log n/n)$ this becomes $O(m\log n)$ [1209.0732].

The density of the matrix determines the speed trade-off. Sparse-random $M$ has density approximately $p\to 0$, with per-bit processing cost $O(pn)=o(n)$ and a simple XOR network. A primitive BCH generator matrix has density $\Theta(1)$ and per-bit cost $\Theta(n)$. In FPGA terms, sparse-random $M$ can be implemented with $O(pnm)$ XOR gates, whereas BCH-based matrices require $O(nm)$ gates and therefore higher area and power [1209.0732].

These deterministic and fixed-matrix approaches sit alongside seeded extractors. A strong seeded extractor is a function
$$
\operatorname{Ext}:\{0,1\}^n\times\{0,1\}^d\to\{0,1\}^m
$$
such that for every distribution $X$ on $\{0,1\}^n$ with $H_\infty(X)\ge k$ and independent uniform seed $Y\sim U_d$,
$$
\Delta\bigl((Y,\operatorname{Ext}(X,Y)),(Y,U_m)\bigr)\le \epsilon.
$$
Recent constructions achieve, for all $k$ and $\epsilon$,
$$
d=O(\log(n/\epsilon)),\qquad m=(1-\eta)k,
$$
with running time $O(n\log^c n)$ after a one-time preprocessing step in the low-$k$ regime. The same work gives an instantiation of Trevisan’s extractor with evaluation time
$$
T_{\mathrm{RAM}}(n,m,\epsilon)=O(n)+m\log(1/\epsilon)\operatorname{polylog}n,
$$
and therefore truly linear time $O(n)$ when
$$
m\le n/(\log(1/\epsilon)\operatorname{polylog}n).
$$
These seeded constructions directly yield privacy amplification protocols with communication equal to the seed length [2411.07473].

The contrast is structural. Sparse linear transformations require no external seed, no look-ahead, and no multiprecision arithmetic, while seeded extractors are defined for every source of min-entropy at least $k$ and achieve strong guarantees conditioned on the seed. Within the linear-extractor landscape, the principal axes are therefore the source model, whether a seed is available, the algebraic structure of the map, and the trade-off between explicitness, extraction rate, and evaluation complexity [1209.0732] [2411.07473].

Source: https://www.emergentmind.com/topics/linear-extractors