Lindeberg Replacement Method: Comparison Technique

• The Lindeberg replacement method is a comparison technique that sequentially replaces parts of a random object to prove distributional approximations and universality.
• It employs a telescoping sum and Taylor expansions to isolate the effects of mean and variance mismatches while controlling cumulative errors.
• Its applications span central limit theorems, deep neural network universality, stable-law approximations, and random matrix theory, demonstrating practical versatility.

The Lindeberg replacement method is a comparison technique for proving distributional approximation and universality by successively replacing individual coordinates, summands, layers, or columns of a random object with better understood counterparts and controlling the cumulative error through a telescoping decomposition. In the formulations developed for smooth observables, the basic mechanism is a coordinatewise Taylor expansion; in other settings it is coupled with Kolmogorov forward equations, Stein kernels, Gaussian smoothing, or representation-theoretic decompositions. The method appears in generalized form for smooth functions of independent coordinates (Korada et al., 2010), in stable approximation without Fourier inversion (Chen et al., 2018), in total-variation versions of the central limit theorem (Dung et al., 4 Nov 2025), in quantitative universality for deep neural networks (Giovagnini et al., 4 May 2026), and in column-swapping arguments for random matrices over finite local rings (Lvov, 15 Jan 2026).

1. General scheme and abstract formulation

A standard formulation begins with two random vectors $U=(U_1,\dots,U_n)$ and $V=(V_1,\dots,V_n)$ in $\mathbb{R}^n$, each with independent components, and a smooth test function $f:\mathbb{R}^n\to\mathbb{R}$. In the generalized principle stated in "Applications of Lindeberg Principle in Communications and Statistical Learning" (Korada et al., 2010), one defines

$$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$

assumes $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$, and requires bounded first, second, and third coordinate derivatives: $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$ Under these hypotheses,

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$

The corresponding “local” version replaces global derivative bounds by expectations of derivatives evaluated along mixed vectors $(U_{1:i-1},s,V_{i+1:n})$, together with integral remainders involving $\partial_i^3 f$. This variant is designed for high-dimensional applications in which uniform control of all $V=(V_1,\dots,V_n)$0 over $V=(V_1,\dots,V_n)$1 is unavailable (Korada et al., 2010).

The core identity is the telescoping sum

$V=(V_1,\dots,V_n)$2

Each summand isolates a one-coordinate replacement. A Taylor expansion in the $V=(V_1,\dots,V_n)$3-th coordinate then separates the effect of mean mismatch, variance mismatch, and the third-order remainder. This suggests that the replacement method is best understood not as a single theorem but as a modular proof pattern: telescoping plus a one-step comparison estimate.

2. Taylor expansion, moment matching, and cancellation

In the smooth setting, the method operates by expanding the observable in the coordinate being replaced. The proof sketch in (Korada et al., 2010) gives the basic form: $V=(V_1,\dots,V_n)$4 and similarly for $V=(V_1,\dots,V_n)$5. Because $V=(V_1,\dots,V_n)$6 is independent of $V=(V_1,\dots,V_n)$7, expectations factorize, producing the explicit contributions of the first two moments and a remainder term controlled by third derivatives and third moments.

When the first two moments match, the leading terms cancel and only the third-order remainder remains. This is the mechanism emphasized in several later applications. In the deep-network setting, for example, the layerwise weights in the original and Gaussian networks have matching first two moments, so the first two Taylor terms cancel and the remainder is $V=(V_1,\dots,V_n)$8, which after summing over $V=(V_1,\dots,V_n)$9 terms gives $\mathbb{R}^n$0 per layer (Giovagnini et al., 4 May 2026). In that sense, moment matching converts a qualitative exchange argument into a quantitative one.

The same structural idea persists when the one-step estimate is not written as an ordinary Euclidean Taylor formula. In stable approximation, the expansion is “Taylor-like” and the compensating term is expressed through the generator $\mathbb{R}^n$1 of the stable Lévy process rather than the Gaussian second derivative (Chen et al., 2018). In the total-variation central limit theorem, the object being replaced is a Stein kernel contribution $\mathbb{R}^n$2 by its mean $\mathbb{R}^n$3, and the telescoping is carried out inside Stein’s identity (Dung et al., 4 Nov 2025). A plausible implication is that the decisive feature is not the literal form of the expansion, but the existence of a one-step approximation whose leading part is additive and whose error is summable.

3. Stable-law approximation and the generator method

"Approximation to the stable law by Lindeberg principle" develops the method for one-dimensional possibly asymmetric $\mathbb{R}^n$4-stable laws with $\mathbb{R}^n$5 (Chen et al., 2018). For fixed $\mathbb{R}^n$6, $\mathbb{R}^n$7, and when $\mathbb{R}^n$8 with $\mathbb{R}^n$9, a real random variable $f:\mathbb{R}^n\to\mathbb{R}$0 has the law $f:\mathbb{R}^n\to\mathbb{R}$1 if its characteristic function is

$f:\mathbb{R}^n\to\mathbb{R}$2

with the usual log-correction form when $f:\mathbb{R}^n\to\mathbb{R}$3. Equivalently, $f:\mathbb{R}^n\to\mathbb{R}$4 is the one-dimensional Lévy process at time one with generator

$f:\mathbb{R}^n\to\mathbb{R}$5

The comparison is performed in the smooth Wasserstein distance. For $f:\mathbb{R}^n\to\mathbb{R}$6,

$f:\mathbb{R}^n\to\mathbb{R}$7

and

$f:\mathbb{R}^n\to\mathbb{R}$8

The random variables $f:\mathbb{R}^n\to\mathbb{R}$9 are assumed to lie in the domain of normal attraction described by

$$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$0

with $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$1 bounded and vanishing at infinity. The normalized sums are

$$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$2

where $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$3 if $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$4, and $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$5 otherwise.

The replacement framework interpolates between the normalized sum of the $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$6's and an i.i.d. sum of $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$7's. With suitable $$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$8,

$$a_i=\bigl|\mathbb{E}[U_i]-\mathbb{E}[V_i]\bigr|,\qquad b_i=\bigl|\mathbb{E}[U_i^2]-\mathbb{E}[V_i^2]\bigr|,$$9

For the $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$0-term, Lemma 3.9 gives, when $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$1,

$$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$2

with a remainder controlled by $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$3, a tail integral involving $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$4, and $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$5. For the $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$6-term, the Kolmogorov forward equation

$$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$7

produces compensating generator terms. The paper states that in the telescoping sum the forward-equation terms exactly cancel the compensating $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$8-terms from the Taylor-like expansion (Chen et al., 2018).

The resulting rates are explicit. Under $$\max_i\{\mathbb{E}|U_i|^3+\mathbb{E}|V_i|^3\}\le M_3<\infty$$9 for large $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$0, there is $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$1 such that for every $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$2,

$$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$3

In particular,

$$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$4

with the paper also giving $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$5 for $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$6 and $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$7 for $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$8. The paper further states that it is the first time that the general stable central limit theorem is proved by the Lindeberg principle, and that the theorem with $$\bigl|\partial_i^r f(x)\bigr|\le L_r(f)\qquad \forall x\in\mathbb{R}^n,\;i=1,\dots,n,\;r=1,2,3.$$9 is proved by a new method other than Fourier analysis (Chen et al., 2018).

4. Central-limit theorems in total variation

"Total variation bounds in the Lindeberg central limit theorem" reworks the replacement idea in a non-i.i.d. Gaussian approximation problem, but with total-variation distance rather than a smooth test-function metric (Dung et al., 4 Nov 2025). Let

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$0

where the $$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$1 are independent, centered, and absolutely continuous. The total-variation distance from $$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$2 is

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$3

The theorem assumes finite standardised Fisher information

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$4

sets $$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$5, $$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$6, and $$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$7. Under these hypotheses,

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$8

Equivalently,

$$\bigl|\mathbb{E}[f(U)]-\mathbb{E}[f(V)]\bigr| \le \sum_{i=1}^n\Bigl(a_iL_1(f)+\tfrac12 b_iL_2(f)\Bigr) +\tfrac16 nL_3(f)M_3.$$9

The proof combines Stein’s equation with a Lindeberg-style telescoping. For each bounded $(U_{1:i-1},s,V_{i+1:n})$0, $(U_{1:i-1},s,V_{i+1:n})$1 solves

$(U_{1:i-1},s,V_{i+1:n})$2

and Lemma 2.1 yields $(U_{1:i-1},s,V_{i+1:n})$3 and $(U_{1:i-1},s,V_{i+1:n})$4. Each $(U_{1:i-1},s,V_{i+1:n})$5 is equipped with a Stein kernel $(U_{1:i-1},s,V_{i+1:n})$6, and the replacement step consists of replacing $(U_{1:i-1},s,V_{i+1:n})$7 by its mean $(U_{1:i-1},s,V_{i+1:n})$8. Inserting this into Stein’s identity produces

$(U_{1:i-1},s,V_{i+1:n})$9

The paper explicitly describes this as the classical telescoping argument of Lindeberg, but carried out in the language of Stein kernels (Dung et al., 4 Nov 2025).

A central consequence is an exact equivalence statement under bounded Fisher information: if $\partial_i^3 f$0, then the usual Lindeberg condition

$\partial_i^3 f$1

together with the Feller condition $\partial_i^3 f$2, is equivalent to $\partial_i^3 f$3 (Dung et al., 4 Nov 2025). The paper therefore states that, under suitable assumptions, Lindeberg’s condition is sufficient and necessary for convergence in total variation.

5. Layerwise exchange in deep neural networks

In "Universality in Deep Neural Networks: An approach via the Lindeberg exchange principle," the method is adapted to the infinite-width limit of fully connected deep networks with general weights (Giovagnini et al., 4 May 2026). The setting fixes depth $\partial_i^3 f$4 and widths $\partial_i^3 f$5, and defines switched-layers networks $\partial_i^3 f$6 whose first $\partial_i^3 f$7 layers use the original weights $\partial_i^3 f$8 and whose last $\partial_i^3 f$9 layers use independent Gaussian matrices $V=(V_1,\dots,V_n)$00 with the same variance $V=(V_1,\dots,V_n)$01.

The core weak comparison theorem states that if

$V=(V_1,\dots,V_n)$02

and if $V=(V_1,\dots,V_n)$03 and $V=(V_1,\dots,V_n)$04 have the same architecture, same biases, and same activation $V=(V_1,\dots,V_n)$05, but the hidden weights in $V=(V_1,\dots,V_n)$06 satisfy Assumption 2.1 while those in $V=(V_1,\dots,V_n)$07 are Gaussian with matching first two moments, then for every

$V=(V_1,\dots,V_n)$08

one has

$V=(V_1,\dots,V_n)$09

If instead one assumes Gaussian biases and only $V=(V_1,\dots,V_n)$10, then the same bound holds for any bounded $V=(V_1,\dots,V_n)$11, or for $V=(V_1,\dots,V_n)$12 with error controlled by $V=(V_1,\dots,V_n)$13 (Giovagnini et al., 4 May 2026).

The telescoping decomposition is layerwise: $V=(V_1,\dots,V_n)$14 Conditioning on the pre-activations of the relevant layer, one views the next layer as a sum of independent random variables and applies a conditional Taylor expansion up to third order plus moment matching to obtain

$V=(V_1,\dots,V_n)$15

The paper describes each layer $V=(V_1,\dots,V_n)$16 as contributing a “replacement error”

$V=(V_1,\dots,V_n)$17

and these add up linearly (Giovagnini et al., 4 May 2026).

After all layers are replaced, one compares the fully Gaussian network to the Gaussian-process limit in $V=(V_1,\dots,V_n)$18. Under Assumptions 2.1, the stated activation regularity, and invertibility of the limiting covariances $V=(V_1,\dots,V_n)$19,

$V=(V_1,\dots,V_n)$20

If all hidden layers have the same width $V=(V_1,\dots,V_n)$21, then

$V=(V_1,\dots,V_n)$22

uniformly in $V=(V_1,\dots,V_n)$23 and $V=(V_1,\dots,V_n)$24 up to the constant $V=(V_1,\dots,V_n)$25 (Giovagnini et al., 4 May 2026). The role of Gaussian biases is explicitly that their smoothing effect reduces the derivative requirement on $V=(V_1,\dots,V_n)$26 from $V=(V_1,\dots,V_n)$27 bounded derivatives to only $V=(V_1,\dots,V_n)$28.

6. Universality, sparse limits, and algebraic variants

The method is used in (Korada et al., 2010) to prove universality properties in communications, statistical learning, and random matrix theory, and also to show that dense systems can be viewed as limits of properly defined sparse systems. The paper states universality and sparse–dense equivalence results for CDMA capacity, LASSO, Wishart spectra, and MIMO capacity. In each case the proof strategy is to express the quantity of interest as $V=(V_1,\dots,V_n)$29 for a smooth $V=(V_1,\dots,V_n)$30, verify bounds on its third derivatives, and apply the modified Lindeberg theorem. For example, for standard-type spreading matrices $V=(V_1,\dots,V_n)$31,

$V=(V_1,\dots,V_n)$32

and analogous convergence statements are given for LASSO and for the Stieltjes transform of the Wishart ensemble (Korada et al., 2010).

The same paper formulates a sparse-limit viewpoint by defining a $V=(V_1,\dots,V_n)$33-sparsification $V=(V_1,\dots,V_n)$34 through

$V=(V_1,\dots,V_n)$35

It then states that if the third partial derivatives of $V=(V_1,\dots,V_n)$36 are $V=(V_1,\dots,V_n)$37, replacing a nonzero by an independent Gaussian or another i.i.d. entry induces an error of order $V=(V_1,\dots,V_n)$38, so over $V=(V_1,\dots,V_n)$39 nonzeros the total error is $V=(V_1,\dots,V_n)$40 (Korada et al., 2010). This suggests a form of universality obtained by first passing to analytically simpler sparse models and then returning to dense ones.

"Universality results for random matrices over finite local rings" shows that the replacement method also has a discrete and algebraic form (Lvov, 15 Jan 2026). Here the random object is a matrix $V=(V_1,\dots,V_n)$41 over a finite local ring $V=(V_1,\dots,V_n)$42, and the comparison matrix $V=(V_1,\dots,V_n)$43 has independent Haar-uniform entries. The replacement is not coordinatewise in $V=(V_1,\dots,V_n)$44 but columnwise. For a fixed column $V=(V_1,\dots,V_n)$45, if $V=(V_1,\dots,V_n)$46 uses the original $V=(V_1,\dots,V_n)$47-th column $V=(V_1,\dots,V_n)$48 and $V=(V_1,\dots,V_n)$49 uses an independent Haar-uniform column $V=(V_1,\dots,V_n)$50, then

$V=(V_1,\dots,V_n)$51

where $V=(V_1,\dots,V_n)$52 are the laws of $V=(V_1,\dots,V_n)$53. A central inequality bounds, for any signed measure $V=(V_1,\dots,V_n)$54 on $V=(V_1,\dots,V_n)$55,

$V=(V_1,\dots,V_n)$56

Applying this with $V=(V_1,\dots,V_n)$57 gives a cost $V=(V_1,\dots,V_n)$58 for a single-column swap, and summing over all columns yields

$V=(V_1,\dots,V_n)$59

The same argument extends to the span of the column-space and to the joint law of $V=(V_1,\dots,V_n)$60 (Lvov, 15 Jan 2026).

Taken together, these examples show that the Lindeberg replacement method is not tied to a single ambient space, metric, or analytic technology. In the cited works, the exchange step is implemented by ordinary Taylor expansion, a Taylor-like generator expansion, a Stein-kernel substitution, Gaussian smoothing, or a column-swapping argument with Fourier-isotypic decomposition. What remains invariant is the telescoping architecture: a complex comparison is reduced to a sequence of one-step replacements whose errors can be summed.