Pairwise Mean Independence

• Pairwise mean independence is defined as each pair of random elements having expected products equal to the product of their means, bridging full and mere decorrelation.
• It supports key limit theorems and robust statistical tests, enabling convergence results and efficient analyses in high-dimensional and heavy-tailed data.
• This property drives computational advances in representation learning, derandomization, and optimization, improving model robustness and streaming algorithm performance.

Pairwise mean independence is a probabilistic and statistical property that describes the lack of linear or mean-level dependency between pairs of random elements in a collection—generalizing classical independence, but imposing weaker constraints than full mutual independence. It is of central importance across probability theory, statistics, theoretical computer science, and modern applications such as high-dimensional data analysis, self-supervised learning, and robust optimization.

1. Mathematical Definition and Foundational Properties

Pairwise mean independence between a collection $\{X_1,\ldots,X_n\}$ is typically defined as

$$\forall i \neq j: \quad \mathbb{E}[X_i X_j] = \mathbb{E}[X_i]\mathbb{E}[X_j]$$

with a corresponding extension for vector-valued or function-valued random elements via their covariance: $\mathrm{Cov}(X_i, X_j) = 0 \quad \forall\, i\neq j$ Pairwise mean independence sits strictly between full independence (joint law is a product measure) and mere decorrelation. For instance, variables can be pairwise mean independent while remaining jointly dependent in higher-order interactions. This concept is distinguished from pairwise independence, which requires the joint law for every pair to factorize: $P_{(X_i,X_j)} = P_{X_i}\otimes P_{X_j}$.

In classic studies of weak independence conditions, several key notions are formalized:

• Full independence (IND)
• Pairwise independence (PWI)
• Non-positive correlation (NOP): $\mathrm{Cov}(I_i,I_j)\leq 0$ for event indicators $I_i, I_j$
• Pairwise mean independence (sometimes denoted as "D" in measure-theoretic treatments) (Biró et al., 2020) Pairwise mean independence often appears as a minimal sufficient condition in limit theorems, convergence arguments, and combinatorial probabilistic analysis. Its role is particularly prominent in weak converses to the Borel–Cantelli lemma and generalized strong laws of large numbers (Akhmiarova et al., 2022).

2. Theoretical Results and Limit Theorems

Pairwise mean independence is sufficient to yield several classical probabilistic results. For collections $\{A_n\}$ of events, the normalized sum $S_n = \sum_{k=1}^n I_k$ (with $I_k = \mathbb{I}_{A_k}$) will obey the strong law: $$X_n = \frac{S_n}{\mu_n} \xrightarrow{a.s.} 1 \quad\text{as } n\to\infty, \ \mu_n = \mathbb{E}[S_n]$$ provided $\sum_n \mathbb{P}(A_n)$ diverges and the family is pairwise mean independent or at least NOP (Biró et al., 2020).

A further relaxation is shown to be achievable where only pairwise independence (not full independence) is available, and even an infinite subset of summands may have infinite expectation—provided their fraction is asymptotically small (Cesàro-type uniform integrability conditions): $$\lim_{n\rightarrow\infty} \frac{1}{n} \sum_{k=1}^{n} Z_k = 0 \quad\text{a.s.}$$ for mixtures $Z_k$ from two sequences, a pairwise independent collection with $\mathbb{E}[X_k]=0$ eventually and a dependent or heavy-tailed sequence occurring at vanishing frequency (Akhmiarova et al., 2022).

These generalizations are crucial for modeling and understanding systems with heavy tails, nontrivial dependence, or minimal independence structure, as frequently encountered in empirical economics, network theory, actuarial science, and database analytics.

3. Statistical Testing and High-Dimensional Data Analysis

In statistics, pairwise mean independence is a central hypothesis in tests for independence between random vectors, especially in high dimensions. Key advances have been made for settings where one dimension ($p_1$ or $p_2$) is large compared to sample size, a scenario prevalent in genomics or image analysis (Li et al., 2015). The trace-based statistic

$$T_n = \frac{n\,\hat \gamma_{xy}}{\sqrt{2k_n\, \hat \gamma_{xx}\hat \gamma_{yy}}}$$

with

$$\hat \gamma_{xy} = k_n \left[ (S_{xy}S_{yx}) - \frac{1}{n} (S_{xx})(S_{yy}) \right]$$

achieves asymptotic normality, superior robustness, and higher power, outperforming likelihood ratio and canonical correlation-based approaches—even under moderate nonnormality in the data.

In streaming or online settings, algorithms based on implicit tensor representations enable estimation of deviations from mean independence (statistical distance between joint and product distributions) with polylogarithmic memory and a single pass over massive data streams (0903.0034). The $L_1$ statistical distance: $$\Delta(P_{\text{joint}}, P_{\text{product}}) = \frac{1}{2} \|P_{\text{joint}} - P_{\text{product}}\|_1$$ is computable via certifying tournaments and recursive dimension reduction techniques, making pairwise mean independence an actionable property in real-time data architectures.

High-dimensional independence testing further exploits pairwise mean independence: rank-based U-statistics (e.g., Hoeffding’s $D$, Bergsma-Dassios-Yanagimoto’s $\tau^*$) calibrated via Cramér-type deviation theory offer distribution-free procedures, extreme value (Gumbel) calibrated thresholds, and rate-optimal detection of sparse dependencies (Drton et al., 2018). For copula-based models, Moebius-transformed empirical copulas yield tests robust to high $d > n$ regimes, with martingale CLTs guaranteeing correct asymptotic level (Bücher et al., 2022).

4. Analytical and Computational Aspects in Optimization and Learning

In theoretical computer science and optimization, the role of pairwise mean independence is prominent in the paper of constraint satisfaction problems (CSPs) and robust optimization. Approximation resistance of predicates in MAX-CSP is determined by the existence of balanced pairwise independent distributions in the solution set. Under the Unique Games Conjecture, predicates admitting a balanced pairwise independent supporting distribution satisfy

$$\Pr_{x \sim \mu}[x_i = a, x_j = b] = \frac{1}{q^2} \ \forall i \neq j, a,b \in [q]$$

are hard to approximate beyond the random assignment baseline $|P^{-1}(1)| / q^k$ (0802.2300). For monotone submodular set functions, the "pairwise independent correlation gap"—the ratio between the robust optimum under arbitrary correlations and that under pairwise independence constraints—is shown to be tightly bounded (4/3 for $n=3$ or for small/large marginals, less than $e/(e-1)$ for mutual independence), impacting both robust combinatorial optimization and auction mechanism design (Ramachandra et al., 2022).

In self-supervised representation learning, variance-covariance regularization (VCReg) is proven to enforce pairwise independence among learned features. Minimizing

$$\mathcal{L}_{VC} = \sum_{k=1}^P \max(0, 1 - \sqrt{\mathrm{Cov}(Z)_{k,k}}) + \alpha \sum_{k\neq j} \mathrm{Cov}(Z)_{k,j}^2$$

for the output $Z$ of an MLP projector leads to low Hilbert–Schmidt Independence Criterion (HSIC) values, indicative of pairwise mean independence across encoder dimensions. This is theoretically justified for wide, shallow MLPs, with experimental evidence linking pairwise-independent features to improved generalization and blind source recovery in ICA models (Mialon et al., 2022).

5. Distance Covariance and Characterization of Independence

Distance covariance (dCov) and its normalized variant, introduced by Székely, Rizzo, and Bakirov, provide a robust characterization of independence in arbitrary spaces: $$\text{dCov}(X,Y) = \mathrm{Cov}(\Delta(X,X'), \Delta(Y,Y'))$$ where

$$\Delta(X, X') = |X - X'| - E_{X''}[|X - X''|] - E_{X''}[|X'' - X'|] + E[|X'' - X'''|]$$

This doubly centered construction ensures that $\mathrm{dCov}(X,Y) = 0$ if and only if $X$ and $Y$ are independent (given finite first moments) (Raymaekers et al., 18 Jun 2024). Plain covariance of absolute differences vanishes under independence, but the converse does not hold—counterexamples illustrate the necessity of doubly centering. This framework is used in contingency table analysis, nonparametric testing, and modern courses on statistical dependence, emphasizing not just decorrelation but true pairwise mean independence.

6. Applications in Derandomization, Gibbs Measures, and Blind Source Separation

Work-efficient parallel derandomization leverages pairwise independence to simulate concentration inequalities: using random walks where steps are pairwise-independent across iterations, deterministic algorithms achieve polylogarithmic work and depth, reconstructing Chernoff-like bounds in parallel settings, with broader implications for distributed graph algorithms (Ghaffari et al., 2023).

In mean field Gibbs measures with convex pairwise interactions, approximate independence between $k$-particle marginals and the product of their limiting laws is quantified via relative Fisher information, relative entropy, and Wasserstein metrics, with an optimal $O((k/n)^2)$ rate in high temperature (small $\beta$) regimes (Lacker, 2021). This sharply characterizes propagation of chaos and links minimal pairwise mean independence assumptions to practical mixing rates in statistical mechanics.

In non-independent component analysis, pairwise mean independence yields a diagonal zero pattern in cumulant tensors, generalizing identifiability beyond classical ICA. Models imposing only pairwise mean independence remain identifiable; any further weakening loses identifiability, making pairwise mean independence the sharp threshold for robust latent variable recovery (Ribot et al., 8 Oct 2025). Empirical recovery via least-squares optimization over the orthogonal group demonstrates improved stability compared to enforcing full independence.

7. Broader Implications and Future Directions

Pairwise mean independence offers a sharp balance between tractability, identifiability, and mathematical rigor when independence is too strong an assumption. Modern theoretical advances have shown its sufficiency in law of large numbers, its necessity in robust tensor decomposition, and its optimality in high-dimensional independence testing. At the same time, practical algorithms exploiting this property are now staple in data stream processing, representation learning, algorithmic derandomization, statistical physics, and combinatorial optimization.

Current research agendas include:

• Tightening bounds and gap analyses between pairwise and mutual independence in combinatorial settings.
• Generalizing algebraic recovery techniques in non-ICA models under only pairwise mean independence constraints.
• Exploring the role of pairwise mean independence in dynamic networks, time series, and causal inference.
• Extending distance covariance construction to infinite-dimensional settings and non-Euclidean metrics.
• Developing efficient, scalable tools for independence testing in streaming and distributed systems.

Pairwise mean independence thus serves as a robust framework for theoretical, computational, and applied research across several domains where full independence is unachievable or undesirable, but selective decorrelation guarantees are fundamentally useful.