Correlation Neglect: Theory & Practice
- Correlation Neglect is defined as the systematic misperception or simplification of dependency in joint distributions, where outcomes are treated as independent despite underlying correlations.
- It is formalized by contrasting true joint evaluations with product-of-marginals evaluations using relaxed axioms that modify traditional independence assumptions.
- The concept applies across decision theory, human–AI collaboration, anomaly detection, and cosmological inference, influencing welfare outcomes and parameter estimates.
Searching arXiv for recent and foundational papers on correlation neglect across decision theory, games, optimization, anomaly detection, cosmology, and human–AI collaboration. Correlation neglect denotes the omission, misperception, or deliberate simplification of dependence structure when outcomes, signals, samples, or observations are jointly distributed. In decision theory, the canonical form is the evaluation of a lottery through its marginals as if the sources were independent, formalized by the axiom and the representation (Zhang, 2021). In adjacent work on correlation uncertainty, the same phenomenon appears as a restriction of admissible priors to the product measure , or to faces of the multi-marginal polytope enforcing blockwise independence (Bauch et al., 17 Mar 2025). In strategic settings, machine learning, optimization, and scientific inference, the term extends to failures to account for dependence among observed actions, training samples, demand components, or galaxy-pair separations, with consequences that range from overprecision and welfare loss to biased parameter estimates and degraded anomaly detection (Mudekereza, 22 Jan 2025, Fan et al., 2020, 0902.1792, Spezzati et al., 2024).
1. Decision-theoretic meaning and formal definitions
In the modern decision-theoretic literature, correlation neglect is defined on a domain with two sources of risk, , and lotteries . The key contrast is between the true joint distribution and the product of marginals . Under expected utility, evaluation uses the true joint:
Under correlation neglect with expected utility, the decision maker preserves the aggregator but replaces the joint by the product of marginals:
The behavioral content is therefore not source-by-source evaluation, but global evaluation under a misspecified belief that the sources are independent (Zhang, 2021).
This differs sharply from narrow bracketing. Narrow bracketing first reduces each source to a certainty equivalent,
0
and then evaluates the vector of certainty equivalents:
1
Narrow bracketing therefore ignores dependence because it compresses each marginal separately; correlation neglect ignores dependence because it treats the joint lottery “as if” it were the product of marginals. The two heuristics can coincide on product lotteries, but they differ off the product domain 2 (Zhang, 2021).
A worked binary-gamble example makes the distinction operational. With 3, 4, and 5 for 6, perfect positive correlation yields 7, whereas 8; perfect negative correlation yields 9, while 0 remains 1. Correlation neglect thus overvalues positively correlated compound risk by imputing diversification and undervalues negatively correlated risk by imputing extra variance (Zhang, 2021).
2. Axiomatization, weakened independence, and correlation uncertainty
The axiomatization of correlation neglect proceeds by relaxing the vNM independence axiom rather than abandoning regularity. The relevant conditions are Weak Order, Monotonicity, Weak Continuity, Weak Independence, and the axiom of Correlation Neglect,
2
Under these conditions, preferences admit one of three representations: EU-CN, GBIB-CN, or GFIB-CN. The open sets 3 in the generalized piecewise forms are unique; the single-source indices 4 are unique up to positive affine transformations; and in EU-CN, 5 is unique up to positive affine transformations (Zhang, 2021).
If correlation neglect is dropped and replaced by correlation-sensitive restrictions, the admissible representations expand. Under Correlation Consistency, the characterization includes EU, BIB, EU-CN, GBIB-CN, and GFIB-CN; under Forward Correlation Consistency, the symmetric class includes FIB instead of BIB. The conceptual point is that dependence-sensitivity can be restored without returning to full vNM independence: independence is required only where mixtures do not alter the correlation structure (Zhang, 2021).
A closely related but distinct framework studies correlation uncertainty over a Cartesian product of probability spaces. For finite 6, the set of all joint distributions consistent with fixed marginals is
7
This set is a compact convex polytope with
8
and 9 is an interior point if each 0 has full support. Its extreme points are characterized equivalently as maximally zero distributions and as local maximizers of mutual information 1, while 2 is the unique minimizer of 3 (Bauch et al., 17 Mar 2025).
In that framework, correlation neglect appears as a restriction of the prior or prior set to the independence benchmark. Under SEU, Subspace Independence is equivalent to 4. Under MEU, neglect corresponds to a prior set 5 that omits correlated couplings and contains only 6, or only priors in 7 enforcing independence across chosen blocks 8 (Bauch et al., 17 Mar 2025). This suggests a unifying interpretation: the classical axiom 9 and the modern polytope-based restriction 0 formalize the same cognitive simplification at different levels—choice over lotteries in one case, admissible beliefs over couplings in the other.
3. Strategic interaction, social learning, and welfare effects
In games and social learning, correlation neglect is the failure to account for dependence among observed actions. The baseline environment has a unit-mass continuum of agents with private type 1, binary actions 2, and utility from 3 given by
4
where 5 is the fraction choosing 6, and 7 is increasing and continuous. Each agent observes 8 peers’ actions, forms an estimate 9 from the sample average 0, and chooses 1 iff 2 (Mudekereza, 22 Jan 2025).
The data-generating process is explicitly correlated. In the Bernoulli formulation,
3
so 4 for 5. As 6, the variance of the sample mean does not vanish:
7
Agents who infer as if the sample were iid therefore become overprecise: they understate posterior variance and react too strongly to redundant evidence (Mudekereza, 22 Jan 2025).
The equilibrium concept capturing this misspecification is correlated sampling equilibrium with statistical inference, or CoSESI. Let
8
and
9
Then 0 is the unique solution of
1
Existence and uniqueness hold for any 2, 3, and inference procedure 4 increasing in 5 (Mudekereza, 22 Jan 2025).
The consequences are persistent. If 6, best responses remain noisy even with large samples; CoSESI generally does not converge to Nash equilibrium as 7; and for 8, polarization becomes extreme because only 9 matters. With convex 0 and unbiased 1, 2, so welfare is lower under CoSESI than under Nash equilibrium. Comparative statics are similarly sharp: higher 3 worsens outcomes, while larger 4 can help under noise reduction but does not eliminate the distortion if 5 (Mudekereza, 22 Jan 2025).
The model is applied to matching markets, monopoly pricing, and financial markets. In matching, participation and employment fall with 6 and 7. In monopoly pricing, profit in CoSESI exceeds NE profit for all 8 and increases with 9. In financial markets, neglect of borrower-default correlation leads banks to underestimate tail risk, and as 0, the equilibrium fraction continuing can collapse toward crisis values (Mudekereza, 22 Jan 2025).
4. Human–AI collaboration and overlap-driven misaggregation
A recent Bayesian treatment places correlation neglect at the center of human–AI collaboration. The human observes a private signal 1, receives an AI signal 2, and chooses a decision 3 under quadratic loss 4. In the Gaussian specification,
5
with conditional dependence measured by the overlap coefficient
6
In the micro-foundation based on primitive cues, 7, and in the heterogeneous-precision version 8 (Amin et al., 15 Feb 2026).
The Bayesian benchmark orthogonalizes the AI signal. There exists an innovation signal
9
with 0 and
1
The correct posterior mean is
2
Correlation neglect replaces this by the misspecified independent-signal rule
3
which uses 4 as if all of 5 were novel and thereby double-counts the redundant component 6 (Amin et al., 15 Feb 2026).
The expected loss under correlation neglect has a closed form:
7
The first term is the loss under genuine conditional independence; the second is the overlap penalty. This decomposition yields a regime map. Augmentation means 8, impairment means 9, complementarity means 00, and automation means 01. The augmentation threshold is
02
so if 03, AI assistance helps for all 04, whereas if 05, sufficiently weak AI can harm performance. The automation threshold
06
is strictly decreasing in 07 (Amin et al., 15 Feb 2026).
A broader decomposition clarifies the source of these regimes. The realized gain from AI assistance equals the Bayesian marginal informational value 08 minus a behavioral penalty measuring the divergence between the optimal and actual actions. Under correct aggregation, 09 is strictly increasing in 10 and strictly decreasing in 11; failures of augmentation or complementarity therefore arise from misaggregation rather than from the absence of informational value per se (Amin et al., 15 Feb 2026).
5. Machine learning: neglect of inter-sample correlation in anomaly detection
In unsupervised anomaly detection, correlation neglect denotes the treatment of each sample 12 independently, without modeling how samples relate to one another. The motivating claim is that normal data typically lie on structured manifolds or clusters where nearby samples share attributes, so neglecting complex correlation among data samples suppresses information about how normal points co-occur and vary jointly. The proposed remedy in “Correlation-aware Deep Generative Model for Unsupervised Anomaly Detection” is CADGMM, which integrates graph construction, dual encoding, feature reconstruction, and latent-density estimation (Fan et al., 2020).
Given samples 13, CADGMM builds an undirected 14-NN graph with binary adjacency
15
Correlation strengths are then learned by a graph attention mechanism rather than fixed by hand-crafted edge weights:
16
A feature MLP produces 17, the graph encoder produces 18, and fusion is defined by element-wise addition followed by a linear layer,
19
A decoder reconstructs 20, and a separate estimation network fits a GMM to
21
where 22 stacks reconstruction-error features such as Euclidean and cosine errors (Fan et al., 2020).
Anomaly scoring is energy-based. The estimation network outputs mixture memberships 23, from which
24
are estimated end-to-end. The energy is the negative log-likelihood under the learned GMM:
25
Training minimizes reconstruction loss, energy, covariance regularization, and latent-norm regularization (Fan et al., 2020).
Empirically, CADGMM improves F1 over baselines that largely neglect inter-sample correlations. On KDD99 10%, CADGMM achieves F1 26 with Precision 27 and Recall 28, compared with DAGMM 29 and ALAD 30. On Arrhythmia, CADGMM attains F1 31, compared with IF 32 and DAGMM 33. On Satellite, CADGMM reaches F1 34, compared with DAGMM 35 and ALAD 36. Under 1–5% anomaly injection into KDD99 training data, CADGMM F1 remains 37, whereas DAGMM drops 38 and OC-SVM 39. The paper attributes these gains to attention-based neighborhood aggregation that aligns the latent representation with manifold neighborhoods of normal samples (Fan et al., 2020).
6. Robust optimization, cosmological inference, and the scope of acceptable neglect
In stochastic optimization, correlation neglect takes the form of solving under the independent product distribution when only marginals are known. Let 40 be a random binary demand vector and 41 the corresponding random subset. The robust objective minimizes worst-case expected cost over the polytope of distributions with prescribed marginals,
42
whereas the independent model minimizes 43 under independent Bernoulli marginals. The correlation gap at fixed 44 is
45
with 46 a worst-case distribution (0902.1792).
The main theorem identifies when neglecting correlation is a controlled approximation. If, for every feasible 47, 48 is non-decreasing in 49 and admits an 50-cost-sharing scheme, then
51
For monotone submodular 52, 53, giving the tight bound 54. For stochastic uncapacitated facility location, the gap is 55; for stochastic Steiner tree, 56. But the paper also shows that neglect can be catastrophic: for supermodular functions, the gap can be 57, and for monotone subadditive functions, 58 (0902.1792). Correlation neglect is therefore not uniformly irrational; its validity depends on the structure of the cost function.
In cosmological large-scale-structure inference, the phrase designates a different but technically analogous simplification: neglect of wide-angle terms, radial modes, and unequal-time correlations in the galaxy two-point correlation function. The full 3D wide-angle model keeps the exact geometry and redshift evolution,
59
with 60 mapped from the triangle variables by
61
The flat-sky, single-62 approximation compresses all pairs to an effective redshift and ignores opening-angle dependence; a partial improvement averages flat-sky kernels over the redshift bin; and the proposed 3D-radial model neglects purely wide-angle angular couplings but retains full 3D radial information and unequal-time correlations (Spezzati et al., 2024).
The quantified effect is a parameter-estimation bias that grows with bin thickness and the maximum scale included. For SPHEREx-like thick bins, flat-sky modeling shifts the best-fit growth index 63 by 64, and shifts in 65 are approximately within 66, reaching up to 67 in the cases considered. In nDGP forecasts, the same approximation induces 68 shifts in 69 and 70 shifts in 71. The 3D-radial model yields best fits essentially aligned with the full 3D wide-angle results, while changes in error bars are marginal, with figure of merit variation below 72 across models (Spezzati et al., 2024).
Taken together, these results delimit the scope of the concept. Correlation neglect may be a behavioral axiom, a prior restriction, an approximation scheme, or a modeling deficiency, but in every case the technical issue is the same: a joint structure is replaced by a product, a reduced face of a coupling polytope, or an equal-time/effective-redshift surrogate. Whether this is benign, useful, or severely misleading depends on the geometry of the underlying dependence, the decision rule applied to it, and the extent to which downstream inference is sensitive to redundancy, multimodality, or unequal-time structure.