Aggregation Bias & Non-Collapsibility

Updated 7 May 2026

Aggregation bias and non-collapsibility are phenomena where aggregating data over heterogeneous groups leads to systematic distortions in association measures.
These concepts reveal why measures like the odds ratio may display reversal effects (Simpson’s paradox) and emphasize the need for appropriate standardization.
They inform the development of robust meta-analytic, recalibration, and hierarchical aggregation methods to mitigate bias and improve predictive accuracy.

Aggregation bias and non-collapsibility are structural phenomena arising whenever summarizing or aggregating data over heterogeneous groups introduces systematic distortion into measures of association or estimated effects. These concepts underpin core interpretational issues in applied statistics, epidemiology, and causal inference, and are inherent to both conventional estimators (such as the odds ratio under marginalization) and modern algorithmic aggregation (such as recursive majority-vote in hierarchical classification). Their study reveals which summaries are robust to aggregation, clarifies when “Simpson’s paradox” may arise, and dictates the appropriate analytic strategies for unbiased effect estimation and valid prediction.

1. Formal Definitions and Core Criteria

Collapsibility is the property that the marginal measure of association equals a specified function of the stratum-specific measures whenever the stratum-specific effect is homogeneous. For a binary exposure $X \in \{0,1\}$ , binary outcome $D \in \{0,1\}$ , and covariate $C$ not affected by $X$ , denote stratum-specific risks as $p_0(c) = \Pr(D=1|X=0,C=c)$ and $p_1(c) = \Pr(D=1|X=1,C=c)$ . Define standardized risks under a fixed distribution $\Prst(C)$:

$p_0^* = \sum_c p_0(c)\,\Prst(C=c), \qquad p_1^* = \sum_c p_1(c)\,\Prst(C=c).$

A measure of association $M$ is collapsible if for all $c$ ,

$D \in \{0,1\}$ 0

This criterion formalizes collapsibility as a property of $D \in \{0,1\}$ 1 itself, independent of any particular confounder (Kenah, 30 Jun 2025).

Non-collapsibility occurs when, despite homogeneity across strata, the marginal measure fails to coincide with the common stratum-specific value under any weighting scheme:

$D \in \{0,1\}$ 2

This property is intrinsic to the functional form of $D \in \{0,1\}$ 3 and can be rigorously formalized in the counterfactual framework (Huitfeldt et al., 2016).

Aggregation bias is the systematic difference between a marginal (or coarsened, recursively aggregated) measure and the corresponding stratum-specific or non-coarsened measure, even in the absence of confounding—arising solely from the non-collapsibility of the measure or the structure of the aggregation procedure (Kenah, 30 Jun 2025, Galam, 26 Jun 2025).

2. Geometric and Mathematical Characterization

The geometry of collapsibility is captured by Rothman diagrams: in the unit square $D \in \{0,1\}$ 4, contours $D \in \{0,1\}$ 5 represent equal values of the association measure. The key criterion is:

A measure of association is collapsible if and only if all of its contour lines are straight segments in the $D \in \{0,1\}$ 6 plane (Kenah, 30 Jun 2025).

For classical measures:

Risk difference: $D \in \{0,1\}$ 7 ( $D \in \{0,1\}$ 8), corresponding to parallel straight lines.
Risk ratio: $D \in \{0,1\}$ 9 ( $C$ 0), corresponding to rays through the origin.
Odds ratio: $C$ 1, yielding curves: $C$ 2, which are strictly convex/concave except at the null.

For risk difference and risk ratio, convex combinations of stratum-specific $C$ 3 that share a value of $C$ 4 remain on the same contour—standardization preserves the measure. For the odds ratio, the standardized point formed from strata lying on the same contour departs from the contour, leading to attenuation towards the null (non-collapsibility) (Kenah, 30 Jun 2025).

3. Specific Measures, Structural Bias, and Simpson’s Paradox

Measures of association behave as follows regarding collapsibility:

Risk difference and risk ratio are collapsible under simple aggregation schemes, with risk difference using the marginal covariate distribution and risk ratio employing the marginal distribution conditional on the unexposed (Huitfeldt et al., 2016).
Odds ratio is not collapsible under any weighting scheme: even when the stratum-specific odds ratios are identical, their marginalization produces a value strictly closer to unity (Huitfeldt et al., 2016, Kenah, 30 Jun 2025).

Simpson’s paradox (sign reversal under aggregation) is an explicit manifestation of non-collapsibility in variation-independent association measures (such as the odds ratio or log-linear interaction parameters). A parameter is directionally collapsible if the sign of the association cannot change upon aggregation; odds ratios, and any function depending only on conditionals, cannot possess this property. The only directionally collapsible association (under two mild invariance criteria) is the simple signed linear contrast of cell probabilities (Rudas, 2014).

Illustrative example (2×2 stratified table):

Two groups, both with positive conditional associations, can produce a marginal odds ratio < 1 (indicating reversal) when aggregated—classical Simpson's reversal (Rudas, 2014).
The linear contrast (DI) preserves sign across all strata and marginalizations.

4. Aggregation Bias Beyond Classical Regression: Recursive Hierarchical Aggregation

Hierarchical aggregation procedures—such as those encountered in recursive majority-vote algorithms for classification—exhibit structural non-collapsibility independent of statistical association measures (Galam, 26 Jun 2025). Consider discrete units (pixels) colored red, blue, or white. At each hierarchical level, a coarse-grain aggregate receives a macro-color only if a strict majority occurs; otherwise, it's labeled ambiguous (white).

Analytically, the probability of a white macro-aggregate increases after each application of the majority rule, inflating the proportion of ambiguous outcomes even if they are rare at the micro level. This artificial inflation exemplifies aggregation bias: it arises solely from the local coarse-graining rule and cannot be eliminated by collapsing levels in any equivalent single-step aggregation. Importantly, the procedure is structurally non-collapsible: marginal and conditional (stage-wise) computations yield systematically different results. These biases persist independent of sample size and reflect the recursive structure's built-in asymmetry (Galam, 26 Jun 2025).

5. Practical Manifestations and Estimation

a. Meta-analytic and Case-Control Settings

Non-collapsibility directly biases marginal effect estimation in meta-analysis of binary outcomes (e.g., network meta-analysis with logit links). Contrasts based on marginal odds ratios are attenuated toward the null when studies involve mixtures of risk groups. Importantly, this occurs even without confounding or effect modification: the functional form of the odds ratio suffices. The “bookend” approach, which models each study as a mixture of two latent subgroups estimated from risk extremes, mitigates this bias but requires substantive modeling assumptions regarding latent group structure and effect homogeneity (Campbell et al., 28 Feb 2026).

In case-control and outcome-dependent samples, the problem is compounded by the lack of identifiability for marginal odds ratios and risk-based measures. In such settings, geometric aggregation (i.e., using the geometric mean) of stratum-specific odds ratios restores collapsibility, producing the geometric odds ratio (GOR): $C$ 5. The GOR is collapsible under geometric aggregation and can be consistently estimated using efficient influence function-based estimators for partially identifiable settings (Coston et al., 2022).

b. Predictive Model Recalibration

In recalibrating logistic regression risk models to new populations, naively rescaling all odds multiplicatively (using a marginal odds ratio) under-corrects for population prevalence shifts—again due to non-collapsibility. Adjusting via the conditional odds ratio, which can be approximated from the mean and variance of predicted risks, yields improved calibration. This effect is pronounced when the risk score discrimination (variance) is large: the marginal approach leads to systematic under-correction (Sadatsafavi et al., 2021).

c. Algebraic Conditions for Collapsibility

Average collapsibility (weaker than pointwise/homogeneous collapsibility) holds under explicit algebraic and probabilistic criteria. For ordinary regression (linear, Poisson, logistic, negative-binomial), sufficient conditions include homogeneity of the effect or relevant conditional-independence assumptions (e.g., $C$ 6) (Vellaisamy, 2011). Violations produce systematic differences between conditional and marginal measures.

6. Logical Independence of Aggregation Bias, Non-Collapsibility, and Confounding

Association-measure modification (effect-modification), confounding, and collapsibility are logically independent:

Association-measure modification (effect modification): existence of different stratum-specific values for $C$ 7 (different contours).
Confounding: discrepancy between the crude (unadjusted) point and the standardized segment connecting stratum-specific points.
Collapsibility: geometric property (straightness of $C$ 8’s contours) independent of both (Kenah, 30 Jun 2025).

Thus, scenarios exist where a measure is collapsible but confounded, non-collapsible yet unconfounded, or effect-modified without confounding. Rothman diagrams and related geometric representations distinguish these phenomena clearly, preventing misattribution of bias sources.

7. Mitigation and Structural Recommendations

For classical estimation:

Standardization (aggregate risks then form the contrast) for any functional removes aggregation bias induced by non-collapsibility.
For the odds ratio, geometric aggregation (GOR) resolves the lack of collapsibility but may lose interpretability in specific contexts (Coston et al., 2022).

For algorithmic or recursive aggregation:

Avoid hard majority rules in hierarchical classification; prefer probabilistic or distribution-preserving aggregation to minimize bias accumulation (Galam, 26 Jun 2025).

In practice, explicit diagnostic checks for risk heterogeneity and sensitivity analyses comparing marginal and stratum-specific (or bookend-adjusted) estimates are essential, particularly in network meta-analysis or model recalibration scenarios (Campbell et al., 28 Feb 2026, Sadatsafavi et al., 2021).

In summary, aggregation bias and non-collapsibility are structural, mathematically defined phenomena that can arise even in well-designed studies without confounding and affect both statistical inference and algorithmic data reduction. Their manifestation depends critically on the properties of the association measure and the mechanics of data aggregation or standardization. Recognizing, diagnosing, and mitigating these effects are necessary for unbiased causal inference, robust association estimation, and the development of reliable aggregation procedures in both classical and modern statistical practice (Kenah, 30 Jun 2025, Huitfeldt et al., 2016, Coston et al., 2022, Rudas, 2014, Campbell et al., 28 Feb 2026, Sadatsafavi et al., 2021, Vellaisamy, 2011, Galam, 26 Jun 2025).