The IQ-Motion Confound in Multi-Site Autism fMRI May Be Inflated by Site-Correlated Measurement Uncertainty

Abstract: Multi-site autism neuroimaging studies routinely control for the confound between full-scale IQ and head motion by regressing framewise displacement against IQ scores and removing shared variance. This procedure assumes that ordinary least squares (OLS) provides an unbiased estimate of the confound magnitude. We tested this assumption on the ABIDE-I phenotypic dataset (n=935 subjects across 19 international scanning sites) using Probability Cloud Regression, an errors-in-variables (EIV) estimator that models per-observation measurement uncertainty in both variables. IQ measurement error was derived from published Wechsler test-retest reliability coefficients; response-side uncertainty was represented by a site-level proxy equal to the within-site standard deviation of mean framewise displacement. Three findings emerged. First, OLS overestimates the IQ-motion slope by a factor of 4.67 relative to the EIV-corrected estimate when the bias factor is computed from the full-precision fitted coefficients (OLS -0.00125, EIV -0.00027 mm per IQ point after rounding for display). Second, under leave-site-out cross-validation a single pooled predictor of raw FD produces negative out-of-sample R^2 at all 19 sites (overall R^2 = -0.074), indicating that the pooled predictor does not transport cleanly across sites once site information is removed. Third, the direction of the EIV-corrected slope is robust across all 64 configurations of an 8x8 sensitivity grid spanning 12-fold ranges of each noise parameter. These results suggest that pooled OLS may overstate the IQ-motion association in ABIDE-I, but direct downstream consequences for motion-correction pipelines remain to be quantified using raw motion traces and connectivity-level re-analysis. Formal EIV methods appear to remain uncommon in multi-site neuroimaging confound estimation.

• The paper demonstrates that OLS overestimates the IQ–motion association by nearly five times compared to the PCR method.
• It employs an EM-based PCR approach to jointly model measurement uncertainty, revealing the limitations of pooled confound estimates.
• Leave-site-out cross-validation and sensitivity analyses indicate that site-level noise critically drives inflated confound estimates in multi-site studies.

Measurement-Error Inflation of the IQ–Motion Confound in Multi-Site Autism fMRI

Introduction

This paper systematically deconstructs a prevailing statistical assumption in multi-site neuroimaging studies of autism: that ordinary least squares (OLS) regression yields an unbiased estimate of the confound between full-scale IQ and head motion. The author demonstrates that, in the case of the ABIDE-I dataset, measurement error that co-varies with site-level noise robustly inflates the pooled OLS association between IQ and head motion—as measured by mean framewise displacement (FD)—by a factor of nearly five. The study situates itself at the intersection of multi-site studies, measurement theory, and confound adjustment, and proposes a formal errors-in-variables approach, Probability Cloud Regression (PCR), as a corrective.

Methods and Measurement Uncertainty Model

Exploiting the phenotypic ABIDE-I dataset ($n=935$​, 19 sites), the author implements PCR, an EM-based errors-in-variables estimator, to jointly model measurement uncertainty for both variables. IQ measurement error for each subject is derived from age- and test-appropriate Wechsler test-retest reliability coefficients. For mean FD, response-side uncertainty is proxied by the within-site standard deviation of mean FD, yielding site-specific $\sigma_y$ estimates.

PCR estimates the latent true variables for each subject by treating every observation as a 2D Gaussian with known measurement variances, iteratively optimizing regression parameters via the EM framework. When uncertainties are homoscedastic, this method reduces to classical Deming regression; in the present heteroscedastic context, it generalizes the approach and more realistically accommodates site-level error structures.

Leave-site-out (LOSO) cross-validation is adopted to assess the portability of the pooled confound estimate, a critical test given site-level heterogeneity and the risk of information leakage in conventional $k$-fold CV.

Main Results

Three core findings emerge:

1. Substantial OLS Overestimation: The OLS estimate of the IQ–motion slope ($-0.00125$ mm per IQ point) is 4.67 times larger in magnitude than the EIV-corrected PCR slope ($-0.00027$ mm per IQ point). Both are negative, confirming that higher IQ is associated with lower mean FD, but the effect is dramatically overstated by OLS.

   Figure 2: Scatter plot of all 935 ABIDE-I subjects with OLS and PCR EIV-corrected regression lines, colored by motion tier.

2. Non-Transportability of Pooled Predictors: Under LOSO cross-validation, pooled regression predictors of raw FD fail catastrophically, with negative $R^2$ at all sites ($R^2 = -0.074$ overall). The pooled model's predictions are consistently worse than a naive baseline.
3. Robustness Across Noise Parameter Space: Sensitivity analysis across all plausible values of IQ and FD uncertainty confirms both the sign and inflation magnitude of the OLS bias are stable and not dependent on parameter tuning.

   Figure 4: Sensitivity analysis showing OLS slope overestimation vs. PCR across 64 $(\sigma_x, \sigma_y)$ settings; effect is robust to uncertainty in measurement error assumptions.

Additionally, analysis of site tiers sorted by mean FD shows that the apparent IQ–motion association is concentrated in high-motion, high-uncertainty sites, with minimal or zero association in high-precision, low-motion sites. This site-level structure drives OLS inflation.

Figure 1: Site-level IQ-motion slope versus site mean framewise displacement, with bubble size proportional to $\sqrt{n}$.

Figure 3: Within-tier OLS slopes contrasted with pooled OLS and PCR estimates.

Discussion and Interpretation

Reverse Attenuation Mechanism

Contrary to the “attenuation toward zero” predicted by classical errors-in-variables theory, ABIDE-I exhibits “reverse attenuation”: OLS exaggerates the magnitude of the IQ–motion association. This reversal arises because site-level noise proxies ($\sigma_y$) and apparent slope magnitude are positively correlated across sites; noisy, high-motion sites dominate the pooled fit. Consequently, OLS combines inflated slopes from imprecise sites, pulling the group estimate away from the more reliable within-site associations.

Implications for Confound Correction Pipelines

The finding that OLS-based confound estimates can be so profoundly inflated has immediate methodological implications. If multi-site neuroimaging pipelines use pooled OLS slopes to calibrate downstream corrections (e.g., regressing out motion confounds from fMRI connectivity measures), such corrections may be over-aggressive. This is particularly likely in pipelines that do not explicitly model measurement error or site-by-site heterogeneity, and could lead to over-removal of neural signal in high-precision, low-motion sites.

The LOSO CV analysis demonstrates that pooled confound metrics do not generalize across sites—highlighting the risk of assuming global transportability in multi-site settings where measurement reliability and effect structure are themselves site-dependent.

Relationship to Analytic Variability and Harmonization

The bias source identified here is orthogonal to analytic flexibility assessed in many-analyst studies (e.g., Botvinik-Nezer et al.), affecting confound estimation even if analysis pipelines are held constant. Harmonization protocols (ComBat and derivatives) address site effects in derived measures, not regression-based confound estimates, and are thus complementary but insufficient to resolve the issue highlighted here.

Limitations

The PCR method depends on proxy noise estimates for FD (using within-site SD of mean FD), which may mix true subject variability with measurement error, and works at the phenotypic summary level rather than individual time-series. Separate estimation for ASD and control subsets is not provided. Generalization to other datasets and to connectivity-level consequences remains to be established.

Future Directions

Further work should extend PCR and EIV modeling to datasets with raw, time-resolved motion traces, enabling more precise subject-level noise models. An immediate priority is re-estimation of group differences in functional connectivity using EIV-corrected confound parameters to directly quantify the magnitude of downstream overcorrection attributable to OLS inflation. Application to other large-scale data resources (ABIDE-II, UK Biobank, HCP) will test the generality of the reverse attenuation regime.

Conclusion

This study demonstrates that, in multi-site autism fMRI datasets, pooled OLS estimation of the IQ–motion confound can be substantially inflated by site-correlated measurement uncertainty. The use of formal errors-in-variables methods such as PCR reveals that the true association is considerably weaker, and that pooled predictors of motion confounds do not generalize across sites. Methodological vigilance for measurement error and site heterogeneity in confound estimation is required for accurate correction and valid inference in large-scale neuroimaging.