When Does Gene Regulatory Network Inference Break? A Controlled Diagnostic Study of Causal and Correlational Methods on Single-Cell Data

Abstract: Despite theoretical advantages, causal methods for Gene Regulatory Network (GRN) inference from single-cell RNA-seq data consistently fail to match or outperform correlation-based baselines in many realistic benchmarks, a persistent puzzle which casts doubt on the value of causality for this task. We argue that existing benchmarks are insufficiently controlled to answer this question because they evaluate on real or semi-real data where multiple pathologies co-occur, confounding failure modes, and obscuring the specific conditions under which different inference methods excel or fail. To address this gap, we introduce a controlled diagnostic framework that isolates seven biologically motivated pathologies (dropout, latent confounders, cell-type mixing, feedback loops, network density, sample size, and pseudotime drift) and measure how six representative methods spanning three inference paradigms degrade as each pathology intensifies. Across 6,120 controlled experiments, we find that causal methods genuinely dominate in clean and structurally favorable regimes, but specific pathologies (notably dropout and latent confounders) selectively neutralize their advantages. We further introduce an error-type decomposition that reveals methods with similar aggregate accuracy commit qualitatively different errors. To probe whether single-pathology effects persist when multiple stressors co-occur, we perform an interaction sweep over the three most impactful pathologies and find that their joint effects are sub-additive, while also exposing density-conditional cross-overs invisible to single-dial analysis. Our findings offer a nuanced understanding of when and why different methods succeed or fail for GRN inference, providing actionable insights for method development and practical guidance for practitioners.

• The paper introduces a controlled simulator that independently varies seven single-cell pathologies, revealing which conditions cause causal and correlational GRN methods to fail.
• Causal methods lead on clean and structurally challenging data: NOTEARS reaches 0.992 undirected AUPRC in clean settings, 0.953 at high network density, and 0.923 under feedback.
• Heavy dropout sharply reduces MI and GENIE3 performance, while confounding and cell-type mixing limit all methods; error-type analysis therefore provides more actionable guidance than AUPRC alone.

Motivation and problem statement

Gene Regulatory Network (GRN) inference from single-cell RNA-seq data has produced a persistent empirical puzzle: despite their theoretical advantages, causal structure-learning methods (PC, GES, NOTEARS) consistently fail to reliably outperform simple correlation-based baselines such as Pearson correlation or GENIE3 on realistic benchmarks, a pattern documented systematically by CausalBench [2501.xxxx] and geneRNIB. The authors argue that this conclusion is premature because existing benchmarks evaluate on real or semi-real data where multiple pathologies—dropout, latent confounding, cell-type heterogeneity, non-stationarity—co-occur, making it impossible to attribute failures to specific causes. Their response is not another benchmark but a controlled diagnostic framework: a synthetic simulator built on SERGIO-style linear structural causal models (SCMs) with seven independently controllable pathology "dials," swept one at a time while holding all else fixed.

Diagnostic framework

The simulator generates single-cell expression from a linear additive-noise SCM over $p = 25$ genes with $n = 800$ cells at default settings, using a DAG sampled via a binomial parent-count model with biased parent selection. The seven pathologies are: dropout (expression-dependent zero-inflation calibrated to marginal rates up to $\delta = 0.8$), latent confounders ($k$ unobserved Gaussian factors loading on ~30% of genes), cell-type mixing (a fraction of cells drawn from an independent second SCM), feedback loops (reverse edges added with probability $\phi$), network density ($\rho \in [0.05, 0.3]$), sample size ($n \in [200, 3200]$), and pseudotime drift (time-varying weight scaling along ten pseudotime chunks). A deliberate design choice is that the primary simulator is a linear Gaussian SCM—the exact assumption class of NOTEARS and GES—so any degradation observed cannot be attributed to simulator mismatch; a nonlinear tanh-SCM replication confirms qualitative robustness of conclusions.

Six methods spanning five paradigms are evaluated: Pearson correlation, mutual information (MI) with 6-bin discretization, GENIE3 (random forest feature importances), PC (Fisher-$z$ conditional independence tests), GES (greedy BIC forward search), and NOTEARS ($\ell_1$-penalized least squares with the algebraic acyclicity constraint). Evaluation uses undirected AUPRC as the primary metric (to avoid penalizing symmetric methods for missing orientation), directed AUPRC, and a novel error-type decomposition that classifies top-$K$ predictions as true, reversed, confounded, missed, or spurious relative to the ground-truth DAG.

Headline results: degradation is pathology-specific

Across 2,100 controlled experiments per SCM (7 pathologies × 5 levels × 10 replicates × 6 methods), the clean setting is emphatically not the hard case: NOTEARS achieves 0.992 undirected AUPRC and GES 0.944 under default conditions, versus 0.811–0.895 for the correlational baselines. Causal methods remain the top performer at the hardest level of six of seven pathology dimensions. Three patterns stand out:

Pathology	Pearson	MI	GENIE3	PC	GES	NOTEARS
Dropout (0→0.8)	−0.28	−0.69	−0.67	−0.41	−0.43	−0.46
Confounders (0→16)	−0.21	−0.22	−0.25	−0.28	−0.25	−0.34
Cell-type mix (0→0.5)	−0.31	−0.29	−0.37	−0.42	−0.41	−0.42
Feedback (0→0.5)	−0.18	−0.19	−0.10	−0.12	−0.09	−0.07
Density (0.05→0.3)	−0.41	−0.40	−0.34	−0.36	−0.27	−0.05
Sample size (200→3200)	+0.02	+0.04	+0.07	+0.05	+0.01	+0.02
Pseudotime (0→1.5)	−0.14	−0.09	−0.09	−0.20	−0.05	−0.05

Dropout is the exception and the most discriminating pathology. MI and GENIE3 collapse by roughly 0.7 AUPRC because equal-frequency discretization and tree splits become uninformative when ~80% of entries are zero. Pearson degrades most gracefully (−0.28); at $n = 800$0, Pearson is nominally best but only by a margin smaller than its standard error—the operative finding being that heavy dropout neutralizes structural advantages rather than reversing them. This directly explains why correlational baselines appear competitive on dropout-laden real data.

Latent confounders and cell-type mixing are great equalizers. All six methods compress into narrow bands (0.59–0.70 under confounding; 0.50–0.57 under mixing). The authors are explicit that this is a theoretical limitation, not an implementation failure: without interventional data or instruments, no observational method can separate direct effects from confounded associations when causal sufficiency is violated.

NOTEARS alone survives structural pathologies. It retains 0.923 AUPRC under maximal feedback, 0.953 at density 0.3 (where GES falls to 0.724), 0.991 at $n = 800$1, and 0.944 under pseudotime drift—settings where correlational approaches lose 0.09–0.41 AUPRC. Its advantage stems from the combination of the $n = 800$2 sparsity penalty and the global acyclicity constraint, which suppresses indirect associations and approximates the dominant acyclic subgraph under weak feedback.

Error-type decomposition

The error decomposition shows that similar aggregate AUPRC masks qualitatively different failures, with direct methodological implications. Reversal rate is the sharpest separator: NOTEARS and GES keep reversal rates below 16% across all pathologies, while symmetric methods' reversal rates match their true-positive rates—under feedback, NOTEARS achieves ~70% true and 3% reversed edges versus Pearson's ~23%/23%. Under heavy dropout, NOTEARS and GES fill all prediction slots but with 57% spurious edges ("hallucination"), whereas MI and GENIE3 produce near-empty effective prediction sets ("collapse"). Under high density, GES accumulates ~62% confounded predictions while NOTEARS maintains 53% true edges at the cost of more misses. Two methods with identical AUPRC therefore require different fixes—a point the authors argue should change evaluation practice beyond scalar metrics.

Pathology interactions

A factorial sweep over dropout × confounders × density (64 joint regimes, 5 seeds each) yields two findings. First, joint effects are markedly sub-additive: the drop from the cleanest to the worst joint cell is smaller than the sum of single-axis drops for every method, with interaction terms ranging from −0.96 (MI) and −0.86 (GENIE3)—already saturated by dropout alone—to −0.29 (NOTEARS). Second, the winner map exposes a density-conditional cross-over invisible to single-dial analysis: NOTEARS owns the low-dropout half of the grid across all densities, but at $n = 800$3, GES displaces Pearson across the entire heavy-dropout row, consistent with BIC-based forward search benefiting from richer dependency structure once edges are abundant. The sub-additivity result supports treating single-pathology orderings as a useful reference, though the cross-over demonstrates they are not fully predictive.

Limitations

The authors concede three limitations plainly. The primary simulator is a linear Gaussian SCM matched to NOTEARS/GES assumptions; the nonlinear tanh replication preserves qualitative patterns but changes absolute difficulty and some rankings, so quantitative results should not be transferred uncritically. Scale is limited to $n = 800$4 genes—representative of a transcription-factor module but far from genome scale, where complexity may reorder methods. Most importantly, the study is entirely simulation-first; a systematic mapping between simulated pathologies and measurable properties of real perturbational datasets remains open, as does whether the observed regime boundaries hold at genomic scale. Latent confounding is identified as essentially unsolved from observational data alone, motivating interventional approaches.

Conclusion

This paper reframes the "correlation beats causal" puzzle as a composition of identifiable, pathology-specific failure modes rather than a verdict against causality. Causal methods genuinely dominate in clean and structurally favorable regimes; dropout selectively destroys discretization- and tree-based scores; confounding compresses all method classes uniformly; and joint pathologies compose sub-additively with occasional density-conditional cross-overs. The practical guidance—prefer Pearson or impute under heavy dropout, use interventions under confounding, prefer NOTEARS for sparse well-sampled networks, and always inspect error decompositions—is actionable, and the controlled-dial methodology offers a reusable template for diagnosing future GRN inference methods against specific, isolated stressors.