Multi-Method Causal Evidence Synthesis: Ranking Candidate Drivers by Convergent Cross-Method Evidence from Observational Data

Abstract: Practitioners inferring causality from observational data usually rely on a single method and treat its output as causal truth. Recent tools select an optimal method for a dataset, and recent ensembles aggregate multiple causal-discovery algorithms into one graph, but little work pools evidence across different mathematical traditions, including non-causal ones. We present Multi-Method Causal Evidence Synthesis (MCES), a framework that ranks which candidate drivers in an observational system are most likely relevant to a set of outcomes, and with what strength of evidence. MCES runs eleven methods across eight mathematical traditions on observational panel data and pools their outputs into a Convergent Evidence Score (CES), a linear opinion pool. CES quantifies convergence of evidence across analytical lenses: the degree to which methods with different assumptions point to the same driver-outcome relationship. It does not claim causal identification in the interventionist sense; it supports hypothesis prioritization, not a transferable probability of causation. MCES first applies Structural-Behavioral Decomposition to remove definitional (algebraic) relationships, then runs all methods, normalizes outputs to [0,1], and pools them. We distinguish MCES from method selection, structural ensembles, prediction ensembles, and literature synthesis. Using synthetic data with embedded ground truth, the Sachs protein-signaling benchmark, six Bayesian-network structure benchmarks, and two further synthetic domains, we show MCES ranks true edges near the top (Precision@5 = 1.0, Precision@10 = 0.96 on the primary scenario), with a low empirical rate of null pairs reaching Moderate-or-higher convergence. Our central point is not that the pool beats every individual method, but that no single method is uniformly best across the evaluated scenarios, so MCES offers a method-agnostic default.

• The paper introduces MCES, a framework that pools normalized evidence from 11 analytical methods across eight mathematical traditions to rank driver–outcome hypotheses by cross-method convergence rather than establish interventionist causality.
• The framework achieves Precision@5 of 1.00 and Precision@10 of approximately 0.97 on the primary synthetic panel, while showing that no single method consistently dominates and that panel width matters more than time-series length.
• Structural–Behavioral Decomposition improves top-three precision from 0.333 to 1.000 when algebraic identity relationships create false positives, but MCES remains scenario-specific, conservative, and unable to resolve hidden confounding or recover direction without a declared driver–outcome partition.

The paper presents Multi-Method Causal Evidence Synthesis (MCES), a framework for ranking candidate driver–outcome relationships in observational panel data by pooling evidence from eleven analytical methods spanning eight mathematical traditions. Rather than selecting a single "best" method or recovering a causal graph, MCES computes a Convergent Evidence Score (CES) — a weighted linear opinion pool of normalized, non-commensurable evidence measures — and uses it to prioritize hypotheses. The authors are explicit that CES measures cross-method convergence, not interventionist causal identification, and that its calibration is scenario-specific with unestablished transportability.

Motivation and positioning

The paper targets a common practice failure: analysts commit to one method (regression, SHAP-based importance) and treat its output as causal truth, inheriting that method's assumptions and blind spots. Existing automation addresses this by selection (Causal-Copilot, CausalTune), which discards all non-selected evidence; ensemble causal discovery aggregates multiple algorithms but only within the graph-recovery family; literature-level triangulation (Shi et al.) requires published studies rather than raw data. The claimed novelty is deliberately narrow: pooling non-commensurable evidence from causal and explicitly non-causal traditions to rank a declared driver–outcome grid, a combination the authors state has not been studied.

A distinctive pre-processing step, Structural–Behavioral Decomposition, removes algebraic-identity edges (e.g., Revenue = Price × Volume when price and volume are also candidate drivers). The authors scope this precisely: it is a no-op unless identity components appear in the feature set, and the controlled experiment confirms the gain is confined to top-of-list precision in exactly that setting.

Framework

For each candidate pair $(x_j, y_l)$, each method produces raw evidence, normalized via a method-specific $\phi_k$ into $[0,1]$, then pooled:

$$CES(x_j, y_l) = \frac{\sum_k w_k\,\tilde{e}_k(x_j, y_l)}{\sum_k w_k}$$

The suite spans partial correlation, lasso, distance correlation, mixed-effects regression, RF+SHAP, Predictive Power Score, Granger causality, interrupted time series, transfer entropy, Bayesian network structure learning, and CausalForestDML. Two properties qualify interpretation: six methods use within-outcome relative scaling, so raw CES is ordinal and comparable primarily within an outcome; and several methods apply hard significance gates, so the pool is smooth only above the gates. Missingness is handled by weight renormalization at the method level but conservative zero-fill at the pair level. Pairs are classified into Weak/Moderate/Strong convergence bands, with the Tier-3/4 requirement for Strong convergence enforced as an explicit rule rather than derived from weights, since renormalization in degraded modes (e.g., cross-sectional data) would otherwise break the guarantee.

The theoretical contribution is a variance argument: pooled-score variance is non-decreasing in cross-method correlations $\rho_{jk}$, so assumption diversity reduces score variance. The authors measure the proposition's own quantity directly — fixed-pair mean cross-method correlation across seeds is $\bar\rho = 0.13$ — but concede that reduced variance does not automatically translate into improved ranking accuracy.

Empirical results

On the primary synthetic panel ($N=23$, $T=20$, 95 drivers, 6 outcomes, 18 true edges, 20 seeds), the ensemble achieves Precision@5 = 1.0 and Precision@10 ≈ 0.97, with F1@10 of 0.686 ± 0.035. The paper's most candid finding is that the ensemble does not uniformly dominate: lasso (0.714) and partial correlation (0.711) edge it out on this scenario, while NOTEARS wins on linear scenarios (0.70 vs. 0.69 primary; 0.82 vs. 0.73 confounded) but collapses on nonlinear data (0.42 vs. 0.74). Leave-one-out ablation shows no single load-bearing member (max F1 drop is small), and a supervised logistic combiner gains only ~+0.03 F1 over uniform pooling — confirming weight-insensitivity (mean pairwise Spearman 0.945 across five schemes).

Key quantitative findings:

Result	Value
Primary scenario Precision@5 / @10	1.00 / ~0.97
Null pairs reaching Moderate convergence	0.0003 (primary); 0.000 on four other scenarios
All-null negative control (Moderate rate)	<0.5%, ~16× primary
Sachs benchmark Precision@5	1.00 (top pairs are canonical signaling edges)
bnlearn benchmarks Precision@5	1.0 on five of six networks
Decomposition: Precision@3 without → with SBD	0.333 → 1.000
Sample-size sweep	Panel width binding (F1@10 = 0.33 at N=6 vs. 0.71 at N=46); length saturates by T=20

The false-positive analysis is a strength: Benjamini–Hochberg gating is default, and the low null-pair rate at Moderate convergence holds across a threshold sweep (only 0.51% even at CES ≥ 0.3), though retention drops sharply — only ~50% of true pairs reach the default threshold, and almost none reach Strong. The bands are conservative by construction.

An E9 ground-truth-free diagnostic shows high-CES pairs carry mean out-of-sample predictive lift well above null pairs, though pairwise rank correlation between CES and lift is weak — CES separates groups but does not finely order pairs.

Two audits deserve emphasis. First, an orientation audit on the bnlearn benchmarks shows the declared driver/outcome partition is load-bearing: removing it drops Precision@5 from 1.0 to 0.2–0.6, because most members are direction-symmetric. MCES ranks a declared grid; it does not recover orientation. Second, the nonlinear scenario shows the pool matches but does not exceed its best members' recall (0.667) — synthesis does not manufacture detection power absent from every component.

Limitations and open questions

The paper concedes substantial limitations plainly. Hidden confounding, feedback loops, and collider bias affect all members simultaneously since they share the same data; pooling mitigates only method-specific failure modes. Raw CES requires isotonic calibration for probabilistic interpretation, and that calibration's transportability to new domains is unestablished. Non-panel methods treat rows as exchangeable despite panel dependence, so their inferential outputs are screening statistics rather than calibrated tests. The 0.4/0.7 band cutoffs are design conventions, not estimated quantities. No formal false-discovery bound is proved. The evaluation relies mostly on synthetic data; Sachs and the bnlearn networks provide external structure but not field-collected panels. Open questions include whether learned weights with redundancy pruning (e.g., Granger/transfer entropy on near-Gaussian temporal data) improve accuracy, whether reverse-direction penalization can relax the declared partition, and what structural property makes Hailfinder harder than larger, denser networks.

Conclusion

MCES converts the long-standing qualitative recommendation of causal triangulation into a computable, reproducible ranking procedure over a declared driver–outcome grid, with Structural–Behavioral Decomposition eliminating a specific identity-induced false-positive class and empirical false-positive control replacing recovery guarantees. Its central claim is deliberately modest and empirically supported: because no single method — internal or external — is uniformly best across the evaluated scenarios, convergent multi-method evidence offers a defensible method-agnostic default for hypothesis prioritization, not a substitute for experimental identification or a transferable probability of causation.