Causal-Audit: A Framework for Risk Assessment of Assumption Violations in Time-Series Causal Discovery

Abstract: Time-series causal discovery methods rely on assumptions such as stationarity, regular sampling, and bounded temporal dependence. When these assumptions are violated, structure learning can produce confident but misleading causal graphs without warning. We introduce Causal-Audit, a framework that formalizes assumption validation as calibrated risk assessment. The framework computes effect-size diagnostics across five assumption families (stationarity, irregularity, persistence, nonlinearity, and confounding proxies), aggregates them into four calibrated risk scores with uncertainty intervals, and applies an abstention-aware decision policy that recommends methods (e.g., PCMCI+, VAR-based Granger causality) only when evidence supports reliable inference. The semi-automatic diagnostic stage can also be used independently for structured assumption auditing in individual studies. Evaluation on a synthetic atlas of 500 data-generating processes (DGPs) spanning 10 violation families demonstrates well-calibrated risk scores (AUROC > 0.95), a 62% false positive reduction among recommended datasets, and 78% abstention on severe-violation cases. On 21 external evaluations from TimeGraph (18 categories) and CausalTime (3 domains), recommend-or-abstain decisions are consistent with benchmark specifications in all cases. An open-source implementation of our framework is available.

• The paper introduces a calibrated risk assessment framework that quantifies assumption violation risks in time-series causal discovery.
• It integrates diagnostic tests with Bayesian calibration and abstention policies to enhance method selection and reduce false causal positives.
• Synthetic benchmarks and external validations confirm high reliability improvements and risk mitigation in diverse violation regimes.

Causal-Audit: Risk Assessment of Assumption Violations in Time-Series Causal Discovery

Overview and Motivation

The Causal-Audit framework introduces a rigorous, decision-theoretic methodology for pre-discovery risk assessment in time-series causal structure learning. It systematically quantifies the probability that algorithmic outputs from causal discovery methods (specifically PCMCI+ and VAR-based Granger causality) will be rendered unreliable due to violations of key methodological assumptions. The core innovation centers on calibrated probabilistic risk scoring and abstention-aware recommendations, formalized in a three-stage pipeline with strong empirical calibration and external validation. The framework is supported by a comprehensive synthetic benchmark (Synthetic DGP Atlas) spanning diverse assumption-violation regimes, and accompanies an open-source implementation.

Figure 1: The Causal-Audit pipeline: Tier 1 provides diagnostic auditing; Tier 2 augments with calibrated risk and abstention-aware recommendations.

Assumption Diagnostics and Risk Dimensions

Causal-Audit operationalizes violation assessment across five core families: stationarity, sampling regularity, persistence (autocorrelation), nonlinearity, and confounding proxies. Diagnostics include unit-root and KPSS tests, structural break detection, gap-variation for irregular sampling, Little MCAR for missingness, integral autocorrelation time, nonlinear prediction RMSE differences, VIF for collinearity, and the Chow test for parameter stability. Effect-size–not binary–outputs are aggregated, supporting nuanced assessment over rigid thresholds.

Diagnostics from each family map to four main risk dimensions: nonstationarity ($R_{\mathrm{nonstat}}$), irregularity ($R_{\mathrm{irreg}}$), persistence ($R_{\mathrm{persist}}$), and confounding proxy ($R_{\mathrm{confound}}$). Nonlinear effects are flagged separately due to inadequate corpus diversity for robust calibration and method-specific impact profiles.

Figure 2: Illustrative failures: different assumption violations in time-series data and the consequent inferred causal graphs with true versus erroneous edges.

Calibrated Risk Aggregation and Uncertainty Quantification

Stage II aggregates diagnostics into risk probabilities using a domain-informed, hierarchically Bayesian-calibrated logistic model. Weights reflect diagnostic informativeness (e.g., drift magnitude dominates nonstationarity, VIF reflects confounding instability) and are refined on the Synthetic DGP Atlas. An isotonic regression layer post-processes these probabilities, ensuring empirical calibration (with all final risk dimensions achieving calibration slope near 1.0 and expected calibration error $<$ 0.05).

Uncertainty intervals leverage a parametric bootstrap, adding conservative inflation for effective sample size reductions due to autocorrelation, and empirical SHAP analysis confirms attribution aligns with theoretical expectations.

Figure 3: Sigmoid risk mapping: demonstrates risk probability transitions and composite decision boundaries for stationarity diagnostics in Granger and PCMCI+.

Figure 4: SHAP analysis: diagnostic features' contributions to each risk dimension, affirming intended design.

Decision Policy: Method Recommendation and Abstention

The decision logic incorporates asymmetric cost-aware utility, explicitly encoding that false causal positives entail greater scientific risk than missed discoveries. Admissibility thresholds are method- and violation-specific, reflecting theoretical justifications and empirical robustness. For each dataset, the framework recommends admissible methods if risk estimates fall below calibrated thresholds; otherwise, it abstains. Abstention is treated as a positive, scientific action signaling the unsuitability of the input data for reliable causal discovery.

Composite risk for a candidate method is conservatively defined as the maximum across the four dimensions. Currently, the policy supports PCMCI+ and VAR-Granger, with extension planned for transfer entropy and confounder-robust approaches.

Synthetic Benchmarking and Validation

The Synthetic DGP Atlas constitutes 500 datasets, partitioned by variable count, time-series length, and precisely parameterized violation profiles across ten families. Four are core: each targets a calibrated risk dimension individually, facilitating unconfounded evaluation of diagnostic and decision modules.

Figure 5: Core violation benchmarking: illustrates family-specific risk diagonal dominance and off-target risk spillover due to interaction effects.

Figure 6: Risk score severity distributions across core violation families; confirms smooth calibration suitability and full-spectrum coverage.

Validation demonstrates that the Causal-Audit pipeline yields AUROC above 0.95 on held-out synthetic data, with consistently well-calibrated uncertainty intervals (nominal 95% – empirical 97.2%).

Figure 7: Reliability diagrams: observed versus predicted failure frequencies, indicating excellent calibration across all risk dimensions.

Figure 8: Cross-validation demonstrates performance stability across resampling and out-of-distribution regimes.

External Validation and Generalizability

On the TimeGraph and CausalTime external benchmarks, Causal-Audit matches or abstains in exact accordance with the known data-generating properties. For TimeGraph, categories with trends, seasonality, or compound violations consistently yield abstention, while clean and tractable-violation datasets are recommended. On CausalTime, high-dimensional, short-series settings trigger confounding proxy warnings due to multicollinearity, yet receive recommendations correctly, since the constraint is soft.

Quantitative results:

• 62% reduction in selective FPR among recommended data relative to the baseline.
• 78% abstention on severe-violation cases, preventing analysis on scenarios with high expected failure.
• On external validation, 21/21 decisions were consistent with benchmark expectation profiles.

Computational Cost

The total audit overhead scales at approximately $O(N^2 T^2)$, dominated by persistence diagnostics for typical datasets (e.g., $N=10$, $T=1000$), representing $\sim$10% of PCMCI+ runtime. The minimal incremental cost supports practical deployment in large-scale and interactive analyses.

Implications and Future Directions

Causal-Audit establishes risk-calibrated, abstention-aware assumption validation as a principled requisite for time-series causal discovery. The framework advances pre-hoc reliability estimation beyond post-hoc uncertainty quantification, thereby reducing silent failures and preventing the propagation of spurious scientific claims. By formalizing abstention and providing diagnostic attribution, the framework fosters transparent, reproducible reporting in causal analysis pipelines.

Practically, the method enables adaptive method selection across large heterogeneous datasets and supports structured assumption auditing for individual studies. The Synthetic DGP Atlas and open-source implementation provide strong support for community replication and extension.

Theoretical implications concern methodological calibration transfer. The framework's high generalization accuracy across unseen violation combinations and external, out-of-distribution realisms, indicates robust feature-risk mapping; however, compound violation regimes and nonlinearity remain areas for future model enrichment and dedicated calibration corpora. Future work will also incorporate computationally efficient FFT-based persistence diagnostics and refine risk treatment for high-dimensionality (e.g., dimensionality-normalized VIF).

Conclusion

Causal-Audit delivers a formally calibrated, interpretable, and abstention-capable risk assessment pipeline for time-series causal discovery. Its strong calibration, selective reliability improvements, and validated abstention behavior establish a new standard for pre-discovery assumption validation—mitigating risks of silent failure and enabling trustworthy causal inference in observational time-series analysis. The general framework—with modular diagnostics, configuration, and extensibility—positions it for rapid adaptation to expanding methodological catalogs and domain-specific calibration regimes.