Causal Discovery Algorithms

• Causal discovery algorithms are systematic methods that infer underlying cause–effect structures among observed variables using graphical models and conditional independence tests.
• They encompass multiple paradigms—such as constraint-based, score-based, and functional models—tailored to various data types and complex systems.
• Recent innovations include continuous optimization, active learning, and LLM-guided frameworks to enhance scalability, fairness auditing, and experimental design.

Causal discovery algorithms (CDAs) are systematic methods for inferring the underlying causal relationships among observed variables, typically represented as directed acyclic graphs (DAGs), from statistical data. Widely applied in multiple scientific disciplines, CDAs formalize the process of extracting cause–effect structures from patterns of statistical dependence, employing principles such as the causal Markov condition, d-separation, and conditional independence. Modern developments have expanded their reach to complex data types, high-dimensional systems, dynamic processes, and fairness-sensitive settings.

1. Key Foundations and Principles

CDAs rest on the language of graphical models—specifically DAGs and structural causal models (SCMs)—where each variable $X_i$ is modeled as a function of its direct causes ($\operatorname{Pa}(X_i)$) and exogenous noise:

$$X_i := f_i(\operatorname{Pa}(X_i), U_i), \qquad P(X_1,\ldots,X_n) = \prod_{i=1}^n P(X_i \mid \operatorname{Pa}(X_i))$$

Two core axioms structure the logic of CA:

• Causal Markov Condition: Each variable is independent of its non-descendants, conditional on its direct causes.
• Faithfulness: The only conditional independencies present in the observational distribution are those entailed by the DAG's d-separation statements.

d-Separation is the graphical criterion formalizing when a set $C$ blocks all paths between sets $A$ and $B$ in a DAG, guaranteeing $A \perp B \mid C$. This directly underpins constraint-based approaches.

CDAs typically identify only the Markov equivalence class of DAGs from observational data—i.e., the set of all DAGs sharing the same implied conditional independence (CI) structure. Partial DAGs (PDAGs) and completed PDAGs (CPDAGs) are often used to represent these equivalence classes (Huber, 11 Jul 2024).

2. Major Algorithmic Paradigms

The field has produced a rich taxonomy of CDA methodologies, classified according to their statistical assumptions, the types of data and generative mechanisms they target, and their operational strategies (Niu et al., 17 Jul 2024):

Class	Examples (Paper IDs)	Typical Data	Notable Properties
Constraint-based	PC, FCI, tsFCI, PCMCI (Hasan et al., 2023)	I.I.D., time series	Systematic CI testing; sound/complete in the large-sample limit; FCI handles latent confounders
Score-based	GES, FGES, NOTEARS (Hasan et al., 2023)	I.I.D.	Searches DAG space for highest-scoring graph under data fit (e.g., BIC); often non-convex
Functional causal model–based	LiNGAM, ANM, PNL, DirectLiNGAM	I.I.D., time series	Imposes functional form; exploits asymmetries and non-Gaussianity for identifiability
State-space/dynamics-based	CCM, PAI, CMS, IOTA (Niu et al., 17 Jul 2024)	Time series	Designed for dynamic/temporal systems, often targeting lagged effects
Deep learning–based	CGNN, DAG-GNN, CORL, ACD	Numerical, high-dim	Leverages neural nets for function approximation and continuous optimization
Hybrid/other	ARMA-LiNGAM, hybrid RL, SCDA (Niu et al., 17 Jul 2024)	Mixed types	Combine multiple paradigms or inject additional domain knowledge

Algorithms may be tailored to allow cycles (CCD variants (Singh et al., 2017)) or explicitly include latent variables (FCI family) (Hasan et al., 2023).

3. Conditional Independence, Structural Constraints, and Theoretical Issues

All mainstream CDAs hinge on leveraging conditional independence relations (CIs) to prune and orient possible causal links. Key steps—for example, in the PC algorithm—include:

1. Systematic CI testing over variable pairs with growing conditioning sets to produce a "skeleton" graph.
2. Orientation of unshielded colliders (v-structures) to provide directionality, as in $A \to S \leftarrow B$, where $S$ is not in the separating set for $A$ and $B$ (Huber, 11 Jul 2024).
3. Application of additional orientation rules to complete the PDAG (provided no cycles or forbidden colliders are created).

Despite advances, constraint-based methods inherit several limitations:

• They are fundamentally limited by the Markov equivalence class and cannot distinguish among all possible DAGs unless additional (e.g., interventional) data or temporal information are available (Zanga et al., 2023).
• The reliability of CI tests declines with increasing variable count (the curse of dimensionality), limited sample size, or if faithfulness is violated (Ma et al., 2023).
• Some settings, such as quantum correlations, demonstrate that CDAs depending purely on CI fail to distinguish between fundamentally distinct causal mechanisms (e.g., quantum nonlocality vs. classical correlations) (Wood et al., 2012).

Score-based methods search the set of all DAGs for the graph maximizing an objective function (typically penalized likelihood, such as BIC or BDeu). For example, NOTEARS (Niu et al., 17 Jul 2024) optimizes a continuous loss function $S(A)$ over an adjacency matrix $A$ with the acyclicity constraint imposed via

$h(A) = \operatorname{tr}(e^{A \circ A}) - d = 0$

where $A \circ A$ is the Hadamard product and $d$ the number of nodes. DAGMA and related approaches propose alternative constraints for better numerical properties in large graphs (Possner et al., 23 Jul 2024).

4. Algorithmic Innovations and Scalability

Several advances address the scalability of CDAs to high-dimensional or complex domains:

• Divide-and-Conquer and Partitioning: Methods such as causal graph partitioning leverage preliminary "superstructure" graphs—obtained from domain knowledge or fast algorithms—to decompose the variable set into overlapping communities. Local learning is performed within each community, and results are merged while preserving theoretical soundness and efficiency (i.e., in the large-sample limit, the global CPDAG is still recovered) (Shah et al., 10 Jun 2024).
• Active Learning and Experimental Design: Some modern CDAs adopt active learning strategies to minimize the number or cost of interventions needed for full identification, selecting intervention targets ("do" operations) to maximize expected informativeness quantified by a "Power of Intervention" metric (Blondel, 2023).
• Continuous Optimization: Recent methods reformulate causal structure learning as a continuous constrained optimization problem, leveraging smooth acyclicity functions to allow application of efficient gradient-based solvers (Niu et al., 17 Jul 2024).
• Hybrid and LLM-guided Frameworks: Approaches integrating LLMs leverage textual metadata and expert-like reasoning as auxiliary sources of information, often guiding query prioritization, variable pair selection (via composite statistical and semantic scores), or even facilitating bias-path auditing in fairness-critical applications. Notably, empirical studies demonstrate that LLMs should be confined to non-decisional support—such as guiding heuristic search via natural language heuristics—because their autoregressive statistical training is not compatible with the conditional independence–based logic of causal discovery (Wu et al., 1 Jun 2025).

5. Application Domains and Evaluation

CDAs are employed in a wide range of scientific and industrial problems, with domain-specific requirements influencing both algorithm selection and preprocessing steps:

• Biomedical and Genomic Networks: CDAs infer regulatory and signaling interactions, often using partitioning schemes to address high dimensionality (e.g., gene regulatory networks with thousands of nodes (Shah et al., 10 Jun 2024)). Benchmarks like ALARM, Sachs, SynTReN, and DREAM challenges are standard (Hasan et al., 2023), with evaluation metrics including Structural Hamming Distance (SHD), F1 score, true/false positive rates, and intervention/counterfactual accuracy (Singh et al., 2017).
• Social Sciences/Economics: Relaxed assumptions (e.g., allowing for latent confounders with FCI) are often required, and special attention is given to the validity of the back-door and front-door criteria for effect identification (Huber, 11 Jul 2024).
• Manufacturing and Quality Management: CDA-driven root cause analysis (RCA) augments expert-driven procedures to pinpoint failure drivers among interacting process variables, with practical trade-offs between SHD, recall, and runtime depending on the algorithm (e.g., PC, NOTEARS, DAGMA) (Possner et al., 23 Jul 2024).
• Time Series and Spatiotemporal Systems: Specialized algorithms (PCMCI, DyNOTEARS, state-space methods) are designed to handle lagged effects and nonstationarity (Niu et al., 17 Jul 2024).
• Fairness Auditing: Modern frameworks extend CDAs to prioritize fairness-sensitive paths, combining LLM-informed variable selection, active learning, and effect decomposition to isolate direct and indirect influences of sensitive attributes on outcomes (Zanna et al., 21 Mar 2025, Zanna et al., 13 Jun 2025).

For benchmarking, the impact of sample size, noise characteristics, degree of linearity, and the presence of time delays are systematically considered (Niu et al., 17 Jul 2024). Metadata extraction tools can partially automate algorithm selection by characterizing data as i.i.d., time-lagged, linear or nonlinear, and Gaussian or heavy-tailed (Niu et al., 17 Jul 2024).

6. Limitations and Current Challenges

CDAs are subject to several intrinsic and practical challenges:

• Observational Equivalence: Without interventions, causal discovery in the presence of latent confounders or selection bias remains non-identifiable in general, and strong assumptions (causal sufficiency, faithfulness) are often empirically untestable (Wood et al., 2012, Zanga et al., 2023).
• Scalability: With growing variable count, the number of CI tests or possible DAGs expands super-exponentially. Divide-and-conquer frameworks, clustering, and continuous optimization (e.g., NOTEARS, DAGMA) offer partial relief (Shah et al., 10 Jun 2024).
• Aggregation and Vector-valued Variables: Applications involving vector-valued or aggregated variables (as in spatiotemporal climate data or economic indices) require explicit testing of aggregation consistency, as standard component-wise or averaging approaches may yield misleading or statistically inconsistent causal relationships (Ninad et al., 15 May 2025).
• Fairness and Bias: Recovering bias-relevant pathways for downstream evaluation (e.g., mediation by sensitive attributes) under finite samples and noisy conditions is an active research frontier. Integrated LLM-guided frameworks have been shown to robustly recover such pathways when appropriately constrained (Zanna et al., 21 Mar 2025, Zanna et al., 13 Jun 2025).
• Quantum Systems: Standard CDA frameworks, relying solely on conditional independence, are incapable of distinguishing between local and non-local quantum correlations, as illustrated in Bell-type experiments. Any classical causal explanation for Bell inequality–violating correlations in quantum systems necessarily involves parameter fine-tuning—contradicting the faithfulness assumption (Wood et al., 2012).

7. Software Ecosystem and Practical Tools

Extensive open-source toolkits support CDA research and application:

• R: bnlearn, pcalg (implementing PC, FCI, score-based methods).
• Python: Causal Discovery Toolbox (CDT), Tigramite (time series), causal-learn, gCastle, CausalNex.
• Java: TETRAD (comprehensive GUI and batch interfaces).

These platforms are often accompanied by curated benchmark datasets (ASIA, CHILD, ALARM, HEPAR2, Sachs, Tuebingen, DREAM4, UCI Adult, fMRI) and standardized performance metrics (SHD, F1, SID, runtime) (Niu et al., 17 Jul 2024, Hasan et al., 2023).

Summary

Causal discovery algorithms provide the methodical backbone for inferring causal structure from data. Their design is tightly linked to formal assumptions about the data-generating process, the nature of noise, the presence of latent confounders, and domain constraints (such as time, high dimensionality, or fairness requirements). As the field advances, ongoing developments address computational scalability, the integration of expert and semantic information (including LLMs in non-decisional roles), the treatment of aggregated and vector-valued variables, automated error/pruning checks via logical axiomatizations (Ma et al., 2023), and the practical identification of bias and fairness pathways in real-world systems. Despite substantial progress, foundational limitations—especially regarding identifiability from observational data, causal faithfulness, and fine-tuning in quantum or confounded systems—remain active areas of research, requiring cautious interpretation and ongoing methodological innovation.