Neural Causal Models: Bridging Neural Networks and Causality

• Neural Causal Models are neurally parameterized structural causal models that integrate deep neural networks with causal inference principles.
• They enable flexible high-dimensional modeling and provide estimation of observational, interventional, and counterfactual distributions.
• Algorithms leveraging NCMs incorporate graph-induced structural bias to reliably identify and estimate causal effects in complex datasets.

Neural Causal Models (NCMs) are a class of neurally parameterized structural causal models in which the functional mechanisms relating variables in a directed acyclic graph (the causal diagram) are modeled using neural networks, typically feedforward multilayer perceptrons. NCMs unify the expressive power of modern neural network architectures with the formalism of causal inference. This allows not only flexible, high-dimensional modeling of complex data generation processes, but also principled reasoning about interventions, counterfactuals, and causality in the sense articulated by the Pearl causal hierarchy. The following sections describe the foundational principles, methodology, expressivity vs. learnability distinction, practical algorithms, and key theoretical and empirical results regarding NCMs.

1. Formal Structure and Expressivity of Neural Causal Models

An NCM extends the notion of a Structural Causal Model (SCM) $$\mathcal{M} = \langle \mathcal{U}, \mathcal{V}, \mathcal{F}, P(\mathcal{U}) \rangle$$, where $\mathcal{V}$ are endogenous variables, $\mathcal{U}$ are exogenous noise variables, and $\mathcal{F}$ is a collection of structural functions, by modeling each function $f_V$ as a neural network. For each endogenous variable $V_i$, the mechanism is

$$f_{V_i}: D_{U_{V_i}} \times D_{\mathrm{Pa}(V_i)} \to D_{V_i},$$

where $\mathrm{Pa}(V_i)$ are the parents of $V_i$ according to a DAG $\mathcal{G}$, and $U_{V_i}$ is a (possibly multi-dimensional) exogenous noise variable (often taken as independent $\mathrm{Unif}(0,1)$ or $\mathcal{N}(0,I)$). This construction preserves the full expressivity of SCMs:

• For any true SCM $\mathcal{M}^*$, there exists an NCM that is $L_3$-consistent with $\mathcal{M}^*$, meaning it matches the observational, interventional, and counterfactual distributions induced by $\mathcal{M}^*$ (Xia et al., 2021).

The class of "G-constrained" NCMs enforces that the parent sets of each function $f_{V_i}$ and the confounded components of the exogenous variables strictly follow the given causal diagram $\mathcal{G}$, encoding structural inductive bias.

2. Distinction Between Expressivity and Learnability

Although NCMs are universal approximators and can, in principle, represent any SCM, the neural causal hierarchy theorem demonstrates a crucial limitation:

• Expressivity does not guarantee learnability of causal effects. Even if an NCM fits the observational distribution $P(\mathcal{V})$ exactly, it does not follow that it also recovers the correct interventional $P(\mathcal{V} \mid do(x))$ or counterfactual $P(\mathcal{V}_{x} \mid x')$ distributions.

This is a corollary of the general hierarchy theorem: many distinct SCMs (and thus NCMs) can yield identical $P(\mathcal{V})$ but induce different higher-level distributions. Layer 1 (observational) alone cannot yield the information to determine layers 2 (interventional) or 3 (counterfactual) (Xia et al., 2021).

3. Causal Identification, Estimation, and the Role of Inductive Bias

The identification question in NCMs is formally defined as follows: Given a causal graph $\mathcal{G}$ and a target query $Q = P(Y \mid do(X))$, is $Q$ neural-identifiable from $\Omega(\mathcal{G})$ (the set of $\mathcal{G}$-constrained NCMs) if for all $M_1, M_2 \in \Omega(\mathcal{G})$ with $$P^{M_1}(\mathcal{V}) = P^{M_2}(\mathcal{V}) = P^*(\mathcal{V})$$, we have $P^{M_1}(Y \mid do(X)) = P^{M_2}(Y \mid do(X))$? (Xia et al., 2021)

Practical identification and estimation algorithms operate in two stages:

• Neural Effect Identification: Two optimization procedures search for NCM parameterizations that maximize and minimize the target query $Q$, both subject to suffering minimal discrepancy with the observed $P(\mathcal{V})$ (via an $L_1$-consistent constraint).
  • If the difference (max-min gap) is small, $Q$ is identifiable and the value is robustly estimated;
  • If the gap remains large, $Q$ is not identifiable from the data and the assumed structure.

Estimation uses an NCM as a proxy and computes the interventional effect via "mutilation" of the network: replacing the mechanism for $X$ with a constant-value function to simulate intervention. For general (non-identifiable) queries, the algorithm quantifies the bounds (min, max) compatible with the data and $\mathcal{G}$.

The role of the structural (graph-based) inductive bias is essential: it restricts the set of NCMs so that the relevant invariances and confounding constraints are enforced, ensuring that the NCM does not "cheat" by fitting spurious solutions.

4. Algorithms and Theoretical Guarantees

Identification and Estimation Algorithms:

• Neural causal algorithms involve training two parameterizations ($\theta_{\min}$ and $\theta_{\max}$) of the NCM:

$$\begin{align*} \text{Maximize } & Q(\theta) \text{ s.t. } \|P^{\widehat{M}(\theta)}(\mathcal{V}) - P^*(\mathcal{V})\| < \epsilon, \ \text{Minimize } & Q(\theta) \text{ s.t. } \|P^{\widehat{M}(\theta)}(\mathcal{V}) - P^*(\mathcal{V})\| < \epsilon. \end{align*}$$

• The identifiability criterion is satisfied if $|Q_{\max} - Q_{\min}| < \tau$ for a threshold $\tau$ (Xia et al., 2021, Xia et al., 2022).

Soundness and Completeness:

• These algorithms provide necessary and sufficient conditions: if the NCMs matching $P(\mathcal{V})$ all agree on $Q$, the query is identifiable; otherwise, it is not (Xia et al., 2022).
• The identifiability condition for $Q = P(Y \mid do(X))$ is:

$$\forall \widehat{M}_1, \widehat{M}_2 \in \Omega(\mathcal{G}) \text{ with } P^{\widehat{M}_1}(\mathcal{V}) = P^{\widehat{M}_2}(\mathcal{V}) = P^*(\mathcal{V}):\ P^{\widehat{M}_1}(Y \mid do(X)) = P^{\widehat{M}_2}(Y \mid do(X)).$$

5. Empirical Performance and Simulation Studies

Experiments on synthetic and real-world data examined canonical identification scenarios, including:

• Backdoor, frontdoor, “M-structure,” napkin graphs: NCMs correctly characterized identifiable and non-identifiable queries, as measured by the max-min identification gap (Xia et al., 2021).
• For estimation, NCM-based methods achieved true causal effects on identifiable queries, matching the Average Treatment Effect (ATE) estimated via symbolic methods and outperforming naive generative models.
• The identification gap decreased with larger sample sizes, and mean absolute error (MAE) on the ATE decreased correspondingly.
• For both continuous and discrete variables, and for high-dimensional settings, the NCM framework remained robust provided the optimization was successful.

6. Computational Considerations and Limitations

While NCMs are provably expressive enough to approximate any SCM, the computational cost of inference is non-trivial:

• Marginal inference in general NCMs is NP-hard (Zečević et al., 2021).
• Mechanism inference (evaluating a structural function) is tractable for each node, but marginal queries may require exponential time in the number of variables.
• Tractable Neural Causal Models (TNCMs), such as those using SPN (Sum-Product Network) modules, can provide linear-time mechanism inference at the sub-module level, but overall inference in the full model remains NP-hard.
• Inductive biases imposed via the causal diagram are essential for practical learnability because unconstrained models overfit $P(\mathcal{V})$ and fail for $P(\mathcal{V} \mid do(x))$.

A taxonomy of causal model families highlights the following (see (Zečević et al., 2021)):

Model Family	Pearl Hierarchy Level	Causal Identification	Mechanism Inference	Marginal Inference
Non-causal (e.g. OLS, CNN)	$\mathcal{L}_1$	External (do-calculus)	Linear	Linear or quadratic
Partially causal (e.g. iVGAE)	$\mathcal{L}_2$	Embedded	Linear	Linear/quadratic
Full SCM (NCM, TNCM)	$\mathcal{L}_3$	Embedded	Linear (TNCM), quadratic (NCM)	NP-hard

7. Connections, Use Cases, and Broader Impact

Neural Causal Models unify graph-based causal reasoning (do-calculus, d-separation, symbolic identification) with deep differentiable modeling. Their major applications include:

• Quantitative attribution—yielding the Average Causal Effect (ACE) for each input or feature in neural architectures (Chattopadhyay et al., 2019).
• Identification of causal effects in high-dimensional, nonlinear data, including counterfactual and interventional queries.
• Serving as a foundation for causal discovery algorithms in the neural domain.
• Enabling sound, complete, and scalable algorithms for identification and estimation, given a known graph.
• Forming the basis for abstractions and representation learning in settings where semantics must be propagated from lower-level data to higher-level constructs (Xia et al., 5 Jan 2024).

A central implication is that NCMs, while maximally expressive, require structural knowledge (the known causal diagram) for reliable inference; without it, causal queries are generically underdetermined. This renders NCMs tools for both hypothesis testing (by simulating possible SCMs compatible with data and a graph) and for scalable estimation in domains where neural networks are the preferred modeling class. Empirical studies confirm that, when combined with the correct structure, NCMs yield empirically reliable and theoretically sound answers to challenging causal queries absent in conventional black-box ML.

---

Neural Causal Models thus formalize and implement a computational interface between neural network learning and formal causal inference, enabling principled identification, estimation, and reasoning about interventions in complex, high-dimensional data settings (Xia et al., 2021).