---
title: Additive Noise Models (ANMs)
url: https://www.emergentmind.com/topics/additive-noise-models-anms
type: topic
---

# Additive Noise Models (ANMs)

Additive Noise Models (ANMs) provide a foundational framework for causal discovery in both bivariate and multivariate settings, grounding the inference of directed acyclic graph (DAG) structure in testable statistical asymmetries arising from additive noise representations of structural equations. The core assumption is that each observed variable is generated as a function of its parents plus a noise term that is statistically independent of its parents and mutually independent across nodes. Originating in work by Hoyer, Shimizu, Hyvärinen, Peters, and others, ANMs enable the identification of causal direction from observational data under generic nonlinearities and non-Gaussianity, with theoretical guarantees and scalable algorithms underpinning recent advances in causal inference, including structure learning in the presence of latent confounders, mixtures of mechanisms, arbitrary noise, and partial observability.

## 1. Structural Definition and Identifiability Results

A general ANM posits, for each node $X_j$ in a $p$-dimensional random vector $X = (X_1, ..., X_p)$ and associated DAG $G$,
\[
X_j = f_j( X_{\operatorname{Pa}(j)} ) + N_j,
\]
where $\operatorname{Pa}(j)$ are the parents in $G$, $f_j$ is a differentiable, non-constant function, and $N_j$ is a noise variable independent of $X_{\operatorname{Pa}(j)}$ and of all other $N_k$ ($k\ne j$) [1309.6779][2404.05148][2304.03265]. The key identifiability result is that for generic choices of $f_j$ and noise distributions, the true DAG $G$ is fully determined by the joint distribution $p(x)$. Typical sufficiency conditions are: (i) nonlinear $f_j$, (ii) non-Gaussian $N_j$, or (iii) each parent-child triple $(f_j,p_{N_i},p_{N_j})$ avoids a pathological ODE [1309.6779]. In the bivariate case, identifiability holds except for linear-Gaussian or degenerately symmetric settings. For multivariate models, a recursive sufficient condition requires that each conditional parent-child pair satisfies the bivariate identifiability criterion when conditioning on appropriate sets of nondescendants [1309.6779][2410.11759].

### Special Cases and Extensions

- **Linear Non-Gaussian Models (LiNGAM):** Linear $f_j$ with non-Gaussian $N_j$ [2404.05148].
- **Majorization Approach:** Conditional variances vector as a weak majorant identifies the topological order in linear SEMs, generalizing prior variance-based results [2404.05148].
- **Mixtures of Mechanisms:** When data arise from a mixture of ANMs indexed by latent discrete variable $Z$, identifiability is retained under generic conditions via independence between input $X$ and the mechanism parameter [1809.08568].

## 2. Causal Discovery Algorithms

### Regression with Subsequent Independence Test (RESIT)

RESIT is a two-phase algorithm. In phase 1, for each node, regress $X_j$ on its candidate parents and test for independence between residuals and regressors (e.g., via HSIC). The sink node (with minimal dependence) is recursively identified, and parents updated [1309.6779]. Phase 2 prunes extraneous parents. RESIT achieves statistical consistency under exact independence testing and nonparametric regression oracles, though it may be sensitive to noise scaling and high-dimensional dependence testing [1309.6779][2108.11320].

### Score-Matching and Order Search

- **Score Matching:** Causal graphs can be identified by analyzing the score function $s(x) = \nabla_x \log p(x)$ and its Jacobian; leaf nodes are found when the variance of the corresponding diagonal entry is zero, enabling iterative order reconstruction [2203.04413][2304.03265][2306.17361].
- **NoGAM:** Regresses empirical score estimates against regression residuals to identify leaves, without assuming Gaussianity, ensuring consistent recovery across arbitrary noise classes [2304.03265].
- **SCORE:** Computationally efficient kernel-based Stein estimators for score and score-Jacobian enable $O(d n^3)$ complexity algorithms that scale to large $d$, with rigorous guarantees [2203.04413].
- **LoSAM:** Leverages local independence and mutual information tests to establish roots and orderings, handling mixed linear/nonlinear mechanisms and minimizimg conditioning set sizes for efficiency [2410.11759].

### Variance- and Information-Based Sorting

- **$R^2$-SortnRegress:** Relies on the observation that the fraction of explained variance ($R^2_j = 1 - \frac{\operatorname{Var}(\varepsilon_j)}{\operatorname{Var}(X_j)}$) often increases along the true causal order in sampled ANMs; sorting variables by $R^2$ yields approximately correct topological orderings under high $R^2$-sortability, which is robust to data standardization [2303.18211].
- **Majorization Criterion:** For linear SEMs, ordering variables so their conditional variance vector weakly majorizes that of other permutations uniquely identifies the causal ordering [2404.05148].

### Global and Local Search

Brute-force and greedy search strategies (e.g., GDS, LoSAM) optimize independence and/or variance-based scores over DAGs. Recent approaches achieve polynomial time with provable consistency and reduced sample complexity by exploiting local causal substructures and conditioning set minimization [2410.11759][2303.18211].

## 3. Effects of Noise, Latent Structure, and Missing Data

### Noise Level Sensitivity

ANM-based inference is robust only when the noise level in the effect is of comparable scale to the cause. For linear models, accurate causal direction is achievable when the noise-to-signal ratio $r$ is in $[0.3,5]$; outside this range both residual independence and variance-based methods break down [2108.11320][2110.08087]. Nonlinear ANMs yield larger identifiable regimes, but practitioner guidance is to normalize variances, tune independence estimators, and combine strategies for robust inference.

### Latent Confounding and Hidden Mediation

- **Confounders with Additive Noise (CAN):** When both variables are nonlinear functions of a latent confounder plus mutually independent noise, identifiability is possible up to reparameterizations of the confounder, via moments inversion and independence constraints [1205.2640]. The ICAN algorithm alternates low-dimensional projection, independence minimization, and nonparametric regression; empirical results support model recovery under mild smoothness and independence conditions [1205.2640].
- **Unobserved Mediators (ANM-UM and CNANM):** The additive noise property is not preserved under marginalization over nonlinear mediator chains; standard ANM-based scoring and independence tests fail since conditional independence is lost in both directions [2506.23374][1905.09442]. Variational autoencoder (VAE) approaches (CNANM), or novel conditional denoising/diffusion statistics (BiDD), restore identifiability where standard ANM methods collapse [2506.23374][1905.09442]. BiDD achieves robust performance even with multiple nonlinear mediators by leveraging conditional denoising independence [2506.23374].

### Missing Data

In the presence of ignorable missingness, the EM-based MissDAG framework leverages the invertibility of additive noise structure to perform likelihood maximization over the observed data and posterior-imputed missing entries, with joint DAG and function parameter optimization in the M-step [2205.13869]. Classical identifiability results for ANMs carry over, as expected log-likelihoods are preserved, leading to empirically superior structure recovery compared to imputation-then-infer pipelines [2205.13869].

## 4. Model Variants: Mixtures, Mechanism Shifts, and Heterogeneity

- **Mixture of ANMs:** Observational data generated by a finite mixture of ANMs indexed by latent variable $Z$ are generically identifiable, as the existence of a mixture in both directions imposes highly restrictive ODE constraints on moments and densities [1809.08568]. Gaussian Process Partially Observable Models (GPPOM) employ a latent-variable GP regression with HSIC independence penalty for each sample's mechanism parameter, enabling unsupervised causal inference and mechanism clustering with strong accuracy [1809.08568].
- **Causal Mechanism Shifts (iSCAN):** In multi-environment ANMs differing only by soft (mechanism) interventions, the diagonal elements of the Hessian of the mixture score function ($J_{jj}$) isolate shifted nodes by variance testing. iSCAN leverages this property for efficient detection and reconstruction of mechanism shifts without reconstructing full DAGs per environment [2306.17361].

## 5. Practicalities, Theoretical Guarantees, and Empirical Performance

### Theoretical Guarantees

- **Identifiability:** For nonlinear ANMs with independent noise and generic $f_j$, identifiability of the full DAG from observational distribution is established [1309.6779][2304.03265][2410.11759].
- **Robustness to Noise Distribution:** Algorithms such as NoGAM and LoSAM are consistent without Gaussianity, correcting failures in Gaussian-specific scoring [2304.03265][2410.11759].
- **Polynomial-Time Recovery:** Score-matching–based and local-search methods can achieve $O(d^3)$ or $O(dn^3)$ time with sample efficiency scaling polynomially in $d$ [2410.11759][2203.04413][2304.03265]. Majorization and $R^2$-sorting approaches also admit similar computational complexity [2404.05148][2303.18211].

### Empirical Benchmarks

Simulations confirm that ANM-based procedures outperform constraint- and score-based methods (PC, GES, FGES) when non-Gaussianity or nonlinearity is present. Mixture and shift-detection methods (GPPOM, iSCAN) show high ARI and F$_1$ in synthetic and real-world heterogeneous datasets [1809.08568][2306.17361]. $R^2$-SortnRegress achieves accuracy competitive with state-of-the-art on benchmark datasets when $v_{R^2}$ is high ($>$0.8) [2303.18211], while LoSAM achieves topological accuracy even under mixed mechanisms and reduced computational cost compared to NHTS or greedy order-search [2410.11759].

## 6. Limitations and Recent Directions

ANMs assume acyclicity, causal sufficiency, and correct specification of additive noise. Failures arise in linear-Gaussian non-identifiable cases, settings with extreme noise-level ratios, and under irreducible hidden mediation that is nonlinear [2110.08087][2506.23374][1905.09442]. Recent developments address these restrictions via adaptive statistical testing, majorization-based ordering, latent-variable structure, and denoising/diffusion paradigms for direction-finding with latent mediation [2404.05148][2506.23374]. Robust extension to non-additive noise, feedback, or other classes of latent structure remains an open area [2306.17361][2604.03265].

## 7. Connections to Broader Causal Discovery and Outlook

ANMs have established themselves as a principal mechanism for observational causal discovery, subsuming and generalizing constraint- and score-based approaches by leveraging structural independence constraints and function–noise asymmetries. The proliferation of algorithms exploiting score-matching, majorization, regression-independence, and local substructure criteria reflects the centrality of ANMs in modern structure learning. Ongoing research aims to further extend the reach of ANMs to settings with confounding, missing data, dynamic environments, and high-dimensionality, while benchmarking beyond synthetic data remains crucial to assess $R^2$-sortability and identifiability in natural systems [2306.17361][2303.18211][2205.13869]. 

Key sources: [1309.6779], [1205.2640], [2304.03265], [2203.04413], [2410.11759], [2404.05148], [2303.18211], [2108.11320], [2110.08087], [1809.08568], [2306.17361], [2506.23374], [1905.09442], [2205.13869], [2407.10014].

Source: https://www.emergentmind.com/topics/additive-noise-models-anms