Optimization-Free Topological Sort for Causal Discovery via the Schur Complement of Score Jacobians

Abstract: Continuous causal discovery typically couples representation learning with structural optimization via non-convex acyclicity penalties, which subjects solvers to local optima and restricts scalability in high-dimensional regimes. We propose a decoupled paradigm that shifts the causal discovery bottleneck from non-convex optimization to statistical score estimation. We introduce the Score-Schur Topological Sort (SSTS), an algorithm that extracts topological order directly from unconstrained generative models, bypassing constrained structure optimization. We establish that the causal hierarchy leaves a geometric signature within the score function: iterative graph marginalization is mathematically equivalent to computing the Schur complement of the Score-Jacobian Information Matrix (SJIM) under linear conditions. This translates the acyclicity constraint into an algebraic procedure with a dominant cost of O(d^3) operations. For non-linear systems, we formulate the expectation gap of Schur marginalization and introduce Block-SSTS to compress extraction depth, bounding structural error. Empirically, SSTS allows causal structural analysis on non-linear graphs up to d=1000. At this scale, our framework indicates that once the non-convex optimization bottleneck is mathematically bypassed, the structural fidelity of continuous causal discovery is bounded by the finite-sample estimation variance of the global score geometry. By reducing graph extraction to matrix operations, this work reframes scalable causal discovery from a constrained optimization problem to a statistical estimation challenge.

• The paper introduces SSTS, an algorithm that uses score Jacobian information and Schur complements to extract causal order without iterative optimization.
• It decouples causal structure extraction from non-convex optimization, enabling deterministic DAG recovery with significantly fewer edge violations in high dimensions.
• Empirical results show scalable performance with robust recovery in both linear and non-linear models, while highlighting trade-offs between algebraic exactness and statistical estimation.

Optimization-Free Causal Discovery via Score-Schur Topological Sort

Motivation and Context

The paper addresses the structural bottleneck in continuous causal discovery. Traditional combinatorial search algorithms for extracting Directed Acyclic Graphs (DAGs) from observational data are computationally intractable in high dimensions due to exponential scaling. While continuous optimization techniques (e.g., NOTEARS) provided algebraic acyclicity formulations enabling gradient-based methods, these are restricted by non-convex optimization landscapes, resulting in susceptibility to local minima and poor scaling for high-dimensional or non-linear data. Score-based paradigms further enabled generative modeling with topological ordering, but often required iterative score model retraining or complex architectural masking, posing severe efficiency limitations.

The Score-Schur Topological Sort (SSTS) is introduced as an optimization-free causal discovery algorithm, extracting causal topological order directly from unconstrained score networks by leveraging the geometric and algebraic signatures encoded within the Score-Jacobian Information Matrix (SJIM). The core contribution is the decoupling of causal structure extraction from non-convex optimization, reframing causal discovery as a statistical estimation problem governed by finite-sample limits.

Theoretical Contributions

Algebraic Mapping of Causal Structure

SSTS operates by utilizing the SJIM, defined as the expected negative Hessian of the log-density, $I = \mathbb{E}_{p(x)}[-\nabla_x^2 \log p(x)]$. In linear Gaussian Additive Noise Models (ANMs), the SJIM contains explicit topological information:

• Leaf Node Identifiability: Diagonal entries $I_{ii}$ reach their minimum for true leaf nodes, facilitating their detection without structural optimization.
• Schur Complement Marginalization: Once a leaf node is isolated, its marginalization is equivalent to the Schur complement of the SJIM with respect to the leaf index. This mechanism is mathematically exact in linear ANMs, enabling sequential, deterministic extraction of topological order by repeated Schur eliminations.

For non-linear ANMs, expectation and matrix inversion operations do not commute. The paper formalizes the non-linear marginalization gap ($\Delta$), arising from the covariance of gradient vectors, $$-\frac{1}{\sigma^2} Cov_{p(x)}(\nabla_{x_{\setminus l}} f_l)$$. This gap is localized to the parent neighborhood of marginalized nodes and induces accumulating extraction error during sequential marginalizations.

Decoupling Structural Extraction

Conventional score-based methods require iterative retraining or masking for marginalization. SSTS eliminates this by leveraging a single pre-trained score network and algebraically extracting structure via matrix operations. The transformation replaces constrained optimization with deterministic algebraic elimination, reducing the computational complexity to $\mathcal{O}(d^3)$ (where $d$ is the variable dimension) for matrix extraction.

Figure 1: SSTS efficiently recovers causal graph structure on non-linear manifolds, maintaining favorable performance-efficiency trade-offs and scaling to $d=100$ dimensions in under 15 seconds.

Algorithmic Extensions

High-Dimensional and Sparse Regimes

• Sparse Jacobian Prior: High-dimensional score estimation induces numerical ill-conditioning and variance. The paper introduces a Group Lasso ($\ell_{1,2}$) penalty on the network’s input layer, truncating weak connections and yielding a well-conditioned, symmetric SJIM for robust extraction.
• Block-SSTS Marginalization: Parallel leaf nodes often occur in sparse DAGs. With Block-SSTS, nodes with nearly identical diagonal energies are marginalized in blocks, reducing sequential Schur inversion depth and limiting the expectation gap cascade, crucial for scalability to $d=1000$ variables.

  Figure 2: Ablation studies show variance suppression via sparsity priors in data-starved regimes and reduction in error accumulation as block tolerance compresses matrix inversion depth under extreme scaling.

Empirical Results

Linear and Non-linear ANM Recovery

• Linear ANMs: Experiments confirm exact topological extraction with zero edge violations up to $d=1000$ both for analytical and empirical precision matrices.
• Non-linear ANMs: SSTS achieves SHD $I_{ii}$0 with extraction times of $I_{ii}$1 seconds at $I_{ii}$2, far outperforming constrained optimizers (DAGMA: SHD $I_{ii}$3, $I_{ii}$4s).
• Exact Schur Complement vs. Expectation: Computing the exact sample-wise Schur complement prior to expectation strictly reduces edge violations, but incurs an $I_{ii}$5 computational cost, highlighting a trade-off between algebraic fidelity and scalability.

Biological Data and Mechanism Robustness

• Sachs Protein Network: On real-world data ($I_{ii}$6), SSTS attains SHD $I_{ii}$7 and TPR $I_{ii}$8, outperforming DAGMA and Scalable SCORE methods in both accuracy and efficiency.
• Distributional Shifts and Mechanisms: SSTS is robust to multiplicative noise but fails under post-nonlinear mechanisms (PNL), where non-linear mappings obscure the diagonal energy signature required for algebraic extraction.

Scaling and Finite-Sample Boundaries

SSTS scales block-wise extraction efficiently to $I_{ii}$9 with a bounded VRAM ($\Delta$0 GB) and direct extraction times ($\Delta$1s). As dimensionality increases, the statistical estimation variance of the score Jacobian dominates structural fidelity, emphasizing the transition from optimization-bounded to estimation-bounded causal discovery.

Practical and Theoretical Implications

The paradigm shift brought by SSTS has several implications:

• Computational Bottleneck Shift: By eliminating acyclicity constraints during extraction, SSTS transfers the bottleneck from optimization to statistical estimation, making accurate score learning essential for high-dimensional, non-linear causal discovery.
• Metric Robustness: The paper formalizes the mathematical degradation in rank correlation metrics (e.g., Kendall’s $\Delta$2), advocating for edge violation counts as the rigorous structural metric, invariant to topological ambiguity.
• Limits of Algebraic Extraction: The expectation gap in non-linear ANMs, although localizable and optionally patchable (at high computational cost), poses a fundamental challenge for causality in arbitrary non-linear regimes.

Future Directions

Potential advances may include:

• Covariance Patch Mechanisms: Developing scalable methods to compensate the non-linear expectation gap without $\Delta$3 complexity.
• Score Estimation Methods: Improving generative modeling architectures to reduce finite-sample estimation variance in extreme regimes.
• Generalization Beyond ANM: Extending algebraic extraction to post-nonlinear and other generative mechanisms with modified diagonal energy signatures.

Conclusion

SSTS reframes continuous causal discovery as a statistical estimation-driven process by algebraically mapping topological extraction to Schur complements of score Jacobians. Exactness holds in linear regimes; approximation errors arise in non-linear regimes, governed by the expectation gap and score estimation variance. The fundamental bottleneck is now determined by statistical fidelity rather than optimization locality, offering scalable causal structure discovery for high-dimensional, non-linear systems.