Structural Causal Bottleneck Models

Abstract: We introduce structural causal bottleneck models (SCBMs), a novel class of structural causal models. At the core of SCBMs lies the assumption that causal effects between high-dimensional variables only depend on low-dimensional summary statistics, or bottlenecks, of the causes. SCBMs provide a flexible framework for task-specific dimension reduction while being estimable via standard, simple learning algorithms in practice. We analyse identifiability in SCBMs, connect them to information bottlenecks in the sense of Tishby & Zaslavsky (2015), and illustrate how to estimate them experimentally. We also demonstrate the benefit of bottlenecks for effect estimation in low-sample transfer learning settings. We argue that SCBMs provide an alternative to existing causal dimension reduction frameworks like causal representation learning or causal abstraction learning.

• The paper demonstrates that using low-dimensional bottleneck variables effectively captures causal interactions in high-dimensional datasets.
• It introduces a methodology that ensures identifiability under additive noise and injective effect functions, validated by empirical experiments.
• Empirical results indicate enhanced causal effect estimation and transfer learning, especially in sparse data scenarios.

Structural Causal Bottleneck Models

Introduction

Structural causal bottleneck models (SCBMs) offer a significant advancement in modeling causal relationships among high-dimensional variables. These models pivot on the assumption that causal interactions can be sufficiently captured by low-dimensional bottleneck variables. This framework provides robust tools for targeted dimension reduction, crucial for analyzing complex high-dimensional data found commonly in fields like neuroscience and climate science. SCBMs address the challenges associated with large data spaces by reducing them to manageable bottlenecks, effectively simplifying causal effect estimation using standard learning algorithms.

Theoretical Framework

SCBMs extend the concept of structural causal models (SCMs) by introducing bottleneck functions, which map high-dimensional parent variables to low-dimensional summaries. The essential idea is captured in the equation $$X_j := f_j(Z_{i_1}^i,\dots,Z_{i_k}^i, \boldsymbol{\eta}_j )$$, wherein the child variable depends on its parents solely through bottlenecks. These models are structurally equivalent, meaning they preserve all causal and statistical relationships under bijective transformations of bottleneck spaces.

Figure 1: Results of the identifiability experiments across various settings, indicating successful learning of bottleneck variables.

Identifiability

Identifiability within SCBMs is established under the assumption of additive noise and the injectivity of effect functions. Two SCBMs are structurally equivalent if they differ only by invertible transformations of the bottleneck spaces. This ensures that any learned bottleneck variables reflect the true low-dimensional summaries of the causal parents, effectively up to a nonlinear bijection. Empirical results demonstrate high identifiability scores across varying settings, underscoring the robustness of SCBMs in accurately estimating bottleneck variables.

Figure 2: Identifying bottlenecks with misspecified assumed bottleneck dimension, highlighting true dimension as a lower bound for identifiability.

Practical Implications

The empirical validation of SCBMs showcases substantial benefits, especially in transfer learning scenarios where causal connections from sparse datasets can be inferred efficiently. When traditional covariates are scarce, leveraging bottleneck variables for conditioning improves the effective sample size, enhancing the reliability of causal effect estimation. This approach is especially potent in environmental studies where continuous data is limited.

Figure 3: Mean absolute error in estimating effects using bottleneck conditioning, revealing marked benefits in low-sample regimes.

Comparisons and Connections

SCBMs distinguish themselves from causal representation learning by focusing on within-model operations that abstract random vectors specifically for causal effect inquiries, rather than recovering latent causal variables. Additionally, SCBMs differ by permitting invertible transformations and emphasize surjection for dimension reduction, unlike representation learning which typically insists on bijective mappings for identificability.

Conclusion

SCBMs are a vital addition to causal modeling, providing a flexible, dimension-reducing framework suitable for dealing with complex high-dimensional data. Theoretical guarantees of identifiability and scalability of estimation methods position SCBMs as a pragmatic alternative to existing causal inference methodologies. Future research can explore application-specific estimations and the broader utility of bottleneck variables across different domains.