Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Identifiable Latent Neural Causal Models (2403.15711v1)

Published 23 Mar 2024 in cs.LG, stat.ME, and stat.ML

Abstract: Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. It is particularly good at predictions under unseen distribution shifts, because these shifts can generally be interpreted as consequences of interventions. Hence leveraging {seen} distribution shifts becomes a natural strategy to help identifying causal representations, which in turn benefits predictions where distributions are previously {unseen}. Determining the types (or conditions) of such distribution shifts that do contribute to the identifiability of causal representations is critical. This work establishes a {sufficient} and {necessary} condition characterizing the types of distribution shifts for identifiability in the context of latent additive noise models. Furthermore, we present partial identifiability results when only a portion of distribution shifts meets the condition. In addition, we extend our findings to latent post-nonlinear causal models. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations. Our algorithm, guided by our underlying theory, has demonstrated outstanding performance across a diverse range of synthetic and real-world datasets. The empirical observations align closely with the theoretical findings, affirming the robustness and effectiveness of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Identification of partially observed linear causal models: Graphical conditions for the non-gaussian and heterogeneous cases. In NeurIPS, 2021.
  2. Interventional causal representation learning. In International Conference on Machine Learning, pp.  372–407. PMLR, 2023.
  3. Learning linear bayesian networks with latent variables. In ICML, pp.  249–257, 2013.
  4. Weakly supervised causal representation learning. arXiv preprint arXiv:2203.16437, 2022.
  5. Learning linear causal representations from interventions under general nonlinear mixing. arXiv preprint arXiv:2306.02235, 2023.
  6. Triad constraints for learning causal structure of latent variables. In NeurIPS, 2019.
  7. Handling sparsity via the horseshoe. In Artificial intelligence and statistics, pp.  73–80. PMLR, 2009.
  8. Image-based profiling for drug discovery: due for a machine-learning upgrade? Nature Reviews Drug Discovery, 20(2):145–159, 2021.
  9. The great recession and mental health in the united states. Clinical Psychological Science, 7(5):900–913, 2019.
  10. Robust causal structure learning with some hidden variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81(3):459–487, 2019.
  11. Multi-domain causal structure learning in linear systems. Advances in neural information processing systems, 31, 2018.
  12. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660–1672, 2021.
  13. beta-vae: Learning basic visual concepts with a constrained variational framework. In ICLR, 2017.
  14. Nonlinear causal discovery with additive noise models. In NeurIPS, volume 21, pp.  689–696. Citeseer, 2008.
  15. Causal discovery from heterogeneous/nonstationary data. JMLR, 21(89), 2020.
  16. Latent hierarchical causal structure discovery with rank constraints. Advances in Neural Information Processing Systems, 35:5549–5561, 2022.
  17. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
  18. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.  859–868. PMLR, 2019.
  19. Systematic evaluation of causal discovery in visual model based reinforcement learning. arXiv preprint arXiv:2107.00848, 2021.
  20. Variational autoencoders and nonlinear ica: A unifying framework. In AISTAS, pp.  2207–2217. PMLR, 2020.
  21. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  22. Partial disentanglement for domain adaptation. In International Conference on Machine Learning, pp.  11455–11472. PMLR, 2022.
  23. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. arXiv preprint arXiv:2107.10098, 2021.
  24. URL https://openfmri.org/dataset/ds000031/.
  25. Causal representation learning for instantaneous and temporal effects in interactive systems. In The Eleventh International Conference on Learning Representations, 2022a.
  26. Citris: Causal identifiability from temporal intervened sequences. In International Conference on Machine Learning, pp.  13557–13603. PMLR, 2022b.
  27. Variational bayesian dropout with a hierarchical prior. In CVPR, 2019.
  28. Identifying weight-variant latent causal models. arXiv preprint arXiv:2208.14153, 2022.
  29. Identifiable latent polynomial causal models through the lens of change. arXiv preprint arXiv:2310.15580, 2023.
  30. Pearl, J. Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge, 2000.
  31. Causal discovery with continuous additive noise models. JMLR, 15(58):2009–2053, 2014.
  32. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017.
  33. Climate-driven shifts in marine species ranges: Scaling from organisms to communities. Annual review of marine science, 12:153–179, 2020.
  34. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  35. Linear causal disentanglement via interventions. arXiv preprint arXiv:2211.16467, 2022.
  36. Estimation of linear non-gaussian acyclic models for latent factors. Neurocomputing, 72(7-9):2024–2027, 2009.
  37. Learning the structure of linear latent variable models. JMLR, 7(2), 2006.
  38. Disentanglement by nonlinear ica with general incompressible-flow networks (gin). arXiv preprint arXiv:2001.04872, 2020.
  39. Causation, Prediction, and Search. MIT Press, Cambridge, MA, 2nd edition, 2001.
  40. Score-based causal representation learning with interventions. arXiv preprint arXiv:2301.08230, 2023.
  41. Self-supervised learning with data augmentations provably isolates content from style. In Advances in neural information processing systems, 2021.
  42. Causal balancing for domain generalization. arXiv preprint arXiv:2206.05263, 2022.
  43. Generalized independent noise condition for estimating latent variable causal graphs. In NeurIPS, 2020.
  44. Identification of linear non-gaussian latent hierarchical structure. In International Conference on Machine Learning, pp.  24370–24387. PMLR, 2022a.
  45. Multi-domain image generation and translation with identifiability guarantees. In The Eleventh International Conference on Learning Representations, 2022b.
  46. Learning temporally causal latent processes from general temporal data. arXiv preprint arXiv:2110.05428, 2021.
  47. Learning latent causal dynamics. arXiv preprint arXiv:2202.04828, 2022.
  48. Identifiability guarantees for causal disentanglement from soft interventions. arXiv preprint arXiv:2307.06250, 2023.
  49. On the identifiability of the post-nonlinear causal model. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada, 2009.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now