Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Linear Causal Disentanglement via Interventions (2211.16467v3)

Published 29 Nov 2022 in stat.ML and cs.LG

Abstract: Causal disentanglement seeks a representation of data involving latent variables that relate to one another via a causal model. A representation is identifiable if both the latent model and the transformation from latent to observed variables are unique. In this paper, we study observed variables that are a linear transformation of a linear latent causal model. Data from interventions are necessary for identifiability: if one latent variable is missing an intervention, we show that there exist distinct models that cannot be distinguished. Conversely, we show that a single intervention on each latent variable is sufficient for identifiability. Our proof uses a generalization of the RQ decomposition of a matrix that replaces the usual orthogonal and upper triangular conditions with analogues depending on a partial order on the rows of the matrix, with partial order determined by a latent causal model. We corroborate our theoretical results with a method for causal disentanglement that accurately recovers a latent causal model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Identification of partially observed linear causal models: Graphical conditions for the non-gaussian and heterogeneous cases. Advances in Neural Information Processing Systems, 34:22822–22833, 2021.
  2. Properties from mechanisms: an equivariance perspective on identifiable representation learning. In International Conference on Learning Representations, 2021.
  3. Weakly supervised representation learning with sparse perturbations. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022a. URL https://openreview.net/forum?id=6ZI4iF_T7t.
  4. Interventional causal representation learning. In NeurIPS 2022 Workshop on Neuro Causal and Symbolic AI (nCSI), 2022b.
  5. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25–29, 2000.
  6. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013.
  7. Weakly supervised causal representation learning. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022. URL https://openreview.net/forum?id=rUXOUBuUcg5.
  8. Triad constraints for learning causal structure of latent variables. Advances in neural information processing systems, 32, 2019.
  9. Network structure learning under uncertain interventions. Journal of the American Statistical Association, pp.  1–12, 2022.
  10. Comon, P. Independent component analysis, a new concept? Signal processing, 36(3):287–314, 1994.
  11. Davidson-Pilon, C. lifelines: survival analysis in python. Journal of Open Source Software, 4(40):1317, 2019.
  12. Algebraic factor analysis: tetrads, pentads and beyond. Probability Theory and Related Fields, 138:463–493, 2007.
  13. Moments of minors of wishart matrices. 2008.
  14. On the number of experiments sufficient and in the worst case necessary to identify all causal relations among n variables. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pp.  178–184, 2005.
  15. Greville, T. N. E. Note on the generalized inverse of a matrix product. Siam Review, 8(4):518–521, 1966.
  16. Anchored discrete factor analysis. arXiv preprint arXiv:1511.03299, 2015.
  17. Characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs. The Journal of Machine Learning Research, 13(1):2409–2464, 2012.
  18. Kras mutation: from undruggable to druggable in cancer. Signal transduction and targeted Therapy, 6(1):386, 2021.
  19. Experiment selection for causal discovery. Journal of Machine Learning Research, 14:3041–3071, 2013.
  20. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.  859–868. PMLR, 2019.
  21. Causal discovery from soft interventions with unknown targets: Characterization and learning. Advances in neural information processing systems, 33:9551–9561, 2020.
  22. Variational autoencoders and nonlinear ICA: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pp.  2207–2217. PMLR, 2020.
  23. Learning latent causal graphs via mixture oracles. Advances in Neural Information Processing Systems, 34:18087–18101, 2021.
  24. Enrichr: a comprehensive gene set enrichment analysis web server 2016 update. Nucleic acids research, 44(W1):W90–W97, 2016.
  25. An integrated tcga pan-cancer clinical data resource to drive high-quality survival outcome analytics. Cell, 173(2):400–416, 2018.
  26. Weight-variant latent causal models. arXiv preprint arXiv:2208.14153, 2022.
  27. Identifiable deep generative models via sparse decoding. arXiv preprint arXiv:2110.10804, 2021.
  28. Anchored causal inference in the presence of measurement error. In Conference on uncertainty in artificial intelligence, pp. 619–628. PMLR, 2020.
  29. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference, 2010.
  30. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(2), 2006.
  31. Causal structure learning: A combinatorial perspective. Foundations of Computational Mathematics, 2022.
  32. Permutation-based causal structure learning with unknown intervention targets. In Conference on Uncertainty in Artificial Intelligence, pp. 1039–1048. PMLR, 2020.
  33. Causal structure discovery between clusters of nodes induced by latent factors. In Conference on Causal Learning and Reasoning, pp. 669–687. PMLR, 2022.
  34. Scim: universal single-cell matching with unpaired feature sets. Bioinformatics, 36(Supplement_2):i919–i927, 2020.
  35. Numerical linear algebra, volume 50. Siam, 1997.
  36. Massively parallel phenotyping of coding variants in cancer with perturb-seq. Nature Biotechnology, pp.  1–10, 2022.
  37. Scalable intervention target estimation in linear models. Advances in Neural Information Processing Systems, 34:1494–1505, 2021.
  38. Equivalence and synthesis of causal models. In Proceedings of the Sixth Annual Conference on Uncertainty in Artificial Intelligence, pp.  255–270, 1990.
  39. Generalized independent noise condition for estimating latent variable causal graphs. Advances in Neural Information Processing Systems, 33:14891–14902, 2020.
  40. Identification of linear non-Gaussian latent hierarchical structure. In International Conference on Machine Learning, pp. 24370–24387. PMLR, 2022.
  41. Characterizing and learning equivalence classes of causal DAGs under interventions. In International Conference on Machine Learning, pp. 5541–5550. PMLR, 2018.
  42. CausalVAE: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  9593–9602, 2021.
  43. Contrastive learning inverts the data generating process. In International Conference on Machine Learning, pp. 12979–12990. PMLR, 2021.
Citations (54)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now