Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Implicit Causal Representation Learning via Switchable Mechanisms (2402.11124v4)

Published 16 Feb 2024 in cs.LG

Abstract: Learning causal representations from observational and interventional data in the absence of known ground-truth graph structures necessitates implicit latent causal representation learning. Implicit learning of causal mechanisms typically involves two categories of interventional data: hard and soft interventions. In real-world scenarios, soft interventions are often more realistic than hard interventions, as the latter require fully controlled environments. Unlike hard interventions, which directly force changes in a causal variable, soft interventions exert influence indirectly by affecting the causal mechanism. However, the subtlety of soft interventions impose several challenges for learning causal models. One challenge is that soft intervention's effects are ambiguous, since parental relations remain intact. In this paper, we tackle the challenges of learning causal models using soft interventions while retaining implicit modelling. We propose ICLR-SM, which models the effects of soft interventions by employing a causal mechanism switch variable designed to toggle between different causal mechanisms. In our experiments, we consistently observe improved learning of identifiable, causal representations, compared to baseline approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Bernhard Schölkopf. Causality for machine learning. CoRR, abs/1911.10500, 2019.
  2. Judea Pearl. Causality, cambridge university press (2000). Artif. Intell., 169(2):174–179, 2005.
  3. Causal inference in statistics: A primer. John Wiley and Sons, 2016.
  4. Causality-based feature selection: Methods and evaluations. ACM Comput. Surv., 53(5), 2020. ISSN 0360-0300. doi:10.1145/3409382.
  5. Causalvae: Disentangled representation learning via neural structural causal models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 9593–9602. Computer Vision Foundation / IEEE, 2021a. doi:10.1109/CVPR46437.2021.00947.
  6. Interventional causal representation learning. In International Conference on Machine Learning, ICML, volume 202 of Proceedings of Machine Learning Research, pages 372–407. PMLR, 2023.
  7. Weakly supervised causal representation learning. In NeurIPS, 2022.
  8. CITRIS: causal identifiability from temporal intervened sequences. In International Conference on Machine Learning, ICML, volume 162 of Proceedings of Machine Learning Research, pages 13557–13603. PMLR, 2022a.
  9. Learning causal representations for robust domain adaptation. IEEE Transactions on Knowledge and Data Engineering, pages 1–1, 2021b. doi:10.1109/TKDE.2021.3119185.
  10. DAG-GNN: DAG structure learning with graph neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 7154–7163. PMLR, 2019.
  11. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA. In 1st Conference on Causal Learning and Reasoning, CLeaR, volume 177 of Proceedings of Machine Learning Research, pages 428–484. PMLR, 2022.
  12. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021. doi:10.1109/JPROC.2021.3058954.
  13. General transportability of soft interventions: Completeness results. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 2020.
  14. Causal machine learning: A survey and open problems. CoRR, abs/2206.15475, 2022. doi:10.48550/arXiv.2206.15475.
  15. Invariant causal representation learning for out-of-distribution generalization. In The Tenth International Conference on Learning Representations, ICLR, 2022.
  16. Weakly supervised disentangled generative causal representation learning. J. Mach. Learn. Res., 23:241:1–241:55, 2022.
  17. Causal discovery in heterogeneous environments under the sparse mechanism shift hypothesis. In NeurIPS, 2022.
  18. Dags with NO TEARS: continuous optimization for structure learning. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems NeurIPS, pages 9492–9503, 2018.
  19. Causal discovery from soft interventions with unknown targets: Characterization and learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 2020.
  20. Causal discovery from a mixture of experimental and observational data, 2013.
  21. Identifiability guarantees for causal disentanglement from soft interventions. CoRR, abs/2307.06250, 2023a. doi:10.48550/arXiv.2307.06250.
  22. Causal component analysis, 2023.
  23. Identifiability guarantees for causal disentanglement from soft interventions, 2023b.
  24. Linear causal disentanglement via interventions, 2023.
  25. Score-based causal representation learning with interventions, 2023.
  26. Weakly-supervised disentanglement without compromises. In Proceedings of the 37th International Conference on Machine Learning,ICML, volume 119 of Proceedings of Machine Learning Research, pages 6348–6359. PMLR, 2020.
  27. Efficient neural causal discovery without acyclicity constraints. In The Tenth International Conference on Learning Representations, ICLR. OpenReview.net, 2022b.
  28. Differentiable DAG sampling. In The Tenth International Conference on Learning Representations, ICLR. OpenReview.net, 2022.
  29. Learning linear causal representations from interventions under general nonlinear mixing, 2023.
  30. Nonparametric identifiability of causal representations from unknown interventions, 2023.
  31. Generative causal representation learning for out-of-distribution motion forecasting. In International Conference on Machine Learning, ICML, volume 202 of Proceedings of Machine Learning Research, pages 31596–31612. PMLR, 2023.
  32. On the identifiability of nonlinear ICA: sparsity and beyond. In NeurIPS, 2022.
  33. On the identifiability and estimation of causal location-scale noise models. In International Conference on Machine Learning, ICML, volume 202 of Proceedings of Machine Learning Research, pages 14316–14332. PMLR, 2023.
  34. Causal triplet: An open challenge for intervention-centric causal representation learning. In Conference on Causal Learning and Reasoning, CLeaR, volume 213 of Proceedings of Machine Learning Research, pages 553–573. PMLR, 2023.
  35. Procthor: Large-scale embodied ai using procedural generation. Advances in Neural Information Processing Systems, 35:5982–5994, 2022.
  36. Rescaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100. Int. J. Comput. Vis., 130(1):33–55, 2022. doi:10.1007/s11263-021-01531-2.
  37. Cian Eastwood and Christopher K. I. Williams. A framework for the quantitative evaluation of disentangled representations. In 6th International Conference on Learning Representations, ICLR, 2018.
  38. beta-vae: Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets