Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards the Reusability and Compositionality of Causal Representations (2403.09830v1)

Published 14 Mar 2024 in cs.LG and cs.AI

Abstract: Causal Representation Learning (CRL) aims at identifying high-level causal factors and their relationships from high-dimensional observations, e.g., images. While most CRL works focus on learning causal representations in a single environment, in this work we instead propose a first step towards learning causal representations from temporal sequences of images that can be adapted in a new environment, or composed across multiple related environments. In particular, we introduce DECAF, a framework that detects which causal factors can be reused and which need to be adapted from previously learned causal representations. Our approach is based on the availability of intervention targets, that indicate which variables are perturbed at each time step. Experiments on three benchmark datasets show that integrating our framework with four state-of-the-art CRL approaches leads to accurate representations in a new environment with only a few samples.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013.
  2. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013.
  3. Counterfactuals uncover the modular structure of deep generative models. arXiv preprint arXiv:1812.03253, 2018.
  4. Weakly supervised causal representation learning. In Advances in Neural Information Processing Systems, 2022.
  5. Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3):287–314, 1994.
  6. A model for reasoning about persistence and causation. Computational Intelligence, 5(2):142–150, 1989. https://doi.org/10.1111/j.1467-8640.1989.tb00324.x.
  7. On the transfer of disentangled representations in realistic settings. In International Conference on Learning Representations, 2021.
  8. Generalization and robustness implications in object-centric learning. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162, pages 5221–5285. PMLR, July 2022.
  9. Frederick Eberhardt. Causation and intervention. Unpublished doctoral dissertation, Carnegie Mellon University, page 93, 2007.
  10. Made: Masked autoencoder for distribution estimation. In International conference on machine learning, pages 881–889. PMLR, 2015.
  11. Recurrent independent mechanisms. arXiv preprint arXiv:1909.10893, 2019.
  12. Independent mechanism analysis, a new concept? Advances in neural information processing systems, 34:28233–28248, 2021.
  13. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  14. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.
  15. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations, 2017.
  16. Experiment selection for causal discovery. Journal of Machine Learning Research, 14:3041–3071, 2013.
  17. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
  18. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429–439, 1999.
  19. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859–868. PMLR, 2019.
  20. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207–2217. PMLR, 2020.
  21. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
  22. Improved variational inference with inverse autoregressive flow. Advances in neural information processing systems, 29, 2016.
  23. Synergies between disentanglement and sparsity: a multi-task learning perspective. arXiv preprint arXiv:2211.14666, 2022a.
  24. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA. In First Conference on Causal Learning and Reasoning, 2022b.
  25. Intervention Design for Causal Representation Learning. In UAI 2022 Workshop on Causal Representation Learning, 2022a.
  26. CITRIS: Causal identifiability from temporal intervened sequences. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 13557–13603. PMLR, 17–23 Jul 2022b.
  27. Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems. In The Eleventh International Conference on Learning Representations, 2023a.
  28. BISCUIT: Causal representation learning from binary interactions. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216 of Proceedings of Machine Learning Research, pages 1263–1273. PMLR, 31 Jul–04 Aug 2023b.
  29. On the fairness of disentangled representations. Advances in neural information processing systems, 32, 2019a.
  30. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pages 4114–4124. PMLR, 2019b.
  31. Weakly-supervised disentanglement without compromises. In International Conference on Machine Learning, pages 6348–6359. PMLR, 2020a.
  32. Disentangling factors of variations using few labels. In International Conference on Learning Representations, 2020b. URL https://openreview.net/forum?id=SygagpEKwB.
  33. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  34. Joint causal inference from multiple contexts. The Journal of Machine Learning Research, 21(1):3919–4026, 2020.
  35. Kevin Patrick Murphy. Dynamic bayesian networks: representation, inference and learning. University of California, Berkeley, 2002.
  36. Learning independent causal mechanisms. In International Conference on Machine Learning, pages 4036–4044. PMLR, 2018.
  37. Judea Pearl. Causality. Cambridge university press, 2009.
  38. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017.
  39. Variational inference with normalizing flows. In International conference on machine learning, pages 1530–1538. PMLR, 2015.
  40. Disentanglement of correlated factors via hausdorff factorized support. arXiv preprint arXiv:2210.07347, 2022.
  41. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  42. Learning causal graphs with small interventions. Advances in Neural Information Processing Systems, 28, 2015.
  43. C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101, 1904.
  44. On disentangled representations learned from correlated data. In International Conference on Machine Learning, pages 10401–10412. PMLR, 2021.
  45. Are disentangled representations helpful for abstract visual reasoning? Advances in Neural Information Processing Systems, 32, 2019.
  46. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34:16451–16467, 2021.
  47. S. Wright. Correlation and Causation. 1921.
  48. Temporally Disentangled Representation Learning. In Advances in Neural Information Processing Systems 35, NeurIPS, 2022a.
  49. Learning Temporally Causal Latent Processes from General Temporal Data. In International Conference on Learning Representations, 2022b.
  50. On the identifiability of nonlinear ica: Sparsity and beyond. In Advances in Neural Information Processing Systems.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets