Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DiffusionCounterfactuals: Inferring High-dimensional Counterfactuals with Guidance of Causal Representations (2407.20553v1)

Published 30 Jul 2024 in cs.LG and stat.ME

Abstract: Accurate estimation of counterfactual outcomes in high-dimensional data is crucial for decision-making and understanding causal relationships and intervention outcomes in various domains, including healthcare, economics, and social sciences. However, existing methods often struggle to generate accurate and consistent counterfactuals, particularly when the causal relationships are complex. We propose a novel framework that incorporates causal mechanisms and diffusion models to generate high-quality counterfactual samples guided by causal representation. Our approach introduces a novel, theoretically grounded training and sampling process that enables the model to consistently generate accurate counterfactual high-dimensional data under multiple intervention steps. Experimental results on various synthetic and real benchmarks demonstrate the proposed approach outperforms state-of-the-art methods in generating accurate and high-quality counterfactuals, using different evaluation metrics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Constructing effective personalized policies using counterfactual inference from biased data sets with many features. Machine Learning, 108(6):945–970, Jun 2019.
  2. A simple framework for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR, 13–18 Jul 2020.
  3. Inference on counterfactual distributions. Econometrica, 81(6):2205–2268, 2013.
  4. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 8780–8794. Curran Associates, Inc., 2021.
  5. Deep end-to-end causal inference. In NeurIPS 2022 Workshop on Causality for Real-world Impact, 2022.
  6. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  7. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  8. beta-vae: Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
  9. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020.
  10. Diffusion models for counterfactual explanations. In ACCV, 2022.
  11. Avoiding discrimination through causal reasoning. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  12. CausalGAN: Learning causal implicit generative models with adversarial training. In International Conference on Learning Representations, 2018.
  13. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
  14. Improved precision and recall metric for assessing generative models. CoRR, abs/1904.06991, 2019.
  15. Jack S. Levy. Counterfactuals, causal inference, and historical analysis. Security Studies, 24(3):378–402, 2015.
  16. Diffusion-LM improves controllable text generation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
  17. Generating images with sparse representations. ArXiv, abs/2103.03841, 2021.
  18. A graph autoencoder approach to causal structure learning. arXiv preprint arXiv:1911.07420, 2019.
  19. Improved Denoising Diffusion Probabilistic Models. arxiv pre-print, 12 2021.
  20. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of the 39th International Conference on Machine, 2022.
  21. Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2nd edition, 2009.
  22. Improving the accuracy of medical diagnosis with causal machine learning. Nature Communications, 11(1):3923, Aug 2020.
  23. High-resolution image synthesis with latent diffusion models, 2021.
  24. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proc. of Medical Image Computing and Computer-Assisted Intervention, volume 9351, pages 234–241. Springer, 2015.
  25. Diffusion causal models for counterfactual estimation. In First Conference on Causal Learning and Reasoning, 2022.
  26. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  27. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 2256–2265, Lille, France, 07–09 Jul 2015. PMLR.
  28. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
  29. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
  30. Attention Is All You Need. In Advances in neural information processing systems, pages 5998–6008, 2017.
  31. Causalvae: Disentangled representation learning via neural structural causal models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 9593–9602. Computer Vision Foundation / IEEE, 2021.
  32. DAGs with NO TEARS: Continuous Optimization for Structure Learning. In Advances in Neural Information Processing Systems, 2018.
  33. Do-operation guided causal representation learning with reduced supervision strength. In NeurIPS 2022 Workshop on Causality for Real-world Impact, 2022.
  34. Shadow datasets, new challenging datasets for causal representation learning, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now