Mitigating attribute amplification in counterfactual image generation (2403.09422v1)
Abstract: Causal generative modelling is gaining interest in medical imaging due to its ability to answer interventional and counterfactual queries. Most work focuses on generating counterfactual images that look plausible, using auxiliary classifiers to enforce effectiveness of simulated interventions. We investigate pitfalls in this approach, discovering the issue of attribute amplification, where unrelated attributes are spuriously affected during interventions, leading to biases across protected characteristics and disease status. We show that attribute amplification is caused by the use of hard labels in the counterfactual training process and propose soft counterfactual fine-tuning to mitigate this issue. Our method substantially reduces the amplification effect while maintaining effectiveness of generated images, demonstrated on a large chest X-ray dataset. Our work makes an important advancement towards more faithful and unbiased causal modelling in medical imaging.
- On Pearl’s Hierarchy and the Foundations of Causal Inference, pp. 507–556. Association for Computing Machinery, New York, NY, USA, 1 edition, 2022. ISBN 9781450395861. URL https://doi.org/10.1145/3501714.3501743.
- Representation learning: a review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798—1828, August 2013. ISSN 0162-8828. doi: 10.1109/tpami.2013.50.
- High fidelity image counterfactuals with probabilistic causal models. In International Conference on Machine Learning, pp. 7390–7425. PMLR, 2023.
- Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780–8794, 2021.
- Deep end-to-end causal inference. arXiv preprint arXiv:2202.02195, 2022.
- Algorithmic encoding of protected characteristics in chest x-ray disease detection models. Ebiomedicine, 89, 2023.
- Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
- Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
- Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):1–8, 2019.
- Learning neural causal models from unknown interventions. arXiv preprint arXiv:1910.01075, 2019.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- Causalgan: Learning causal implicit generative models with adversarial training. arXiv preprint arXiv:1709.02023, 2017.
- CausalGAN: Learning causal implicit generative models with adversarial training. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=BJE-4xW0W.
- Causal effect inference with deep latent-variable models. Advances in neural information processing systems, 30, 2017.
- Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
- Measuring axiomatic soundness of counterfactual image models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=lZOUQQvwI3q.
- Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33:857–869, 2020.
- Judea Pearl. Causality. Cambridge university press, 2009.
- Variational inference with normalizing flows. In International conference on machine learning, pp. 1530–1538. PMLR, 2015.
- Diffusion models for causal discovery via topological ordering. arXiv preprint arXiv:2210.06201, 2022.
- Bernhard Schölkopf. Causality for Machine Learning, pp. 765–804. Association for Computing Machinery, New York, NY, USA, 1 edition, 2022. ISBN 9781450395861. URL https://doi.org/10.1145/3501714.3501755.
- Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
- Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256–2265. PMLR, 2015.
- Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015.
- Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=PxTIG12RRHS.
- Implicit causal models for genome-wide association studies. arXiv preprint arXiv:1710.10742, 2017.
- Conditional density estimation with bayesian normalising flows. arXiv preprint arXiv:1802.04908, 2018.
- The causal-neural connection: Expressiveness, learnability, and inference. Advances in Neural Information Processing Systems, 34:10823–10836, 2021.
- Neural causal models for counterfactual identification and estimation. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=vouQcZS8KfW.
- Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9593–9602, 2021.
- Relating graph neural networks to structural causal models. arXiv preprint arXiv:2109.04173, 2021.
- Towards causal foundation model: on duality between causal inference and attention. arXiv preprint arXiv:2310.00809, 2023.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.