Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 178 tok/s
Gemini 2.5 Pro 50 tok/s Pro
GPT-5 Medium 38 tok/s Pro
GPT-5 High 40 tok/s Pro
GPT-4o 56 tok/s Pro
Kimi K2 191 tok/s Pro
GPT OSS 120B 445 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

Longitudinal Causal Image Synthesis (2410.17691v1)

Published 23 Oct 2024 in eess.IV, cs.CV, and q-bio.NC

Abstract: Clinical decision-making relies heavily on causal reasoning and longitudinal analysis. For example, for a patient with Alzheimer's disease (AD), how will the brain grey matter atrophy in a year if intervened on the A-beta level in cerebrospinal fluid? The answer is fundamental to diagnosis and follow-up treatment. However, this kind of inquiry involves counterfactual medical images which can not be acquired by instrumental or correlation-based image synthesis models. Yet, such queries require counterfactual medical images, not obtainable through standard image synthesis models. Hence, a causal longitudinal image synthesis (CLIS) method, enabling the synthesis of such images, is highly valuable. However, building a CLIS model confronts three primary yet unmet challenges: mismatched dimensionality between high-dimensional images and low-dimensional tabular variables, inconsistent collection intervals of follow-up data, and inadequate causal modeling capability of existing causal graph methods for image data. In this paper, we established a tabular-visual causal graph (TVCG) for CLIS overcoming these challenges through a novel integration of generative imaging, continuous-time modeling, and structural causal models combined with a neural network. We train our CLIS based on the ADNI dataset and evaluate it on two other AD datasets, which illustrate the outstanding yet controllable quality of the synthesized images and the contributions of synthesized MRI to the characterization of AD progression, substantiating the reliability and utility in clinics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (114)
  1. J. Peters et al., “Causal inference on discrete data using additive noise models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2436–2450, 2011.
  2. J. Lin et al., “Towards causality-aware inferring: A sequential discriminative approach for medical diagnosis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 11, pp. 13363–13375, 2023.
  3. K. Olesen, “Causal probabilistic networks with both discrete and continuous variables,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 3, pp. 275–279, 1993.
  4. K. Li et al., “Prediction of human activity by discovering temporal sequence patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 8, pp. 1644–1657, 2014.
  5. C. A. Kulikowski, “Artificial intelligence methods and systems for medical consultation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2, no. 5, pp. 464–476, 1980.
  6. K. Oh et al., “Learn-explain-reinforce: Counterfactual reasoning and its guidance to reinforce an alzheimer’s disease diagnosis model,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4843–4857, 2023.
  7. M. Ben-Bassat et al., “Pattern-based interactive diagnosis of multiple disorders: The medas system,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2, no. 2, pp. 148–160, 1980.
  8. Y. Pan et al., “Disease-image-specific learning for diagnosis-oriented neuroimage synthesis with incomplete multi-modality data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 10, pp. 6839–6853, 2022.
  9. G. Pu et al., “Controllable image synthesis with attribute-decomposed gan,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 2, pp. 1514–1532, 2023.
  10. C. R. Green et al., “Functional decline in alzheimer’s disease: a longitudinal study,” Journal of the American Geriatrics Society, vol. 41, no. 6, pp. 654–661, 1993.
  11. D. K. Johnson et al., “Longitudinal study of the transition from healthy aging to alzheimer disease,” Archives of neurology, vol. 66, no. 10, pp. 1254–1259, 2009.
  12. E. McDade et al., “Longitudinal cognitive and biomarker changes in dominantly inherited alzheimer disease,” Neurology, vol. 91, no. 14, pp. e1295–e1306, 2018.
  13. T. Karras et al., “A style-based generator architecture for generative adversarial networks,” in IEEE/CVF CVPR, pp. 4401–4410, 2019.
  14. S. Hong et al., “3d-stylegan: A style-based generative adversarial network for generative modeling of three-dimensional medical images,” in Deep Generative Models, and Data Augmentation, Labelling, and Imperfections: First Workshop, DGM4MICCAI 2021, and First Workshop, DALI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, October 1, 2021, Proceedings 1, pp. 24–34, Springer, 2021.
  15. T. Karras et al., “Analyzing and improving the image quality of stylegan,” in IEEE/CVF CVPR, pp. 8110–8119, 2020.
  16. E. Richardson et al., “Encoding in style: a stylegan encoder for image-to-image translation,” in IEEE/CVF CVPR, pp. 2287–2296, 2021.
  17. O. Tov et al., “Designing an encoder for stylegan image manipulation,” ACM Transactions on Graphics (TOG), vol. 40, no. 4, pp. 1–14, 2021.
  18. J. Pearl, Causality. Cambridge University Press, 2 ed., 2009.
  19. Y. Liu et al., “Learning temporal causal graphs for relational time-series analysis,” in ICML, pp. 687–694, 2010.
  20. M. Parascandola et al., “Causation in epidemiology,” Journal of Epidemiology & Community Health, vol. 55, no. 12, pp. 905–912, 2001.
  21. H. Wold, “Causality and econometrics,” Econometrica: Journal of the Econometric Society, pp. 162–177, 1954.
  22. B. Kuipers et al., “Causal reasoning in medicine: analysis of a protocol,” Cognitive Science, vol. 8, no. 4, pp. 363–385, 1984.
  23. N. Pawlowski et al., “Deep structural causal models for tractable counterfactual inference,” Proc. Adv. Neural Inf. Process. Syst., vol. 33, pp. 857–869, 2020.
  24. C. K. Assaad et al., “Survey and evaluation of causal discovery methods for time series,” Journal of Artificial Intelligence Research, vol. 73, pp. 767–819, 2022.
  25. P. S. Aisen et al., “Clinical core of the alzheimer’s disease neuroimaging initiative: progress and plans,” Alzheimer’s & Dementia, vol. 6, no. 3, pp. 239–246, 2010.
  26. D. L. Beekly et al., “The national alzheimer’s coordinating center (nacc) database: the uniform data set,” Alzheimer Disease & Associated Disorders, vol. 21, no. 3, pp. 249–258, 2007.
  27. P. J. LaMontagne et al., “Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease,” MedRxiv, pp. 2019–12, 2019.
  28. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  29. O. Ronneberger et al., “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI, pp. 234–241, Springer, 2015.
  30. I. Goodfellow et al., “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  31. J. Ho et al., “Denoising diffusion probabilistic models,” Proc. Adv. Neural Inf. Process. Syst., vol. 33, pp. 6840–6851, 2020.
  32. C. F. Baumgartner et al., “Visual feature attribution using wasserstein gans,” in IEEE/CVF CVPR, pp. 8309–8319, 2018.
  33. S. Momeni et al., “Generative model of brain microbleeds for mri detection of vascular marker of neurodegenerative diseases,” Frontiers in Neuroscience, vol. 15, p. 778767, 2021.
  34. J. M. Edmund et al., “A review of substitute ct generation for mri-only radiation therapy,” Radiation Oncology, vol. 12, pp. 1–15, 2017.
  35. V. M. H. Phan et al., “Structure-preserving synthesis: Maskgan for unpaired mr-ct translation,” in MICCAI, pp. 56–65, Springer, 2023.
  36. W. Lin et al., “Bidirectional mapping of brain mri and pet with 3d reversible gan for the diagnosis of alzheimer’s disease,” Frontiers in Neuroscience, vol. 15, p. 646013, 2021.
  37. S. Hu et al., “Bidirectional mapping generative adversarial networks for brain mr to pet synthesis,” IEEE Trans. Med. Imag, vol. 41, no. 1, pp. 145–157, 2022.
  38. Y. Pan et al., “Disease-image specific generative adversarial network for brain disease diagnosis with incomplete multi-modal neuroimages,” in MICCAI, pp. 137–145, Springer, 2019.
  39. Y. Wang et al., “3d auto-context-based locality adaptive multi-modality gans for pet synthesis,” IEEE Trans. Med. Imag, vol. 38, no. 6, pp. 1328–1339, 2019.
  40. Y. Li et al., “A review of the deep learning methods for medical images super resolution problems,” Irbm, vol. 42, no. 2, pp. 120–133, 2021.
  41. S. U. Dar et al., “Image synthesis in multi-contrast mri with conditional generative adversarial networks,” IEEE Trans. Med. Imag, vol. 38, no. 10, pp. 2375–2388, 2019.
  42. Y. Zhang et al., “Unified multi-modal image synthesis for missing modality imputation,” arXiv preprint arXiv:2304.05340, 2023.
  43. C. Fan et al., “Tr-gan: Multi-session future mri prediction with temporal recurrent generative adversarial network,” IEEE Trans. Med. Imag, vol. 41, no. 8, pp. 1925–1937, 2022.
  44. Y. Zhao et al., “Prediction of alzheimer’s disease progression with multi-information generative adversarial network,” IEEE J. Biomed. Health Inform., vol. 25, no. 3, pp. 711–719, 2020.
  45. Z. Ning et al., “Ldgan: Longitudinal-diagnostic generative adversarial network for disease progression prediction with missing structural mri,” in MLMI, pp. 170–179, Springer, 2020.
  46. J. C. Reinhold et al., “A structural causal model for mr images of multiple sclerosis,” in MICCAI, pp. 782–792, Springer, 2021.
  47. A. Kumar et al., “Counterfactual image synthesis for discovery of personalized predictive image markers,” in MICCAI Workshop on Medical Image Assisted Blomarkers’ Discovery, pp. 113–124, Springer, 2022.
  48. P. Sanchez et al., “What is healthy? generative counterfactual diffusion for lesion localization,” in MICCAI Workshop on Deep Generative Models, pp. 34–44, Springer, 2022.
  49. C. L. Van Broeckhoven, “Molecular genetics of alzheimer disease: identification of genes and gene mutations,” European neurology, vol. 35, no. 1, pp. 8–19, 1995.
  50. E. Levy-Lahad et al., “Candidate gene for the chromosome 1 familial alzheimer’s disease locus,” Science, vol. 269, no. 5226, pp. 973–977, 1995.
  51. N. D. Beckmann et al., “Multiscale causal networks identify vgf as a key regulator of alzheimer’s disease,” Nature communications, vol. 11, no. 1, p. 3942, 2020.
  52. T. L. Spires-Jones et al., “The intersection of amyloid beta and tau at synapses in alzheimer’s disease,” Neuron, vol. 82, no. 4, pp. 756–771, 2014.
  53. H.-C. Huang et al., “Accumulated amyloid-β𝛽\betaitalic_β peptide and hyperphosphorylated tau protein: relationship and links in alzheimer’s disease,” Journal of Alzheimer’s disease, vol. 16, no. 1, pp. 15–27, 2009.
  54. R. Rajmohan et al., “Amyloid-beta and phosphorylated tau accumulations cause abnormalities at synapses of alzheimer’s disease neurons,” Journal of Alzheimer’s Disease, vol. 57, no. 4, pp. 975–999, 2017.
  55. P. H. Reddy et al., “Amyloid beta and phosphorylated tau-induced defective autophagy and mitophagy in alzheimer’s disease,” Cells, vol. 8, no. 5, p. 488, 2019.
  56. P. A. Thomann et al., “Association of total tau and phosphorylated tau 181 protein levels in cerebrospinal fluid with cerebral atrophy in mild cognitive impairment and alzheimer disease,” Journal of Psychiatry and Neuroscience, vol. 34, no. 2, pp. 136–142, 2009.
  57. W. Pelkmans et al., “Tau-related grey matter network breakdown across the alzheimer’s disease continuum,” Alzheimer’s Research & Therapy, vol. 13, pp. 1–11, 2021.
  58. A. Bejanin et al., “Tau pathology and neurodegeneration contribute to cognitive impairment in alzheimer’s disease,” Brain, vol. 140, no. 12, pp. 3286–3300, 2017.
  59. V. Kasantikul et al., “Relation of age and cerebral ventricle size to central canal in man: Morphological analysis,” Journal of neurosurgery, vol. 51, no. 1, pp. 85–93, 1979.
  60. K. Skullerud, “Variations in the size of the human brain. influence of age, sex, body length, body mass index, alcoholism, alzheimer changes, and cerebral atherosclerosis.,” Acta neurologica Scandinavica. Supplementum, vol. 102, pp. 1–94, 1985.
  61. M. P. Earnest et al., “Cortical atrophy, ventricular enlargement and intellectual impairment in the aged,” Neurology, vol. 29, no. 8, pp. 1138–1138, 1979.
  62. C. A. Raji et al., “Age, alzheimer disease, and brain structure,” Neurology, vol. 73, no. 22, pp. 1899–1905, 2009.
  63. R. Guerreiro et al., “The age factor in alzheimer’s disease,” Genome medicine, vol. 7, no. 1, pp. 1–3, 2015.
  64. C. Kawas et al., “Age-specific incidence rates of alzheimer’s disease: the baltimore longitudinal study of aging,” Neurology, vol. 54, no. 11, pp. 2072–2077, 2000.
  65. Y. Stern et al., “Influence of education and occupation on the incidence of alzheimer’s disease,” Jama, vol. 271, no. 13, pp. 1004–1010, 1994.
  66. M. Paradise et al., “Systematic review of the effect of education on survival in alzheimer’s disease,” International psychogeriatrics, vol. 21, no. 1, pp. 25–32, 2009.
  67. C. M. Mazure et al., “Sex differences in alzheimer’s disease and other dementias,” The Lancet Neurology, vol. 15, no. 5, pp. 451–452, 2016.
  68. R. Li et al., “Sex differences in cognitive impairment and alzheimer’s disease,” Frontiers in neuroendocrinology, vol. 35, no. 3, pp. 385–403, 2014.
  69. L. Guo et al., “Sex differences in alzheimer’s disease: Insights from the multiomics landscape,” Biological psychiatry, vol. 91, no. 1, pp. 61–71, 2022.
  70. G. Karas et al., “Global and local gray matter loss in mild cognitive impairment and alzheimer’s disease,” Neuroimage, vol. 23, no. 2, pp. 708–716, 2004.
  71. G. Karas et al., “A comprehensive study of gray matter loss in patients with alzheimer’s disease using optimized voxel-based morphometry,” Neuroimage, vol. 18, no. 4, pp. 895–907, 2003.
  72. S. A. Rombouts et al., “Unbiased whole-brain analysis of gray matter loss in alzheimer’s disease,” Neuroscience letters, vol. 285, no. 3, pp. 231–233, 2000.
  73. L. Ferrarini et al., “Shape differences of the brain ventricles in alzheimer’s disease,” Neuroimage, vol. 32, no. 3, pp. 1060–1069, 2006.
  74. P. M. Thompson et al., “Mapping hippocampal and ventricular change in alzheimer disease,” Neuroimage, vol. 22, no. 4, pp. 1754–1766, 2004.
  75. Y.-Y. Chou et al., “Mapping correlations between ventricular expansion and csf amyloid and tau biomarkers in 240 subjects with alzheimer’s disease, mild cognitive impairment and elderly controls,” Neuroimage, vol. 46, no. 2, pp. 394–410, 2009.
  76. M. De Leon et al., “Longitudinal cerebrospinal fluid tau load increases in mild cognitive impairment,” Neuroscience letters, vol. 333, no. 3, pp. 183–186, 2002.
  77. L. Letenneur, V. Gilleron, et al., “Are sex and educational level independent predictors of dementia and alzheimer’s disease? incidence data from the paquid project,” Journal of Neurology, Neurosurgery & Psychiatry, vol. 66, no. 2, pp. 177–183, 1999.
  78. L. Fratiglioni et al., “Prevalence of alzheimer’s disease and other dementias in an elderly urban population: relationship with age, sex, and education,” Neurology, vol. 41, no. 12, pp. 1886–1886, 1991.
  79. B. C. Riedel et al., “Age, apoe and sex: triad of risk of alzheimer’s disease,” The Journal of steroid biochemistry and molecular biology, vol. 160, pp. 134–147, 2016.
  80. R. Mayeux et al., “Plasma aβ𝛽\betaitalic_β40 and aβ𝛽\betaitalic_β42 and alzheimer’s disease: relation to age, mortality, and risk,” Neurology, vol. 61, no. 9, pp. 1185–1190, 2003.
  81. E. Dicks et al., “Modeling grey matter atrophy as a function of time, aging or cognitive decline show different anatomical patterns in alzheimer’s disease,” NeuroImage: Clinical, vol. 22, p. 101786, 2019.
  82. N. Villain et al., “Sequential relationships between grey matter and white matter atrophy and brain metabolic abnormalities in early alzheimer’s disease,” Brain, vol. 133, no. 11, pp. 3301–3314, 2010.
  83. S. A. Mulaik, Linear causal modeling with structural equations. CRC press, 2009.
  84. G. Zhao et al., “Diagnose like a radiologist: Hybrid neuro-probabilistic reasoning for attribute-based medical image diagnosis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 7400–7416, 2021.
  85. X. Shen et al., “Challenges and opportunities with causal discovery algorithms: application to alzheimer’s pathophysiology,” Scientific reports, vol. 10, no. 1, p. 2975, 2020.
  86. Y. Iturria-Medina et al., “Multifactorial causal model of brain (dis) organization and therapeutic intervention: Application to alzheimer’s disease,” Neuroimage, vol. 152, pp. 60–77, 2017.
  87. Y. Iturria-Medina et al., “Early role of vascular dysregulation on late-onset alzheimer’s disease based on multifactorial data-driven analysis,” Nature communications, vol. 7, no. 1, p. 11934, 2016.
  88. C. R. Jack et al., “Tracking pathophysiological processes in alzheimer’s disease: an updated hypothetical model of dynamic biomarkers,” The lancet neurology, vol. 12, no. 2, pp. 207–216, 2013.
  89. M. C. Donohue et al., “Estimating long-term multivariate progression from short-term data,” Alzheimer’s & Dementia, vol. 10, pp. S400–S410, 2014.
  90. J. Jia et al., “Biomarker changes during 20 years preceding alzheimer’s disease,” New England Journal of Medicine, vol. 390, no. 8, pp. 712–722, 2024.
  91. A. Abdulaal et al., “Deep structural causal modelling of the clinical and radiological phenotype of alzheimer’s disease,” in NeurIPS 2022 Workshop on Causality for Real-world Impact, 2022.
  92. H. Hu et al., “Improved causal models of alzheimer’s disease,” in 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 274–283, 2021.
  93. R. V. Marinescu et al., “Tadpole challenge: prediction of longitudinal evolution in alzheimer’s disease,” arXiv preprint arXiv:1805.03909, 2018.
  94. R. V. Marinescu et al., “The alzheimer’s disease prediction of longitudinal evolution (tadpole) challenge: Results after 1 year follow-up,” arXiv preprint arXiv:2002.03419, 2020.
  95. Y. Utsumil et al., “Personalized gaussian processes for forecasting of alzheimer’s disease assessment scale-cognition sub-scale (adas-cog13),” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4007–4011, IEEE, 2018.
  96. C. Sevilla-Salcedo et al., “Multi-task longitudinal forecasting with missing values on alzheimer’s disease,” Computer Methods and Programs in Biomedicine, vol. 226, p. 107056, 2022.
  97. M. M. Ghazi et al., “Robust parametric modeling of alzheimer’s disease progression,” NeuroImage, vol. 225, p. 117460, 2021.
  98. M. Nguyen et al., “Predicting alzheimer’s disease progression using deep recurrent neural networks,” NeuroImage, vol. 222, p. 117203, 2020.
  99. S. Hochreiter et al., “Long short-term memory,” Neural Comput, vol. 9, no. 8, pp. 1735–1780, 1997.
  100. C. Durkan et al., “Neural spline flows,” Proc. Adv. Neural Inf. Process. Syst., vol. 32, 2019.
  101. P. L. Spirtes et al., “Causal inference in the presence of latent variables and selection bias,” arXiv preprint arXiv:1302.4983, 2013.
  102. D. M. Chickering, “Optimal structure identification with greedy search,” Journal of machine learning research, vol. 3, no. Nov, pp. 507–554, 2002.
  103. S. Shimizu et al., “Directlingam: A direct method for learning a linear non-gaussian structural equation model,” Journal of Machine Learning Research, vol. 12, no. Apr, pp. 1225–1248, 2011.
  104. J. E. Iglesias et al., “Robust brain extraction across datasets and comparison with publicly available methods,” IEEE Trans. Med. Imag, vol. 30, no. 9, pp. 1617–1634, 2011.
  105. B. B. Avants et al., “Advanced normalization tools (ants),” Insight j, vol. 2, no. 365, pp. 1–35, 2009.
  106. M. Shao et al., “Brain ventricle parcellation using a deep neural network: Application to patients with ventriculomegaly,” NeuroImage: Clinical, vol. 23, p. 101871, 2019.
  107. W. H. Pinaya et al., “Brain imaging generation with latent diffusion models,” in MICCAI Workshop on Deep Generative Models, pp. 117–126, Springer, 2022.
  108. W. H. Pinaya et al., “Generative ai for medical imaging: extending the monai framework,” arXiv preprint arXiv:2307.15208, 2023.
  109. C. Sudlow et al., “Uk biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age,” PLoS medicine, vol. 12, no. 3, p. e1001779, 2015.
  110. A. B. L. Larsen et al., “Autoencoding beyond pixels using a learned similarity metric,” in ICML, pp. 1558–1566, PMLR, 2016.
  111. I. Gulrajani et al., “Improved training of wasserstein gans,” Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017.
  112. L. Sun et al., “Hierarchical amortized gan for 3d high resolution medical image synthesis,” IEEE journal of biomedical and health informatics, vol. 26, no. 8, pp. 3966–3975, 2022.
  113. P. Isola et al., “Image-to-image translation with conditional adversarial networks,” in IEEE/CVF CVPR, pp. 1125–1134, 2017.
  114. J.-Y. Zhu et al., “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in IEEE/CVF CVPR, pp. 2223–2232, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up