Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matching (2405.18816v3)

Published 29 May 2024 in cs.CV and cs.LG

Abstract: Generative models based on flow matching have attracted significant attention for their simplicity and superior performance in high-resolution image synthesis. By leveraging the instantaneous change-of-variables formula, one can directly compute image likelihoods from a learned flow, making them enticing candidates as priors for downstream tasks such as inverse problems. In particular, a natural approach would be to incorporate such image probabilities in a maximum-a-posteriori (MAP) estimation problem. A major obstacle, however, lies in the slow computation of the log-likelihood, as it requires backpropagating through an ODE solver, which can be prohibitively slow for high-dimensional problems. In this work, we propose an iterative algorithm to approximate the MAP estimator efficiently to solve a variety of linear inverse problems. Our algorithm is mathematically justified by the observation that the MAP objective can be approximated by a sum of $N$ ``local MAP'' objectives, where $N$ is the number of function evaluations. By leveraging Tweedie's formula, we show that we can perform gradient steps to sequentially optimize these objectives. We validate our approach for various linear inverse problems, such as super-resolution, deblurring, inpainting, and compressed sensing, and demonstrate that we can outperform other methods based on flow matching. Code is available at https://github.com/YasminZhang/ICTM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. François Roddier. Interferometric imaging in optical astronomy. Physics Reports, 170(2):97–166, 1988.
  2. Peter A Jansson. Deconvolution of images and spectra. Courier Corporation, 2014.
  3. Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1):86–109, 2019.
  4. Paul Suetens. Fundamentals of medical imaging. Cambridge university press, 2017.
  5. Guust Nolet. A breviary of seismic tomography. A breviary of seismic tomography, 2008.
  6. Seismic tomography and the assessment of uncertainty. Advances in geophysics, 55:1–76, 2014.
  7. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  8. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  9. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  10. Variational inference with normalizing flows. In International conference on machine learning, pages 1530–1538. PMLR, 2015.
  11. Skews in the phenomenon space hinder generalization in text-to-image generation, 2024.
  12. Object-conditioned energy-based attention map alignment in text-to-image diffusion models. arXiv preprint arXiv:2404.07389, 2024.
  13. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1):39–56, 2020. doi: 10.1109/JSAIT.2020.2991563.
  14. Compressed sensing using generative models. International Conference on Machine Learning, 2017.
  15. Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  16. Invertible generative models for inverse problems: mitigating representation error and dataset bias. Proceedings of the 37th International Conference on Machine Learning, 2020.
  17. Composing normalizing flows for inverse problems. Proceedings of the 38th International Conference on Machine Learning, 2021.
  18. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, ICLR 2023. The International Conference on Learning Representations, 2023.
  19. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
  20. Latent diffusion energy-based model for interpretable text modeling. arXiv preprint arXiv:2206.05895, 2022.
  21. Learning energy-based prior model with diffusion-amortized mcmc. Advances in Neural Information Processing Systems, 36, 2024.
  22. Unsupervised foreground extraction via deep region competition. Advances in Neural Information Processing Systems, 34:14264–14279, 2021.
  23. Learning concept-based visual causal transition and symbolic reasoning for visual planning. arXiv preprint arXiv:2310.03325, 2023.
  24. P2gnet: Pose-guided point cloud generating networks for 6-dof object pose estimation. arXiv preprint arXiv:1912.09316, 2019.
  25. Watermarking generative tabular data. arXiv preprint arXiv:2405.14018, 2024.
  26. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2022.
  27. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2022.
  28. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In International Conference on Learning Representations, 2024.
  29. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206, 2024.
  30. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. arXiv preprint arXiv:2405.07510, 2024.
  31. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
  32. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In International Conference on Learning Representations, 2019.
  33. Training-free linear image inversion via flows. arXiv preprint arXiv:2310.04432, 2023.
  34. Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper bayes estimators. Inverse Problems, 30(11):114004, 2014.
  35. Maximum a posteriori probability estimates in infinite-dimensional bayesian inverse problems. Inverse Problems, 31(8):085009, 2015.
  36. Thorsten M Buzug. Computed tomography. In Springer handbook of medical technology, pages 311–342. Springer, 2011.
  37. Magnetic resonance imaging: theory and practice. Springer Science & Business Media, 2013.
  38. Handbook of markov chain monte carlo. CRC press, 2011.
  39. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415–1428, 2021.
  40. Score-based diffusion models as principled priors for inverse imaging. In International Conference on Computer Vision (ICCV). IEEE, 2023.
  41. Efficient bayesian computational imaging with a surrogate score-based prior. arXiv preprint arXiv:2309.01949, 2023.
  42. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015.
  43. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  44. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256–2265. PMLR, 2015.
  45. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  46. John Skilling. The eigenvalues of mega-dimensional matrices. Maximum Entropy and Bayesian Methods: Cambridge, England, 1988, pages 455–466, 1989.
  47. Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Statistics-Simulation and Computation, 18(3):1059–1076, 1989.
  48. Bradley Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011.
  49. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730–3738, 2015.
  50. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
  51. The wu-minn human connectome project: an overview. Neuroimage, 80:62–79, 2013.
  52. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. doi: 10.1109/TIP.2003.819861.
  53. The perception-distortion tradeoff. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6228–6237, 2018.
  54. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  55. What’s in a prior? learned proximal networks for inverse problems. In International Conference on Learning Representations, 2024.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now