Image reconstructions using sparse dictionary representations and implicit, non-negative mappings (2312.03180v1)
Abstract: Many imaging science tasks can be modeled as a discrete linear inverse problem. Solving linear inverse problems is often challenging, with ill-conditioned operators and potentially non-unique solutions. Embedding prior knowledge, such as smoothness, into the solution can overcome these challenges. In this work, we encode prior knowledge using a non-negative patch dictionary, which effectively learns a basis from a training set of natural images. In this dictionary basis, we desire solutions that are non-negative and sparse (i.e., contain many zero entries). With these constraints, standard methods for solving discrete linear inverse problems are not directly applicable. One such approach is the modified residual norm steepest descent (MRNSD), which produces non-negative solutions but does not induce sparsity. In this paper, we provide two methods based on MRNSD that promote sparsity. In our first method, we add an $\ell_1$-regularization term with a new, optimal step size. In our second method, we propose a new non-negative, sparsity-promoting mapping of the solution. We compare the performance of our proposed methods on a number of numerical experiments, including deblurring, image completion, computer tomography, and superresolution. Our results show that these methods effectively solve discrete linear inverse problems with non-negativity and sparsity constraints.
- Uri M Ascher and Chen Greif “A first course on numerical methods” SIAM, 2011
- “Image Inpainting” In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’00 USA: ACM Press/Addison-Wesley Publishing Co., 2000, pp. 417–424 DOI: 10.1145/344779.344972
- “Distributed optimization and statistical learning via the alternating direction method of multipliers” In Foundations and Trends® in Machine learning 3.1 Now Publishers, Inc., 2011, pp. 1–122
- Steven L Brunton and J Nathan Kutz “Data-driven science and engineering: Machine learning, dynamical systems, and control” Cambridge University Press, 2019
- “Matrix Completion With Noise” In Proceedings of the IEEE 98, 2010, pp. 925–936 DOI: 10.1109/JPROC.2009.2035722
- Emmanuel J Candes, Justin K Romberg and Terence Tao “Stable signal recovery from incomplete and inaccurate measurements” In Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 59.8 Wiley Online Library, 2006, pp. 1207–1223
- Emmanuel J. Candès and Benjamin Recht “Exact Matrix Completion via Convex Optimization” In Foundations of Computational Mathematics 9.6, 2009, pp. 717–772 DOI: 10.1007/s10208-009-9045-5
- Tony F Chan and Jianhong Shen “Image processing and analysis: variational, PDE, wavelet, and stochastic methods” SIAM, 2005
- Scott Shaobing Chen, David L Donoho and Michael A Saunders “Atomic decomposition by basis pursuit” In SIAM review 43.1 SIAM, 2001, pp. 129–159
- “Computational methods for large-scale inverse problems: a survey on hybrid projection methods” In Siam Review Society for IndustrialApplied Mathematics Publications, 2023
- “The Variable Projected Augmented Lagrangian Method”, 2022 arXiv:2207.08216 [math.NA]
- “VANET based traffic estimation: A matrix completion approach” In 2013 IEEE Global Communications Conference (GLOBECOM), 2013, pp. 30–35 DOI: 10.1109/GLOCOM.2013.6831043
- Michael Elad “Sparse and redundant representations: from theory to applications in signal and image processing” Springer, 2010
- Silvia Gazzola, Per Christian Hansen and James G. Nagy “IR Tools: a MATLAB package of iterative regularization methods and large-scale test problems” In Numerical Algorithms 81.3, 2019, pp. 773–811 DOI: 10.1007/s11075-018-0570-7
- Gene H. Golub, Michael Heath and Grace Wahba “Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter” In Technometrics 21.2 Taylor & Francis, 1979, pp. 215–223 DOI: 10.1080/00401706.1979.10489751
- Per Christian Hansen “Rank-Deficient and Discrete Ill-Posed Problems” Society for IndustrialApplied Mathematics, 1998 DOI: 10.1137/1.9780898719697
- Per Christian Hansen “Discrete inverse problems: insight and algorithms” SIAM, 2010
- Per Christian Hansen and Jakob Sauer Jørgensen “AIR Tools II: algebraic iterative reconstruction methods, improved implementation” In Numerical Algorithms 79.1, 2018, pp. 107–137 DOI: 10.1007/s11075-017-0430-x
- Per Christian Hansen, James G. Nagy and Dianne P. O’Leary “Deblurring Images” Society for IndustrialApplied Mathematics, 2006 DOI: 10.1137/1.9780898718874
- Avinash C Kak, Malcolm Slaney and Ge Wang “Principles of Computerized Tomographic Imaging” Wiley Online Library, 2002
- Linda Kaufman “Maximum likelihood, least squares, and penalized least square for PET” In Medical Imaging, IEEE Transactions on 12, 1993, pp. 200–214 DOI: 10.1109/42.232249
- Dara Kerr “Stunning high-resolution photo shows Earth’s many hues”, 2012 URL: https://www.cnet.com/science/stunning-high-resolution-photo-shows-earths-many-hues/
- Daniel D. Lee and H.Sebastian Seung “Learning the parts of objects by non-negative matrix factorization” In Nature 401.6755, 1999, pp. 788–791 DOI: 10.1038/44565
- “A Survey on Matrix Completion: Perspective of Signal Processing”, 2019 arXiv:1901.10885 [eess.SP]
- Stéphane G Mallat and Zhifeng Zhang “Matching pursuits with time-frequency dictionaries” In IEEE Transactions on signal processing 41.12 IEEE, 1993, pp. 3397–3415
- James G. Nagy and Zdenek Strakos “Enforcing nonnegativity in image reconstruction algorithms” In Mathematical Modeling, Estimation, and Imaging 4121 SPIE, 2000, pp. 182–190 International Society for OpticsPhotonics DOI: 10.1117/12.402439
- Frank Natterer “The Mathematics of Computerized Tomography” SIAM, 2001
- Elizabeth Newman and Misha E. Kilmer “Nonnegative Tensor Patch Dictionary Approaches for Image Compression and Deblurring Applications” In SIAM Journal on Imaging Sciences 13.3, 2020, pp. 1084–1112 DOI: 10.1137/19M1297026
- Jorge Nocedal and Stephen J Wright “Numerical optimization” Springer, 1999
- Stanley Osher, Andrés Solé and Luminita Vese “Image Decomposition and Restoration Using Total Variation Minimization and the H1” In Multiscale Modeling & Simulation 1.3, 2003, pp. 349–370 DOI: 10.1137/S1540345902416247
- “Proximal Algorithms” In Found. Trends Optim. 1.3 Hanover, MA, USA: Now Publishers Inc., 2014, pp. 127–239 DOI: 10.1561/2400000003
- “The Matrix Cookbook” Version 20121115 Technical University of Denmark, 2012 URL: http://www2.compute.dtu.dk/pubdb/pubs/3274-full.html
- Lars Ruthotto, Julianne Chung and Matthias Chung “Optimal experimental design for inverse problems with state constraints” In SIAM Journal on Scientific Computing 40.4 SIAM, 2018, pp. B1080–B1100
- Yousef Saad “Iterative methods for sparse linear systems” SIAM, 2003
- “Sampled Tikhonov regularization for large linear inverse problems” In Inverse Problems 35.11 IOP Publishing, 2019, pp. 114008
- Sara Soltani, Martin S. Andersen and Per Christian Hansen “Tomographic Image Reconstruction using Dictionary Priors” In CoRR abs/1503.01993, 2015 arXiv: http://arxiv.org/abs/1503.01993
- Sara Soltani, Misha E. Kilmer and Per Christian Hansen “A tensor-based dictionary learning approach to tomographic image reconstruction” In BIT Numerical Mathematics 56.4, 2016, pp. 1425–1454 DOI: 10.1007/s10543-016-0607-z
- “Edge-preserving and scale-dependent properties of total variation regularization” In Inverse problems 19.6 IOP Publishing, 2003, pp. S165
- The TensorFlow Team “Flowers”, 2019 URL: http://download.tensorflow.org/example_images/flower_photos.tgz
- “Dictionary learning” In IEEE Signal Processing Magazine 28.2 IEEE, 2011, pp. 27–38
- Joel A Tropp “Greed is good: Algorithmic results for sparse approximation” In IEEE Transactions on Information theory 50.10 IEEE, 2004, pp. 2231–2242
- “Image super-resolution via sparse representation” In IEEE transactions on image processing 19.11 IEEE, 2010, pp. 2861–2873