Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Quantum-Hybrid Stereo Matching With Nonlinear Regularization and Spatial Pyramids (2312.16118v2)

Published 26 Dec 2023 in cs.CV

Abstract: Quantum visual computing is advancing rapidly. This paper presents a new formulation for stereo matching with nonlinear regularizers and spatial pyramids on quantum annealers as a maximum a posteriori inference problem that minimizes the energy of a Markov Random Field. Our approach is hybrid (i.e., quantum-classical) and is compatible with modern D-Wave quantum annealers, i.e., it includes a quadratic unconstrained binary optimization (QUBO) objective. Previous quantum annealing techniques for stereo matching are limited to using linear regularizers, and thus, they do not exploit the fundamental advantages of the quantum computing paradigm in solving combinatorial optimization problems. In contrast, our method utilizes the full potential of quantum annealing for stereo matching, as nonlinear regularizers create optimization problems which are NP-hard. On the Middlebury benchmark, we achieve an improved root mean squared accuracy over the previous state of the art in quantum stereo matching of 2% and 22.5% when using different solvers.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Demonstration of a scaling advantage for a quantum annealer over simulated annealing. Phys. Rev. X, 8:031016, 2018.
  2. Quantum motion segmentation. In European Conference on Computer Vision (ECCV), 2022.
  3. Stephen T Barnard. Stochastic stereo matching over scale. International Journal of Computer Vision, 3(1):17–32, 1989.
  4. Quantum permutation synchronization, 2021.
  5. Visual reconstruction. MIT press, 1987.
  6. Next-generation topology of d-wave quantum processors. arXiv e-prints, 2020.
  7. Beweis des adiabatensatzes. Zeitschrift für Physik, 51(3):165–180, 1928.
  8. The future of quantum computing with superconducting qubits. Journal of Applied Physics, 132(16), 2022.
  9. A naturalistic open source movie for optical flow evaluation. In European Conf. on Computer Vision (ECCV), pages 611–625. Springer-Verlag, 2012.
  10. A practical heuristic for finding graph minors. arXiv, 2014.
  11. The theory and practice of bayesian image labeling. International journal of computer vision, 4:185–210, 1990.
  12. A qubo formulation of the stereo matching problem for d-wave quantum annealers. Entropy, 20, 2018.
  13. D-Wave. Zephyr topology of d-wave quantum processors (d-wave technical report series). https://www.dwavesys.com/media/2uznec4s/14-1056a-a_zephyr_topology_of_d-wave_quantum_processors.pdf, 2021.
  14. D-Wave Systems, Inc. Simulated annealing sampler. https://github.com/dwavesystems/dwave-neal/blob/master/docs/reference/sampler.rst, 2023. online; accessed on the 07.08.2023.
  15. A hybrid quantum-classical algorithm for robust fitting. In Computer Vision and Pattern Recognition (CVPR), pages 417–427, 2022.
  16. Semi-global matching: A principled derivation in terms of message passing. In German Conference on Pattern Recognition, 2014.
  17. Quantum computation by adiabatic evolution. arXiv: Quantum Physics, 2000.
  18. A quantum adiabatic evolution algorithm applied to random instances of an np-complete problem. Science, 292(5516):472–475, 2001.
  19. Sebastian Finsterwalder. Die geometrischen grundlagen der photogrammetrie. Jahresbericht der Deutschen Mathematiker-Vereinigung, 6, 1897.
  20. Flows in Networks. Princeton University Press, 2010.
  21. Borko Furht. Block Matching, pages 55–56. Springer US, Boston, MA, 2008.
  22. Michael A. Gennert. Brightness-based stereo matching. International Conference on Computer Vision (ICCV), pages 139–143, 1988a.
  23. Michael A Gennert. Brightness-based stereo matching. In 1988 Second International Conference on Computer Vision, pages 139–140. IEEE Computer Society, 1988b.
  24. Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023.
  25. Guido Hauck. Neue constructionen der perspective und photogrammetrie. (theorie der trilinearen verwandtschaft ebener systeme, i. artikel.). Journal für die reine und angewandte Mathematik (Crelles Journal), 1883.
  26. An improved quantum solution for the stereo matching problem. In International Conference on Image and Vision Computing New Zealand (IVCNZ), 2021.
  27. Heiko Hirschmuller. Accurate and efficient stereo processing by semi-global matching and mutual information. In Computer Vision and Pattern Recognition (CVPR), 2005.
  28. Hiroshi Ishikawa. Transformation of general binary mrf minimization to the first-order case. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(6):1234–1249, 2011.
  29. Optimization by simulated annealing. Science, 220(4598):671–680, 1983.
  30. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In ICPR, 2006.
  31. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.
  32. Computing visual correspondence with occlusions using graph cuts. In International Conference on Computer Vision (ICCV), 2001.
  33. Solving the max-flow problem on a quantum annealing computer. IEEE Transactions on Quantum Engineering, 1:1–10, 2020.
  34. A survey on deep learning techniques for stereo-based depth estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):1738–1764, 2020.
  35. Trends of quantum computing applications to computer vision. In 2022 International Conference on Platform Technology and Service (PlatCon), pages 7–12, 2022.
  36. Hugh Christopher Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135, 1981.
  37. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Association, 82, 1987.
  38. Nicolaus Henke Niko Mohr Lorenzo Pautasso Ivan Ostojic Linde Wester Matteo Biondi Anna Heid and Rodney Zemmel. Quantum computing: An emerging ecosystem and industry use cases (report), 2021.
  39. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Computer Vision and Pattern Recognition (CVPR), pages 4040–4048, 2016.
  40. An iterative quantum approach for transformation estimation from point sets. In Computer Vision and Pattern Recognition (CVPR), pages 529–537, 2022.
  41. Mikhail G. Mozerov and Joost van de Weijer. Accurate stereo matching by two-step energy minimization. IEEE Transactions on Image Processing, 24(3):1153–1163, 2015.
  42. Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge University Press, 2011.
  43. Matching point sets with quantum circuit learning. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8607–8611, 2022.
  44. Quantum annealing approach: Feature extraction and segmentation of synthetic aperture radar image. In IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pages 3692–3695, 2020.
  45. Synthetic aperture radar image segmentation with quantum annealing. arXiv e-prints, 2023.
  46. A maximum-flow formulation of the n-camera stereo correspondence problem. In International Conference on Computer Vision (ICCV), pages 492–499, 1998.
  47. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision, 47:7–42, 2002.
  48. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV), pages 131–140, 2001.
  49. Adiabatic quantum graph matching with permutation matrix constraints. In International Conference on 3D Vision (3DV), 2020.
  50. Q-match: Iterative shape matching via quantum annealing. In International Conference on Computer Vision (ICCV), 2021.
  51. Adastereo: A simple and efficient approach for adaptive stereo matching. In Computer Vision and Pattern Recognition (CVPR), pages 10328–10337, 2021.
  52. Stereo matching using hierarchical belief propagation along ambiguity gradient. In International Conference on Image Processing (ICIP), 2009.
  53. Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 25(7):787–800, 2003.
  54. DWave Systems. Solver properties. https://docs.dwavesys.com/docs/latest/c_solver_properties.html, 2023. Accessed: 2023-08-22.
  55. Continuous 3d label stereo matching using local expansion moves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:2725–2739, 2018.
  56. Bilateral filtering for gray and color images. In International Conference on Computer Vision (ICCV), pages 839–846, 1998.
  57. Stereo matching by joint energy minimization, 2016.
  58. Adversarial patch attacks on monocular depth estimation networks. IEEE Access, 8:179094–179104, 2020.
  59. Analytical solution for nonadiabatic quantum annealing to arbitrary ising spin hamiltonian. Nature Communications, 13(1), 2022.
  60. Adiabatic quantum computing for multi object tracking. In Computer Vision and Pattern Recognition (CVPR), pages 8811–8822, 2022.

Summary

  • The paper introduces a novel quantum-hybrid stereo matching approach that applies MRF energy minimization with nonlinear regularizers.
  • It integrates a coarse-to-fine spatial pyramid strategy to ensure robust modeling and tractable computations.
  • Empirical results demonstrate improved root mean squared accuracy over previous methods, underscoring its potential for advanced computer vision tasks.

Background on Stereo Matching

Stereo matching is a well-established problem in computer vision that aims to find the correspondence between pixels in a pair of stereo images. Due to its lengthy history, numerous algorithms have been developed, with recent advancements focusing on leveraging quantum hardware. The advantage of quantum computers lies in their potential to tackle NP-hard problems, which are challenging for classical computers.

Quantum-Hybrid Approach

The work at hand presents a quantum-hybrid method for stereo matching that uses Markov Random Field (MRF) energy minimization. Unlike previous techniques limited to linear regularizers, this new approach supports nonlinear regularizers, thus fully tapping into the combinatorial optimization potential of quantum annealing. The method is compatible with D-Wave quantum annealers and represents energies as Quadratic Unconstrained Binary Optimization (QUBO) problems.

Nonlinear Regularization and Spatial Pyramids

Nonlinear regularization is critical for complex optimization problems in stereo matching, as it can address challenges linear models cannot. A general MRF formulation is posited, allowing for robust modeling of these regularizers within a quantum annealing context. In addition, a coarse-to-fine pyramid approach helps provide additional robustness and makes computations manageable with current solvers.

Results and Contributions

Empirically, the proposed method showed an improvement in root mean squared accuracy over prior quantum stereo matching techniques when using classical solvers like Gurobi. While quantum annealers did not yet yield improved solutions due to current hardware limitations, this research represents a significant step forward in quantum computer vision. The key contributions include the novel quantum-hybrid stereo matching approach with nonlinear regularization, the novel modeling strategy for stereo matching on quantum annealers, and the integration of a coarse-to-fine pyramid to add extra regularization and tractability to the problem-solving process. The approach's flexibility suggests potential application in other core computer vision tasks.

Related Work

In the sphere of traditional stereo matching and quantum computer vision, the approach proposed in this paper stands out due to its innovative utilization of quantum annealers to tackle NP-hard problems. It extends the current boundaries by demonstrating how complex nonlinear regularizers can be encoded for quantum processing. This research builds on the ideas of previous works, providing a framework that could potentially revolutionize optimization in stereo matching and other computer vision challenges.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets