Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fast and Interpretable 2D Homography Decomposition: Similarity-Kernel-Similarity and Affine-Core-Affine Transformations (2402.18008v1)

Published 28 Feb 2024 in cs.CV

Abstract: In this paper, we present two fast and interpretable decomposition methods for 2D homography, which are named Similarity-Kernel-Similarity (SKS) and Affine-Core-Affine (ACA) transformations respectively. Under the minimal $4$-point configuration, the first and the last similarity transformations in SKS are computed by two anchor points on target and source planes, respectively. Then, the other two point correspondences can be exploited to compute the middle kernel transformation with only four parameters. Furthermore, ACA uses three anchor points to compute the first and the last affine transformations, followed by computation of the middle core transformation utilizing the other one point correspondence. ACA can compute a homography up to a scale with only $85$ floating-point operations (FLOPs), without even any division operations. Therefore, as a plug-in module, ACA facilitates the traditional feature-based Random Sample Consensus (RANSAC) pipeline, as well as deep homography pipelines estimating $4$-point offsets. In addition to the advantages of geometric parameterization and computational efficiency, SKS and ACA can express each element of homography by a polynomial of input coordinates ($7$th degree to $9$th degree), extend the existing essential Similarity-Affine-Projective (SAP) decomposition and calculate 2D affine transformations in a unified way. Source codes are released in https://github.com/cscvlab/SKS-Homography.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (93)
  1. https://github.com/Tyrant1337/Lu-Decomposition-in-Parallel.
  2. https://github.com/xintaoding/CSAC.
  3. https://en.wikipedia.org/wiki/FLOPS.
  4. OpenCV’s function: getPerspectiveTransform. https://docs.opencv.org/4.5.1/da/d54/group__imgproc__transform.html.
  5. Intel 64 and IA-32 architectures optimization reference manual. https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html.
  6. http://www.vision.caltech.edu/bouguetj/calib_doc/.
  7. Cuda c++ programming guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html.
  8. Cuda toolkit. https://developer.nvidia.com/cuda-toolkit.
  9. https://docs.microsoft.com/en-us/cpp/build/reference/o-options-optimize-code?view=msvc-160.
  10. Parameterizing homographies. Robotics Institute, Pittsburgh, PA, Technical Report CMU-RI-TR-06-11, 2006.
  11. S. Baker and I. Matthews. Lucas-kanade 20 years on: A unifying framework. International journal of computer vision (IJCV), 56(3):221–255, 2004.
  12. A. Bandera and J. M. Pérez. Mean shift based clustering of hough domain for fast line segment detection. Pattern Recognition Letters, 27(6):578–586, 2006.
  13. MAGSAC: marginalizing sample consensus. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  14. MAGSAC++, a fast, reliable and accurate robust estimator. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  15. Randomized hough transform for ellipse detection with result clustering. In International Conference on Computer As A Tool, pages 1397–1400, 2006.
  16. Surf: Speeded up robust features. Computer Vision and Image Understanding (CVIU), 110(3):404–417, 2006.
  17. A fast and robust homography scheme for real-time planar target detection. Journal of Real-Time Image Processing, 15(4):739–758, 2018.
  18. Dsac-differentiable ransac for camera localization. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 3, 2017.
  19. The euroc micro aerial vehicle datasets. 2016.
  20. Camera calibration with enclosing ellipses by an extended application of generalized eigenvalue decomposition. Machine Vision and Applications (MVA), 24(3):513–520, 2013.
  21. Iterative deep homography estimation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  22. J. J. G. R. J. M. M. M. Carlos Campos, Richard Elvira and J. D. Tardós. Orb-slam3: An accurate open-source library for visual, visual-inertial and multi-map slam. IEEE Transactions on Robotics, 37(6):1874–1890, 2021.
  23. Clkn: Cascaded lucas-kanade networks for image alignment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  24. Camera calibration with two arbitrary coplanar circles. In European Conference on Computer Vision (ECCV), 2004.
  25. O. Chum and J. Matas. Matching with prosac - progressive sample consensus. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005.
  26. O. Chum and J. Matas. Matching with prosac-progressive sample consensus. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pages 220–226. IEEE, 2005.
  27. Locally optimized ransac. In Joint Pattern Recognition Symposium, 2003.
  28. T. Collins and A. Bartoli. Infinitesimal plane-based pose estimation. International Journal of Computer Vision (IJCV), 109(3):252–286, 2014.
  29. Deep image homography estimation. arXiv preprint: 1606.03798, 2016.
  30. Superpoint: Self-supervised interest point detection and description. In Conference on Computer Vision and Pattern Recognition Workshop (CVPRW) on Deep Learning for Visual SLAM, 2018.
  31. E. Elhamifar and R. Vidal. Sparse subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2790–2797, 2009.
  32. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47–53, 2022.
  33. M. Fischler and R. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
  34. Matrix computations, 4th ed. The Johns Hopkins University Press, 2013.
  35. Lsd: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 32(4):722–732, 2010.
  36. A versatile homography computation method based on two real points. Image and Vision Computing (IVC), 64(C):23–33, 2017.
  37. Euclidean structure from confocal conics: Theory and application to camera calibration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006.
  38. Computation of homographies. In British Computer Vision Conference (BMVC).
  39. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, pages 147–151, 1988.
  40. R. Hartley and A. Zisserman. Multiple view geometry in computer vision, 2nd ed. Cambridge University Press, 2003.
  41. Unsupervised homography estimation with coplanarity-aware gan. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  42. Computer Graphics: Principles and Practice, Third Edition. The systems programming series. Addison-Wesley, 2014.
  43. J. Illingworth and J. Kittler. A survey of the hough transform. Computer Vision, Graphics, and Image Processing, 44(1):87–116, 1988.
  44. G. Jiang and L. Quan. Detection of concentric circles for camera calibration. In International Conference on Computer Vision (ICCV), 2005.
  45. Image matching across wide baselines: From paper to practice. International Journal of Computer Vision (IJCV), 129:517–547, 2021.
  46. Linear Algebra and Its Applications, Fifth Edition. Pearson Publishers Inc., 2014.
  47. Deep homography estimation for dynamic scenes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  48. A robust o(n) solution to the perspective-n-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 34(7):1444–1450, 2012.
  49. Y. Li and N. R. Gans. Predictive ransac: Effective model fitting and tracking approach under heavy noise and outliers. Computer Vision and Image Understanding, 161:99–113, 2017.
  50. D. Liebowitz and A. Zisserman. Metric rectification for perspective images of planes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  51. Microsoft coco: Common objects in context. In European Conference on Computer Vision (ECCV), 2014.
  52. Deep hough-transform line priors. In European Conference on Computer Vision (ECCV), 2020.
  53. Inverting ransac: Global model detection via inlier rate estimation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5243–5251, 2015.
  54. Content-aware unsupervised deep homography estimation and its extensions. In European Conference on Computer Vision (ECCV), 2020.
  55. C. G. Looney. A new approach to fuzzy clustering. In Computers and Their Applications, pages 268–273, 2000.
  56. D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV), 60(2):91–110, 2004.
  57. L. Magri and A. Fusiello. Multiple structure recovery with t-linkage. Journal of Visual Communication and Image Representation, 49:57–77, 2017.
  58. J. Matas and O. Chum. Randomized ransac with sequential probability ratio test. In International Conference on Computer Vision (ICCV), volume 2, pages 1727–1732, 2005.
  59. F. M. Mcmahon. The livermore fortran kernels: A computer test of numerical performance range. Technical Report, Lawrence Livermore National Laboratory, 15.
  60. J. More. Numerical Analysis, Lecture Notes in Mathematics 630. Springer-Verlag, 1978.
  61. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4):102:1–102:15, July 2022.
  62. Betasac: A new conditional sampling for ransac. In British Machine Vision Conference (BMVC), pages 1–11, 2010.
  63. Unsupervised deep homography: A fast and robust homography estimation model. IEEE Robotics and Automation Letters (RAL), 3(3):2346–2353, 2018.
  64. Groupsac: Efficient consensus in the presence of groupings. In International Conference on Computer Vision (ICCV), pages 2193–2200, 2009.
  65. Unsupervised deep image stitching: Reconstructing stitched features to images. IEEE Transactions on Image Processing (TIP), 30:6184–6197, 2021.
  66. Depth-aware multi-grid deep homography estimation with contextual correlation. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 32(7):4460–4472, 2022.
  67. Homography estimation from image pairs with hierarchical convolutional networks. In International Conference on Computer Vision Workshops (ICCVW), 2017.
  68. Computer Organization and Design, Revised Fourth Edition. Morgan Kaufmann Publishers Inc., 2011.
  69. Sold2: Self-supervised occlusion-aware line description and detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  70. Usac: A universal framework for random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 35(8):2022–2038, 2013.
  71. P. Rangarajan and P. Papamichalis. Estimating homographies without normalization. In International Conference on Image Processing (ICIP), pages 3517–3520, 2009.
  72. Orb: an efficient alternative to sift or surf. In International Conference on Computer Vision (ICCV), 2011.
  73. Algebraic projective geometry. Clarendon Press, 1952.
  74. Zubud ± zurich buildings database for image based recognition. 2003.
  75. Localtrans: A multiscale local transformer network for cross-resolution homography estimation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
  76. N. Simond and P. Rives. Homography from a vanishing point in urban scenes. In International Conference on Intelligent Robots and Systems, volume 1, pages 1005–1010, 2003.
  77. V. Strassen. Gaussian elimination is not optimal. Numerische Mathematik, 13:354–356, 1969.
  78. A benchmark for the evaluation of rgb-d slam systems. In Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.
  79. P. Sturm and S. Maybank. On plane-based camera calibration: A general algorithm, singularities, applications. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 432–437, 1999.
  80. Sosnet: Second order similarity regularization for local descriptor learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  81. R. Toldo and A. Fusiello. Robust multiple structures estimation with j-linkage. In European Conference on Computer Vision (ECCV), pages 537–547, 2008.
  82. Robust detection of degenerate configurations while estimating the fundamental matrix. Computer Vision and Image Understanding, 71(3):312–333, 1998.
  83. R. Vidal. Subspace clustering. Signal Processing Magazine, 28(2):52–68, 2011.
  84. Sting: A statistical information grid approach to spatial data mining. In International Conference on Very Large Data Bases, pages 186–195, 1997.
  85. Z. Wei. Nonparametric estimation of multiple structures with outliers. In International Conference on Dynamical Vision, pages 60–74, 2006.
  86. Coplanar circles, quasi-affine invariance and calibration. Image and Vision Computing (IVC), 24(4):319–326, 2006.
  87. Motion basis learning for unsupervised deep homography estimation with subspace projection. In International Conference on Computer Vision (ICCV), 2021.
  88. Lift: Learned invariant feature transform. In European Conference on Computer Vision (ECCV), 2016.
  89. A general grid-clustering approach. Pattern Recognition Letters, 29(9):1372–1384, 2008.
  90. As-projective-as-possible image stitching with moving dlt. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.
  91. Rethinking planar homography estimation using perspective fields. In Asian Conference on Computer Vision (ACCV), 2018.
  92. Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 22(11):1330–1334, 2000.
  93. Deep lucas-kanade homography for multimodal image alignment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Summary

  • The paper introduces SKS and ACA decompositions that achieve homography estimation with only 169 and 85 FLOPs respectively, enhancing speed and interpretability.
  • It employs sequential similarity, kernel, and affine transformations to efficiently separate geometric distortions in 2D image planes.
  • The approach extends traditional SAP methods, offering a unified framework for real-time camera calibration, image stitching, and motion estimation.

Fast and Interpretable Homography Decomposition for 2D Transformations

Overview of SKS and ACA Transformations

In the domain of geometric vision, accurately and efficiently estimating 2D homography between image planes is crucial for a variety of applications such as camera calibration, image stitching, and monocular motion estimation. This paper introduces two novel methods, termed as Similarity-Kernel-Similarity (SKS) and Affine-Core-Affine (ACA), for the fast and interpretable decomposition of 2D homography. These methods propose a computationally efficient way to decompose a homography matrix into interpretable components corresponding to geometric transformations.

Similarity-Kernel-Similarity (SKS) Decomposition

The SKS decomposition method stratifies a homography into three sequential transformations: the first and last being similarity transformations (computed using two anchor points on each image plane) and the middle being a kernel transformation that represents projective distortion with only four parameters. This novel decomposition achieves higher computational efficiency by reducing the parameters involved and avoiding complex operations such as singular value decomposition (SVD). The approach not only facilitates a rapid computation with only 169 FLOPs but also maintains the robustness essential for practical applications.

Affine-Core-Affine (ACA) Decomposition

Building on SKS, the ACA decomposition further refines the way a homography is computed by employing three anchor points to calculate the initial and final affine transformations, and a core transformation that captures projective distortion between the two intermediate planes. Remarkably, ACA can compute a homography up to a scale with mere 85 FLOPs while dispensing with any division operations, which underscores its computational efficiency and its suitability for real-time applications.

Extension to SAP Decomposition and Affine Transformations

The proposed SKS and ACA decompositions extend and enrich the existing framework of homography decomposition, notably the Similarity-Affine-Projective (SAP) decomposition. Moreover, both methods provide a unified approach to calculate 2D affine transformations, effectively showcasing their versatility and general applicability across different scenarios in computer vision.

Practical Implications and Future Directions

The numerical results and practical implementation demonstrate that SKS and ACA can substantially accelerate the computation of homography in both traditional feature-based RANSAC pipelines and deep learning-based approaches without compromising accuracy. Moreover, the interpretability of the decompositions lends insights into the geometric transformations inherent in image matching tasks. Moving forward, exploring more applications of these methods in other vision tasks and further optimizing their computational efficiency hold promise for advancing real-time image analysis and understanding systems.

In conclusion, the SKS and ACA decompositions present a significant step forward in the computational efficiency and interpretability of 2D homography estimation. With their remarkable computation speed and the capacity to express each element of homography by polynomials of input coordinates, they offer a highly practical solution for a wide range of applications in geometric vision.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets