Papers
Topics
Authors
Recent
Search
2000 character limit reached

Object-level Geometric Structure Preserving for Natural Image Stitching

Published 20 Feb 2024 in cs.CV | (2402.12677v4)

Abstract: The topic of stitching images with globally natural structures holds paramount significance, with two main goals: pixel-level alignment and distortion prevention. The existing approaches exhibit the ability to align well, yet fall short in maintaining object structures. In this paper, we endeavour to safeguard the overall OBJect-level structures within images based on Global Similarity Prior (OBJ-GSP), on the basis of good alignment performance. Our approach leverages semantic segmentation models like the family of Segment Anything Model to extract the contours of any objects in a scene. Triangular meshes are employed in image transformation to protect the overall shapes of objects within images. The balance between alignment and distortion prevention is achieved by allowing the object meshes to strike a balance between similarity and projective transformation. We also demonstrate that object-level semantic information is necessary in low-altitude aerial image stitching. Additionally, we propose StitchBench, the largest image stitching benchmark with most diverse scenarios. Extensive experimental results demonstrate that OBJ-GSP outperforms existing methods in both pixel alignment and shape preservation. Code and dataset is publicly available at \url{https://github.com/RussRobin/OBJ-GSP}.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Jump: virtual reality video. ACM Trans. Graph., 35:198:1–198:13, 2016.
  2. Yolact: Real-time instance segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9157–9166, 2019.
  3. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74:59–73, 2007.
  4. John F. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8:679–698, 1986.
  5. Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9630–9640, 2021.
  6. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:834–848, 2016.
  7. Natural image stitching with the global similarity prior. In European Conference on Computer Vision, 2016.
  8. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022.
  9. Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  10. An implementation of multi sensor based mobile robot with image stitching application. 2014.
  11. dinglufe. Segment anything cpp wrapper. GitHub Repository, 2023. Accessed on 2023-09-20.
  12. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abs/2010.11929, 2020.
  13. Geometric structure preserving warp for natural image stitching. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3678–3686, 2022.
  14. Constructing image panoramas using dual-homography warping. CVPR 2011, pages 49–56, 2011.
  15. Seam-driven image stitching. In Eurographics, 2013.
  16. Region-based convolutional networks for accurate object detection and segmentation. IEEE transactions on pattern analysis and machine intelligence, 38(1):142–158, 2015.
  17. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  18. Mobilenets: Efficient convolutional neural networks for mobile vision applications. ArXiv, abs/1704.04861, 2017.
  19. As-rigid-as-possible shape manipulation. ACM SIGGRAPH 2005 Papers, 2005.
  20. Leveraging line-point consistence to preserve structures for wide parallax image stitching. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12181–12190, 2021.
  21. Learning pixel-wise alignment for unsupervised image stitching. Network, 1(1):1, 2023.
  22. Eurographics symposium on rendering (2007) scene collages and flexible camera arrays.
  23. Deep occlusion-aware instance segmentation with overlapping bilayers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4018–4027, 2021.
  24. Deep virtual reality image quality assessment with human perception guider for omnidirectional image. IEEE Transactions on Circuits and Systems for Video Technology, 30:917–928, 2020.
  25. Segment anything. ArXiv, abs/2304.02643, 2023.
  26. Parallax-tolerant image stitching based on robust elastic warping. IEEE Transactions on Multimedia, 20:1672–1687, 2018.
  27. Dual-feature warping-based motion model estimation. 2015 IEEE International Conference on Computer Vision (ICCV), pages 4283–4291, 2015.
  28. Adaptive as-natural-as-possible image stitching. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1155–1163, 2015a.
  29. Adaptive as-natural-as-possible image stitching. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1155–1163, 2015b.
  30. Seagull: Seam-guided local alignment for parallax-tolerant image stitching. In European Conference on Computer Vision, 2016.
  31. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014.
  32. Smoothly varying affine stitching. CVPR 2011, pages 345–352, 2011.
  33. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20:209–212, 2013.
  34. Unsupervised deep image stitching: Reconstructing stitched features to images. IEEE Transactions on Image Processing, 30:6184–6197, 2021.
  35. Dinov2: Learning robust visual features without supervision. ArXiv, abs/2304.07193, 2023.
  36. Meta pseudo labels. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11552–11563, 2020.
  37. U-net: Convolutional networks for biomedical image segmentation. ArXiv, abs/1505.04597, 2015.
  38. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3431–3440, 2014.
  39. Revisiting unreasonable effectiveness of data in deep learning era. 2017 IEEE International Conference on Computer Vision (ICCV), pages 843–852, 2017.
  40. Lsd: a line segment detector. Image Process. Line, 2:35–55, 2012.
  41. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13:600–612, 2004.
  42. Holistically-nested edge detection. International Journal of Computer Vision, 125:3 – 18, 2015.
  43. Sequential image stitching for mobile panoramas. 2009 7th International Conference on Information, Communications and Signal Processing (ICICS), pages 1–5, 2009.
  44. U2fusion: A unified unsupervised image fusion network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44:502–518, 2020.
  45. Murf: Mutually reinforcing multi-modal image registration and fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:12148–12166, 2023.
  46. Object-contextual representations for semantic segmentation. In European Conference on Computer Vision, 2019.
  47. As-projective-as-possible image stitching with moving dlt. 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 2339–2346, 2013.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free