Papers
Topics
Authors
Recent
Search
2000 character limit reached

Towards Robust Image Stitching: An Adaptive Resistance Learning against Compatible Attacks

Published 25 Feb 2024 in cs.CV | (2402.15959v1)

Abstract: Image stitching seamlessly integrates images captured from varying perspectives into a single wide field-of-view image. Such integration not only broadens the captured scene but also augments holistic perception in computer vision applications. Given a pair of captured images, subtle perturbations and distortions which go unnoticed by the human visual system tend to attack the correspondence matching, impairing the performance of image stitching algorithms. In light of this challenge, this paper presents the first attempt to improve the robustness of image stitching against adversarial attacks. Specifically, we introduce a stitching-oriented attack~(SoA), tailored to amplify the alignment loss within overlapping regions, thereby targeting the feature matching procedure. To establish an attack resistant model, we delve into the robustness of stitching architecture and develop an adaptive adversarial training~(AAT) to balance attack resistance with stitching precision. In this way, we relieve the gap between the routine adversarial training and benign models, ensuring resilience without quality compromise. Comprehensive evaluation across real-world and synthetic datasets validate the deterioration of SoA on stitching performance. Furthermore, AAT emerges as a more robust solution against adversarial perturbations, delivering superior stitching results. Code is available at:https://github.com/Jzy2017/TRIS.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Surf: Speeded up robust features. In ECCV, 404–417. Springer.
  2. Shape-preserving half-projective warps for image stitching. In CVPR, 3254–3261.
  3. Natural image stitching with the global similarity prior. In ECCV, 186–201. Springer.
  4. Evaluating robustness of deep image super-resolution against adversarial attacks. In ICCV, 303–311.
  5. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition. Opt. Commun., 341: 199–209.
  6. Image quality measures and their performance. IEEE Trans. Commun., 43(12): 2959–2965.
  7. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6): 381–395.
  8. Constructing image panoramas using dual-homography warping. In CVPR, 49–56. IEEE.
  9. Advhaze: Adversarial haze attack. arXiv preprint arXiv:2104.13673.
  10. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572.
  11. Reconet: Recurrent correction network for fast and efficient multi-modality image fusion. In ECCV, 539–555. Springer.
  12. Target Oriented Perceptual Adversarial Fusion Network for Underwater Image Enhancement. IEEE Trans. Circuits Syst. Video Technol., 32(10): 6584–6598.
  13. Towards all weather and unobstructed multi-spectral image stitching: Algorithm and benchmark. In ACM MM, 3783–3791.
  14. Multi-Spectral Image Stitching via Spatial Graph Reasoning. In ACM MM, 472–480.
  15. Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifiers. In ICCV.
  16. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  17. Adversarial examples in the physical world. In Artificial intelligence safety and security, 99–112. Chapman and Hall/CRC.
  18. Warping residual based image stitching for large parallax. In CVPR, 8198–8206.
  19. Learning a coordinated network for detail-refinement multiexposure image fusion. IEEE Trans. Circuits Syst. Video Technol., 33(2): 713–727.
  20. Gesenet: A general semantic-guided network with couple mask ensemble for medical image fusion. IEEE Trans. Neural Netw. Learn. Syst.
  21. Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. multimedia, 20(7): 1672–1687.
  22. Adaptive as-natural-as-possible image stitching. In CVPR, 1155–1163.
  23. Smoothly varying affine stitching. In CVPR, 345–352. IEEE.
  24. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055.
  25. A unified image fusion framework with flexible bilevel paradigm integration. The Visual Comput., 39(10): 4869–4886.
  26. Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion. Int. J. Comput. Vis., 1–28.
  27. HoLoCo: Holistic and local contrastive learning network for multi-exposure image fusion. Inf. Fusion, 95: 237–249.
  28. Smoa: Searching a modality-oriented architecture for infrared and visible image fusion. IEEE Signal Process. Lett., 28: 1818–1822.
  29. Learn to search a lightweight architecture for target-aware infrared and visible image fusion. IEEE Signal Process. Lett., 29: 1614–1618.
  30. Twin Adversarial Contrastive Learning for Underwater Image Enhancement and Beyond. IEEE Trans. Image Process., 31: 4922–4936.
  31. A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion. IEEE Trans. Image Process., 30: 1261–1274.
  32. PAIF: Perception-aware infrared-visible image fusion for attack-tolerant semantic segmentation. In ACM MM, 3706–3714.
  33. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis., 60: 91–110.
  34. Subjective and objective quality assessment of stitched images for virtual reality. IEEE Trans. Image Process., 28(11): 5620–5635.
  35. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.
  36. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process., 21(12): 4695–4708.
  37. A view-free image stitching network based on global homography. J. Vis. Commun. Image Represent., 73: 102950.
  38. Unsupervised deep image stitching: Reconstructing stitched features to images. IEEE Trans. Image Process., 30: 6184–6197.
  39. Rao, Y.-J. 1997. In-fibre Bragg grating sensors. Meas. Sci. Technol., 8(4): 355.
  40. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 234–241. Springer.
  41. ORB: An efficient alternative to SIFT or SURF. In ICCV, 2564–2571. Ieee.
  42. Weakly-Supervised Stitching Network for Real-World Panoramic Image Generation. In ECCV, 54–71. Springer.
  43. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In CVPR, 8934–8943.
  44. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.
  45. Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610.
  46. Adversarial Examples for Semantic Segmentation and Object Detection. In ICCV.
  47. When deep fool meets deep prior: Adversarial attack on super-resolution network. In ACM MM, 1930–1938.
  48. Towards robust rain removal against adversarial attacks: A comprehensive benchmark analysis and beyond. In CVPR, 6013–6022.
  49. As-projective-as-possible image stitching with moving DLT. In CVPR, 2339–2346.
  50. Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans. Image Process., 25(7): 3099–3111.
  51. Content-aware unsupervised deep homography estimation. In ECCV, 653–669. Springer.
  52. Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. In CVPR, 5906–5916.
  53. Discrete cosine transform network for guided depth map super-resolution. In CVPR, 5697–5707.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up