Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
116 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
24 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
35 tokens/sec
2000 character limit reached

SafePaint: Anti-forensic Image Inpainting with Domain Adaptation (2404.18136v2)

Published 28 Apr 2024 in cs.CV and cs.MM

Abstract: Existing image inpainting methods have achieved remarkable accomplishments in generating visually appealing results, often accompanied by a trend toward creating more intricate structural textures. However, while these models excel at creating more realistic image content, they often leave noticeable traces of tampering, posing a significant threat to security. In this work, we take the anti-forensic capabilities into consideration, firstly proposing an end-to-end training framework for anti-forensic image inpainting named SafePaint. Specifically, we innovatively formulated image inpainting as two major tasks: semantically plausible content completion and region-wise optimization. The former is similar to current inpainting methods that aim to restore the missing regions of corrupted images. The latter, through domain adaptation, endeavors to reconcile the discrepancies between the inpainted region and the unaltered area to achieve anti-forensic goals. Through comprehensive theoretical analysis, we validate the effectiveness of domain adaptation for anti-forensic performance. Furthermore, we meticulously crafted a region-wise separated attention (RWSA) module, which not only aligns with our objective of anti-forensics but also enhances the performance of the model. Extensive qualitative and quantitative evaluations show our approach achieves comparable results to existing image inpainting methods while offering anti-forensic capabilities not available in other methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. 2001. Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on image processing 10, 8 (2001), 1200–1211.
  2. 2005. Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Transactions on pattern analysis and machine intelligence 27, 4 (2005), 506–517.
  3. High-resolution daytime translation without domain labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7488–7497.
  4. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24.
  5. Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 417–424.
  6. Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions. IEEE Transactions on image processing 24, 6 (2015), 1809–1824.
  7. Object removal by exemplar-based inpainting. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., Vol. 2. IEEE, II–II.
  8. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on graphics (TOG) 31, 4 (2012), 1–10.
  9. Image inpainting using nonlocal texture matching and nonlinear filtering. IEEE Transactions on Image Processing 28, 4 (2018), 1705–1719.
  10. What makes paris look like paris? Commun. ACM 58, 12 (2015), 103–110.
  11. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
  12. Trufor: Leveraging all-round clues for trustworthy image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20606–20615.
  13. Image inpainting via conditional texture and structure dual generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14134–14143.
  14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
  15. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132–7141.
  16. Image completion using planar structure guidance. ACM Transactions on graphics (TOG) 33, 4 (2014), 1–10.
  17. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG) 36, 4 (2017), 1–14.
  18. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.
  19. Keys to better image inpainting: Structure and texture go hand in hand. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 208–217.
  20. Memory-guided unsupervised image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6558–6567.
  21. Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer, 694–711.
  22. Haodong Li and Jiwu Huang. 2019. Localization of deep inpainting using high-pass fully convolutional network. In Proceedings of the IEEE/CVF international conference on computer vision. 8301–8310.
  23. Localization of diffusion-based inpainting in digital images. IEEE Transactions on information forensics and security 12, 12 (2017), 3050–3064.
  24. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV). 85–100.
  25. Coherent semantic attention for image inpainting. In Proceedings of the IEEE/CVF international conference on computer vision. 4170–4179.
  26. Pscc-net: Progressive spatio-channel correlation network for image manipulation detection and localization. IEEE Transactions on Circuits and Systems for Video Technology 32, 11 (2022), 7505–7517.
  27. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision. 3730–3738.
  28. Spectral Normalization for Generative Adversarial Networks. In International Conference on Learning Representations.
  29. Edgeconnect: Structure guided image inpainting using edge prediction. In Proceedings of the IEEE/CVF international conference on computer vision workshops. 0–0.
  30. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2536–2544.
  31. Image inpainting with local and global refinement. IEEE Transactions on Image Processing 31 (2022), 2405–2420.
  32. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2149–2159.
  33. Haiwei Wu and Jiantao Zhou. 2021. IID-Net: Image inpainting detection network via neural architecture search and attention. IEEE Transactions on Circuits and Systems for Video Technology 32, 3 (2021), 1172–1185.
  34. Detection of digital doctoring in exemplar-based inpainted images. In 2008 International conference on machine learning and cybernetics, Vol. 3. IEEE, 1222–1226.
  35. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5505–5514.
  36. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision. 4471–4480.
  37. Aggregated Contextual Transformations for High-Resolution Image Inpainting. IEEE Transactions on Visualization and Computer Graphics 29, 7 (2023), 3266–3280.
  38. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision. 14164–14173.
  39. High-resolution image inpainting with iterative confidence feedback and guided upsampling. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIX 16. Springer, 1–17.
  40. Semantic image inpainting with progressive generative networks. In Proceedings of the 26th ACM international conference on Multimedia. 1939–1947.
  41. Perceptual artifacts localization for inpainting. In European Conference on Computer Vision. Springer, 146–164.
  42. Places: A 10 million image database for scene recognition. IEEE Transactions on pattern analysis and machine intelligence 40, 6 (2017), 1452–1464.
  43. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.
  44. Image inpainting by end-to-end cascaded refinement with mask awareness. IEEE Transactions on Image Processing 30 (2021), 4855–4866.

Summary

We haven't generated a summary for this paper yet.

Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com