· Trending Papers
Emergent Mind

Synthetic images aid the recognition of human-made art forgeries

(2312.14998)
Published Dec 22, 2023 in cs.CV and cs.AI

Abstract

Previous research has shown that Artificial Intelligence is capable of distinguishing between authentic paintings by a given artist and human-made forgeries with remarkable accuracy, provided sufficient training. However, with the limited amount of existing known forgeries, augmentation methods for forgery detection are highly desirable. In this work, we examine the potential of incorporating synthetic artworks into training datasets to enhance the performance of forgery detection. Our investigation focuses on paintings by Vincent van Gogh, for which we release the first dataset specialized for forgery detection. To reinforce our results, we conduct the same analyses on the artists Amedeo Modigliani and Raphael. We train a classifier to distinguish original artworks from forgeries. For this, we use human-made forgeries and imitations in the style of well-known artists and augment our training sets with images in a similar style generated by Stable Diffusion and StyleGAN. We find that the additional synthetic forgeries consistently improve the detection of human-made forgeries. In addition, we find that, in line with previous research, the inclusion of synthetic forgeries in the training also enables the detection of AI-generated forgeries, especially if created using a similar generator.

We're not able to analyze this paper right now due to high demand.

Please check back later (sorry!).

Generate a detailed summary of this paper with a premium account.

We ran into a problem analyzing this paper.

Please try again later (sorry!).

Get summaries of trending AI papers delivered straight to your inbox

Unsubscribe anytime.

References

  1. Alberge D. Christie’s caught up as £30m forgeries send shock waves through the art world; 2010. Available from: https://www.theguardian.com/artanddesign/2010/oct/17/christies-forger-art-scam.

  2. Bailey M. At least 45 Van Goghs may well be fakes: The Art Newspaper Investigates; 2021. Available from: https://www.theartnewspaper.com/1997/07/01/at-least-45-van-goghs-may-well-be-fakes-the-art-newspaper-investigates.

  3. Bell P, Offert F. Reflections on connoisseurship and computer vision. Journal of Art Historiography. 2021;24:BO1.
  4. Fractal analysis of Pollock’s drip paintings. Nature. 1999;399. doi:10.1038/20833.
  5. A digital technique for art authentication. Proc Natl Acad Sci USA. 2004;101:17006–10. doi:10.1073/pnas.0406398101.
  6. Image processing for artist identification. IEEE Signal Processing Magazine. 2008;25(4):37–48. doi:10.1109/MSP.2008.923513.
  7. Visual stylometry using background selection and wavelet-HMT-based Fisher information distances for attribution and dating of impressionist paintings. Signal Processing. 2013;93(3):541–553. doi:10.1016/j.sigpro.2012.09.025.
  8. Quantification of artistic style through sparse coding analysis in the drawings of Pieter Bruegel the Elder. Proceedings of the National Academy of Sciences. 2010;107(4):1279–1283. doi:10.1073/pnas.0910530107.
  9. Rhythmic Brushstrokes Distinguish van Gogh from His Contemporaries: Findings via Automated Brushstroke Extraction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2012;34(6):1159–1176. doi:10.1109/TPAMI.2011.203.
  10. Geometric tight frame based stylometry for art authentication of van Gogh paintings. Applied and Computational Harmonic Analysis. 2016;41(2):590–602. doi:10.1016/j.acha.2015.11.005.
  11. Toward Discovery of the Artist’s Style: Learning to recognize artists by their artworks. IEEE Signal Processing Magazine. 2015;32(4):46–54. doi:10.1109/MSP.2015.2406955.
  12. Dobbs T, Ras Z. On art authentication and the Rijksmuseum challenge: A residual neural network approach. Expert Systems with Applications. 2022;200:116933. doi:10.1016/j.eswa.2022.116933.
  13. Machine: The New Art Connoisseur
  14. Fine-tuning Convolutional Neural Networks for fine art classification. Expert Systems with Applications. 2018;114:107–118. doi:10.1016/j.eswa.2018.07.026.
  15. Pancaroglu D. Artist, Style And Year Classification Using Face Recognition And Clustering With Convolutional Neural Networks. Preprint. 2020;doi:10.5121/csit.2020.101604.
  16. Madhubani Art Classification using transfer learning with deep feature fusion and decision fusion based techniques. Engineering Applications of Artificial Intelligence. 2023;119:105734. doi:10.1016/j.engappai.2022.105734.

  17. Authentication of Vincent van Gogh’s Work. In: Tsapatsoulis N, Panayides A, Theocharides T, Lanitis A, Pattichis C, Vento M, editors. Computer Analysis of Images and Patterns. Lecture Notes in Computer Science. Cham: Springer International Publishing; 2021. p. 371–380. Available from: https://dx.doi.org/10.1007/978-3-030-89131-2_34.

  18. Art Authentication with Vision Transformers. Neural Comput & Applic. 2023;doi:10.1007/s00521-023-08864-8.
  19. Paint it Black: Generating paintings from text descriptions

  20. Image Style Transfer Using Convolutional Neural Networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 2414–2423. Available from: https://dx.doi.org/10.1109/CVPR.2016.265.

  21. Analyzing and Improving the Image Quality of StyleGAN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020. p. 8107–8116. Available from: https://dx.doi.org/10.1109/CVPR42600.2020.00813.
  22. Alias-Free Generative Adversarial Networks
  23. High-Resolution Image Synthesis with Latent Diffusion Models. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2022. p. 10674–10685. Available from: https://dx.doi.org/10.1109/CVPR52688.2022.01042.

  24. Zero-Shot Text-to-Image Generation. In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR; 2021. p. 8821–8831. Available from: https://proceedings.mlr.press/v139/ramesh21a.html.

  25. Learning Transferable Visual Models From Natural Language Supervision. In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR; 2021. p. 8748–8763. Available from: https://proceedings.mlr.press/v139/radford21a.html.

  26. DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models
  27. From Impressionism to Expressionism: Automatically Identifying Van Gogh’s Paintings. In: 2016 IEEE International Conference on Image Processing (ICIP); 2016. p. 141–145. Available from: https://dx.doi.org/10.1109/icip.2016.7532335.

  28. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234–241. Available from: https://dx.doi.org/10.1007/978-3-319-24574-4_28.

  29. Available from: https://doi.org/10.5281/zenodo.10276928.

  30. Harvard. Self-portrait with a bandaged ear and Pipe; 2019. Available from: https://harvardartmuseums.org/collections/object/230942?position=0&context=exhibition&id=707.

  31. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Los Alamitos, CA, USA: IEEE Computer Society; 2021. p. 9992–10002. Available from: https://dx.doi.org/10.1109/ICCV48922.2021.00986.

  32. Tan M, Le Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th International Conference on Machine Learning. vol. 97 of Proceedings of Machine Learning Research. PMLR; 2019. p. 6105–6114. Available from: https://proceedings.mlr.press/v97/tan19a.html.

  33. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009. p. 248–255. Available from: https://dx.doi.org/10.1109/CVPR.2009.5206848.
  34. DOA-GAN: Dual-Order Attentive Generative Adversarial Network for Image Copy-Move Forgery Detection and Localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020. p. 4675–4684. Available from: https://dx.doi.org/10.1109/CVPR42600.2020.00473.
  35. Induced and reduced unbounded operator algebras
  36. Towards Universal Fake Image Detectors that Generalize Across Generative Models

  37. Hierarchical Fine-Grained Image Forgery Detection and Localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2023. p. 3155–3165. Available from: https://openaccess.thecvf.com/content/CVPR2023/html/Guo_Hierarchical_Fine-Grained_Image_Forgery_Detection_and_Localization_CVPR_2023_paper.html.

  38. On The Detection of Synthetic Images Generated by Diffusion Models. In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2023. p. 1–5. Available from: https://dx.doi.org/10.1109/ICASSP49357.2023.10095167.
  39. CNN-Generated Images Are Surprisingly Easy to Spot… for Now. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020. p. 8692–8701. Available from: https://dx.doi.org/10.1109/CVPR42600.2020.00872.
  40. Lighting (In)consistency of Paint by Text
  41. Perspective (In)consistency of Paint by Text

  42. Modigliani A. Nu couché; 1916. Available from: https://commons.wikimedia.org/wiki/File:Amedeo_Modigliani,_1916,_Reclining_Nude_(Nu_couch%C3%A9),_oil_on_canvas,_65.5_x_87_cm,_Foundation_E.G._B%C3%BChrle.jpg.

  43. Modigliani A. Portrait de Chaïm Soutine réalisé en 1917; 1917. Available from: https://commons.wikimedia.org/wiki/File:Amedeo_Modigliani_-_Cha%C3%AFm_Soutine_-_1917_-_001.jpg.

  44. Raffaello Sanzio da Urbino. Madonna Solly; 1502. Available from: https://commons.wikimedia.org/wiki/File:1502_Raffael_Madonna_Solly_anagoria.JPG.

  45. Circle of Raphael. Portrait of a Young Man in Red; about 1505. Available from: https://www.getty.edu/art/collection/object/103REA.

  46. Cascone S. Visitors to Major Modigliani Exhibition Demand a Refund After an Expert Concludes the Majority of Works Were Fake; 2018. Available from: https://news.artnet.com/art-world/art-lovers-demand-refund-fake-modigilani-exhibition-1197245.

Show All 46

Test Your Knowledge

You answered out of questions correctly.

Well done!