2000 character limit reached
Synthetic Image Generation in Cyber Influence Operations: An Emergent Threat? (2403.12207v1)
Published 18 Mar 2024 in cs.CY, cs.AI, and cs.CV
Abstract: The evolution of AI has catalyzed a transformation in digital content generation, with profound implications for cyber influence operations. This report delves into the potential and limitations of generative deep learning models, such as diffusion models, in fabricating convincing synthetic images. We critically assess the accessibility, practicality, and output quality of these tools and their implications in threat scenarios of deception, influence, and subversion. Notably, the report generates content for several hypothetical cyber influence operations to demonstrate the current capabilities and limitations of these AI-driven methods for threat actors. While generative models excel at producing illustrations and non-realistic imagery, creating convincing photo-realistic content remains a significant challenge, limited by computational resources and the necessity for human-guided refinement. Our exploration underscores the delicate balance between technological advancement and its potential for misuse, prompting recommendations for ongoing research, defense mechanisms, multi-disciplinary collaboration, and policy development. These recommendations aim to leverage AI's potential for positive impact while safeguarding against its risks to the integrity of information, especially in the context of cyber influence.
- “Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild” In arXiv preprint arXiv:1906.02569, 2019 URL: https://www.gradio.app/
- Alexanre Alaphilippe “Adding a ’D’ to the ABC disinformation framework”, 2020 URL: https://www.brookings.edu/techstream/adding-a-d-to-the-abc-disinformation-framework/
- AUTOMATIC1111 “stable-diffusion-webui”, 2023 URL: https://github.com/AUTOMATIC1111/stable-diffusion-webui
- Alasdair Baverstock “Narcos at (Instagram) war! From guns and girls to big cats and big piles of cash, El Chapo’s sons spark social media battles between Mexican cartel members showing off their sickening wealth” In https://www.dailymail.co.uk/, 2015 URL: https://www.dailymail.co.uk/news/article-3226232/Narcos-Instagram-war-guns-girls-big-cats-big-piles-cash-El-Chapo-s-sons-spark-social-media-battles-Mexican-cartel-members-showing-sickening-wealth.html
- Sam Blazek “SCOTCH: A framework for rapidly assessing influence operations” In GeoTech Cues, 2021 URL: https://www.atlanticcouncil.org/blogs/geotech-cues/scotch-a-framework-for-rapidly-assessing-influence-operations/
- Kathleen M. Carley “Social cybersecurity: an emerging science” In Computational and Mathematical Organization Theory 26.4, 2020, pp. 365–381 DOI: 10.1007/s10588-020-09322-9
- “Extracting Training Data from Diffusion Models” arXiv:2301.13188 [cs] arXiv, 2023 URL: http://arxiv.org/abs/2301.13188
- “Deep fakes: A looming challenge for privacy, democracy, and national security” Publisher: HeinOnline In Calif. L. Rev. 107, 2019, pp. 1753
- Sean Cordey “Cyber Influence Operations: An Overview and Comparative Analysis” Artwork Size: 38 p. Medium: application/pdf, 2019, pp. 38 p. DOI: 10.3929/ETHZ-B-000382358
- Vanda Felbab-Brown and Ariel Fernando Ávila Martínez “COVID-19 and organized crime: Latin American governments are in a state-making competition with crime” In The Brookings Institution, 2020 URL: https://www.brookings.edu/articles/covid-19-and-organized-crime-latin-american-governments-are-in-a-state-making-competition-with-crime/
- “An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion” arXiv:2208.01618 [cs] arXiv, 2022 URL: http://arxiv.org/abs/2208.01618
- “LoRA: Low-Rank Adaptation of Large Language Models” arXiv:2106.09685 [cs] arXiv, 2021 URL: http://arxiv.org/abs/2106.09685
- “Hugging Face – The AI community building the future.” URL: https://huggingface.co/
- Patrick Know “High Life of Crime - Golden AK-47s, lion cubs and Ferraris… Inside the ultra flash lives of Mexico’s brutal narco gangs – funded by Brit cokeheads and fuelling our violent crime epidemic” In The Sun, 2018 URL: https://www.thesun.co.uk/news/6899362/inside-the-wild-world-of-mexicos-narco-gangsters-who-flaunt-their-gold-plated-guns-tigers-and-obscene-wealth-on-social-media/
- “Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense” arXiv:2303.12132 [cs] arXiv, 2023 URL: http://arxiv.org/abs/2303.12132
- Emanuel Maiberg “AI images detectors are being used to discredit the real horrors of war” In 404 Media, 2023 URL: https://www.404media.co/ai-images-detectors-are-being-used-to-discredit-the-real-horrors-of-war/
- Cameron Martel, Gordon Pennycook and David G Rand “Reliance on emotion promotes belief in fake news” Publisher: Springer In Cognitive research: principles and implications 5, 2020, pp. 1–20
- “Emotion dysregulation and belief in conspiracy theories” Publisher: Elsevier In Personality and Individual Differences 204, 2023, pp. 112042
- Ben Nimmo “Anatomy of an Info-War: How Russia’s Propaganda Machine Works, and How to Counter It”, 2015 URL: https://www.stopfake.org/en/anatomy-of-an-info-war-how-russia-s-propaganda-machine-works-and-how-to-counter-it/
- The Mob Museum - National Museum Organized Crime & Law Enforcement “Mexican Cartels deliver Care Packages to Citizens amid Pandemic”, 2020 URL: https://themobmuseum.org/blog/mexican-cartels-deliver-care-packages-to-citizens-amid-pandemic/
- “Diffusers: State-of-the-art diffusion models” In GitHub repository, 2022 URL: https://github.com/huggingface/diffusers
- “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis” arXiv:2307.01952 [cs] arXiv, 2023 URL: http://arxiv.org/abs/2307.01952
- “High-Resolution Image Synthesis with Latent Diffusion Models” arXiv:2112.10752 [cs] arXiv, 2022 URL: http://arxiv.org/abs/2112.10752
- “DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation” arXiv:2208.12242 [cs] arXiv, 2022 URL: http://arxiv.org/abs/2208.12242
- Douglas V Smith “Military Deception and Operational Art” Center for Naval Warfare Studies, US Naval War College, 1993
- “Deep Unsupervised Learning using Nonequilibrium Thermodynamics” In Proceedings of the 32nd International Conference on Machine Learning 37, Proceedings of Machine Learning Research Lille, France: PMLR, 2015, pp. 2256–2265 URL: https://proceedings.mlr.press/v37/sohl-dickstein15.html
- “Large language models (llms) and empathy-a systematic review” Publisher: Cold Spring Harbor Laboratory Press In medRxiv, 2023, pp. 2023–08
- “DISARM: a Framework for Analysis of Disinformation Campaigns” In 2022 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) Salerno, Italy: IEEE, 2022, pp. 1–8 DOI: 10.1109/CogSIMA54611.2022.9830669
- “Unsplash” URL: https://unsplash.com/
- “Transformers: State-of-the-Art Natural Language Processing” Zenodo, 2020 DOI: 10.5281/ZENODO.7391177
- “Adding Conditional Control to Text-to-Image Diffusion Models” arXiv:2302.05543 [cs] arXiv, 2023 URL: http://arxiv.org/abs/2302.05543
- “Towards robust blind face restoration with codebook lookup transformer” In Advances in Neural Information Processing Systems 35, 2022, pp. 30599–30611
- Melanie Mathys (2 papers)
- Marco Willi (2 papers)
- Michael Graber (2 papers)
- Raphael Meier (19 papers)