Infecting Generative AI With Viruses (2501.05542v1)

Abstract: This study demonstrates a novel approach to testing the security boundaries of Vision-LLM (VLM/ LLM) using the EICAR test file embedded within JPEG images. We successfully executed four distinct protocols across multiple LLM platforms, including OpenAI GPT-4o, Microsoft Copilot, Google Gemini 1.5 Pro, and Anthropic Claude 3.5 Sonnet. The experiments validated that a modified JPEG containing the EICAR signature could be uploaded, manipulated, and potentially executed within LLM virtual workspaces. Key findings include: 1) consistent ability to mask the EICAR string in image metadata without detection, 2) successful extraction of the test file using Python-based manipulation within LLM environments, and 3) demonstration of multiple obfuscation techniques including base64 encoding and string reversal. This research extends Microsoft Research's "Penetration Testing Rules of Engagement" framework to evaluate cloud-based generative AI and LLM security boundaries, particularly focusing on file handling and execution capabilities within containerized environments.

• The paper evaluates the security of Vision-Language Models (VLMs) by embedding EICAR test files within image metadata and testing their handling capabilities.
• Researchers successfully masked the EICAR signature in image metadata and extracted the test file using Python within LLM environments, demonstrating potential for undetected malware handling.
• The findings highlight the critical need for automated file inspections, standardized security frameworks, and real-time detection of multi-stage manipulation attempts in LLM environments.

Security Evaluation of Vision-LLMs through EICAR-Embedded Image Penetration Tests

The research paper "Infecting Generative AI With Viruses" by David A. Noever and Forrest McKee delineates an innovative methodology to assess the security resilience of Vision-LLMs (VLM/LLM). By employing the EICAR test file embedded within JPEG images, the study successfully probes the ability of various LLM platforms to handle potentially malicious data and highlights potential vulnerabilities, specifically in file manipulation and execution. Utilizing OpenAI GPT-4o, Microsoft Copilot, Google Gemini 1.5 Pro, and Anthropic Claude 3.5 Sonnet as their test-beds, the authors introduced viral surrogate challenges described as a novel dimension in LLM security evaluations.

Key Protocols and Findings

1. EICAR String Masking: The paper presents a consistent ability to obscure the EICAR signature in image metadata, avoiding detection. This finding implies that current LLM security frameworks may inadequately detect and manage embedded manipulated data within image metadata.
2. File Extraction and Execution: Using Python within the LLM virtual environments, the research successfully extracted the test file from JPEGs, demonstrating that surrogate malware could be parsed and handled without detection. This capability underscores potential risks in LLMs' handling binary manipulation tasks that lack extensive security measures.
3. Obfuscation Techniques: The research explores techniques such as base64 encoding and string reversal to illustrate complexities in detecting embedded malware. The several obfuscation methods utilized underscore a critical shortfall in existing LLM security protocols, which may not comprehensively isolate or control manipulated exogenous data.

The study promulgates extending the "Penetration Testing Rules of Engagement" framework from Microsoft Research to outline security lapses in generative AI models, primarily addressing file handling and execution tasks within LLM containerized environments.

Implications and Future Directions

The study reveals potential uplift vulnerabilities in LLM environments, where multi-stage manipulations of malicious payloads can progress undetected. The findings encourage further exploration of LLM security, particularly emphasizing the necessity of:

• Automated File Inspections: Developing protocols focused on detecting sophisticated steganographic techniques within image uploads to bolster LLM security resilience.
• Standardized Security Frameworks: Crafting comprehensive security protocols tailored to vision-LLMs that integrate traditional and novel penetration testing methodologies.
• Cross-Platform Vulnerability Investigations: Understanding potential cross-platform weaknesses when handling malicious files across various LLM services.

The tested LLMs’ limited discernment capabilities regarding embedded steganographic content call for enhancements in LLM security frameworks. The researchers suggest prioritizing real-time detection methods of multi-stage manipulation attempts to strengthen prevention controls against potential distributed network attacks.

Conclusion

This paper contributes significantly to the understanding of LLM security boundaries, illustrating the nuanced challenges posed by embedded malware in seemingly innocuous data files. The proposed methodologies and findings prompt the re-evaluation of LLM security implementations in commercial and enterprise applications, potentially contributing to new standards in AI security measures. While immediate threats in containerized LLM environments appear mitigated due to isolation techniques, the study’s implications highlight the urgency for adapting robust security inspections in evolving AI models.