Adversarial Machine Learning -- Industry Perspectives (2002.05646v3)

Abstract: Based on interviews with 28 organizations, we found that industry practitioners are not equipped with tactical and strategic tools to protect, detect and respond to attacks on their Machine Learning (ML) systems. We leverage the insights from the interviews and we enumerate the gaps in perspective in securing machine learning systems when viewed in the context of traditional software security development. We write this paper from the perspective of two personas: developers/ML engineers and security incident responders who are tasked with securing ML systems as they are designed, developed and deployed ML systems. The goal of this paper is to engage researchers to revise and amend the Security Development Lifecycle for industrial-grade software in the adversarial ML era.

• The paper reveals that 25 out of 28 organizations lack proper adversarial ML tools, underscoring the urgent need for robust security enhancements.
• It proposes adapting the Software Security Development Lifecycle to secure ML systems by addressing unique adversarial threats.
• The study advocates integrating adversarial testing, static/dynamic analysis, and tailored incident response into continuous ML security processes.

Industry Perspectives on Adversarial Machine Learning

The presented paper addresses the critical gap between the current practices and necessary advancements required to secure Machine Learning (ML) systems against adversarial attacks, from an industry perspective. It draws from interviews with 28 organizations, spanning sectors such as finance, healthcare, and government, to highlight that industry practitioners are inadequately equipped with both tactical and strategic tools to protect ML systems. The paper serves as a call to action, aiming to amend the existing Security Development Lifecycle (SDL) to include provisions against adversarial ML threats.

Key Contributions and Findings

1. Need for Adequate Tools and Guidance: The paper reveals that 25 out of the 28 organizations acknowledged lacking the appropriate tools and guidance to secure their ML systems. This includes a deficiency in adversarial ML knowledge, indicating an urgent need for an enhancement of current security practices and resources.
2. Security Development Lifecycle (SDL) Framework: The authors propose applying the SDL framework, which is traditionally used for securing software, to ML systems. The fundamental step involves recognizing the unique nature of adversarial ML attacks which differ significantly from traditional software vulnerabilities.
3. Survey Insights: The paper highlights several gaps through the lens of software developers/ML engineers and security incident responders. It suggests improvements in several areas, including:

• Establishing a curated repository of adversarial ML attacks akin to the MITRE ATT&CK framework in traditional cybersecurity.
• Developing adversarial ML-specific secure coding practices.
• Implementing static and dynamic analysis tools tailored for ML systems.
• Enhancing auditing and logging within ML environments to aid incident response.
• Integrating adversarial testing in continuous integration/continuous delivery pipelines.

Implications and Future Research Agenda

The authors emphasize that ML is rapidly becoming integral to many organizations, and securing these systems is paramount as the technology scales. The paper outlines the potential paths for future research and application:

• Detection and Monitoring: This involves developing robust and shareable adversarial detection mechanisms that integrate seamlessly with existing security information and event management (SIEM) systems.
• Red Teaming and Transparency Centers: Calling for an industry-standard practice for simulating adversarial attacks (red teaming) and establishing transparency centers where source code can be scrutinized for vulnerabilities.
• Incident Response and Forensics: Developing comprehensive strategies to quantify, contain, and understand the blast radius of attacks on ML systems.

Challenges and Research Directions

The authors identify the transformation required in both theoretical and practical spheres:

• Tracking and Scoring ML Vulnerabilities: Just as Common Vulnerabilities and Exposures (CVE) help track software vulnerabilities, a system is needed for ML systems that accommodates the unique challenges and risks associated with adversarial threats.
• Integration into Existing Processes: Ensuring the enhanced security features and mechanisms can integrate smoothly into existing organizational processes with minimal disruption.
• Coordinated Industry Response: The need for collective industry effort to fill gaps in knowledge, tools, and strategies to combat adversarial ML challenges.

Conclusion

By highlighting these gaps, the paper provides a detailed road map for academia and industry to come together to secure ML systems. It underlines the necessity of adopting an SDL approach tailored for adversarial ML and fosters an environment where adversarial vulnerabilities are treated with the same diligence as traditional software vulnerabilities, thereby protecting the value and integrity ML brings to organizations globally.