CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (1801.08535v3)

Abstract: The popularity of ASR (automatic speech recognition) systems, like Google Voice, Cortana, brings in security concerns, as demonstrated by recent attacks. The impacts of such threats, however, are less clear, since they are either less stealthy (producing noise-like voice commands) or requiring the physical presence of an attack device (using ultrasound). In this paper, we demonstrate that not only are more practical and surreptitious attacks feasible but they can even be automatically constructed. Specifically, we find that the voice commands can be stealthily embedded into songs, which, when played, can effectively control the target system through ASR without being noticed. For this purpose, we developed novel techniques that address a key technical challenge: integrating the commands into a song in a way that can be effectively recognized by ASR through the air, in the presence of background noise, while not being detected by a human listener. Our research shows that this can be done automatically against real world ASR applications. We also demonstrate that such CommanderSongs can be spread through Internet (e.g., YouTube) and radio, potentially affecting millions of ASR users. We further present a new mitigation technique that controls this threat.

• The paper presents a robust method for generating adversarial audio that embeds covert commands into music to deceive both open-source and commercial ASR systems.
• It employs a PDF-ID matching algorithm with gradient descent optimization, achieving 100% success in WAV-to-API attacks and 96% in over-air tests.
• The study underscores the urgent need for advanced defenses, like audio turbulence and squeezing, to secure voice-controlled devices from covert adversarial attacks.

An Overview of "CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition"

The utilization of automatic speech recognition (ASR) systems, such as Amazon Alexa and Google Assistant, has increased significantly in various applications, thereby raising essential questions regarding their security robustness. The discussed paper, "CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition," explores potential vulnerabilities within state-of-the-art ASR platforms. The authors focus on the creation of adversarial audio inputs woven into music tracks, termed "CommanderSongs," which are engineered to be undetectable by human listeners while being accurately processed by ASR systems to execute unauthorized commands. The paper presents comprehensive empirical evaluations demonstrating successful attacks on both open-source and commercial ASR systems, further emphasizing the broader implications of adversarial manipulations in voice recognition technologies.

Technical Contributions and Results

A notable contribution of this paper lies in the development of an innovative technique for creating adversarial sound samples. The authors utilize the Kaldi toolkit to craft these adversarial audio inputs that exploit DNN vulnerabilities, targeting the probability density function in acoustic models. This process involves a PDF-ID sequence matching algorithm coupled with gradient descent optimization, allowing the precise embedding of malicious voice commands into innocuous music tracks. These perturbations aim to maintain auditory fidelity to original sounds for the human ear while steering deep neural networks toward predefined target outputs in speech recognition scenarios.

The experiments conducted demonstrate significant success in both WAV-to-API (WTA) attacks, where adversarial wave files are directly fed to ASR APIs, and WAV-Air-API (WAA) attacks, which engage with ASR systems through speaker playback. The researchers claim a 100% success rate in WTA evaluations with an SNR range of 14-18.6 dB, signifying minimal perturbation. For practical attacks over the air, WAA trials achieved a 96% success rate with various standard consumer-grade audio devices, affirming the practicality of these adversarial manipulations in real-world contexts.

Broader Implications and Future Directions

The findings from this research carry substantial implications for both the security community and vendors of ASR systems. By highlighting the feasibility of remotely deliverable and electronically undetectable adversarial audio attacks, the paper underscores the pressing need for robust countermeasures to safeguard voice-controlled devices and applications. The successful transferability of these CommanderSongs to commercial entities, such as iFLYTEK, and the potential propagation through channels like radio and YouTube amplify the scope of potential threats, necessitating collaborative efforts for effective defenses.

The development of defense mechanisms against such adversarial threats is paramount. This paper briefly explores potential strategies including audio turbulence and audio squeezing, which introduce controlled perturbations or compressions in audio inputs to disrupt adversarial patterns without compromising legitimate operations. These approaches present a promising area for future research and innovation, focusing on enhancing the resilience of machine learning models against surreptitious manipulations in voice-controlled ecosystems.

In conclusion, "CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition" introduces a rigorous analytical and empirical framework for understanding and addressing security challenges in ASR systems. It invites the research community to further investigate the defensive techniques against adversarial audio attacks, ensuring the ongoing trust and reliability of intelligent voice-based technologies in everyday applications.