Auditory Prompt Injection: Methods and Threats

• Auditory prompt injection is the use of sound-based adversarial signals to inject malicious instructions into speech-driven models either through ASR transcription or direct audio channel processing.
• It distinguishes between direct and indirect methods, analyzing threat models such as near-ultrasonic covert playback and context-agnostic adversarial audio delivery.
• Research using systems like SWhisper and AudioHijack demonstrates high success rates in prompt manipulation while revealing challenges in detection, mitigation, and secure multimodal integration.

to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse\nids=['(Ling et al., 14 Mar 2026)','(Chen et al., 16 Apr 2026)','(Gosmar et al., 19 Jan 2026)','(Tai, 8 Jun 2026)','(Rossi et al., 2024)']\nfor i in ids:\n url=f'http://export.arxiv.org/api/query?id_list={i}'\n try:\n data=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\n print(i, 'ok', data[:200].replace('\n',' '))\n except Exception as e:\n print(i, 'ERR', e)\nPY"]} to=container.exec code สำนักเลขานุการ to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request\nurl='https://export.arxiv.org/api/query?id_list=([2603.13847](/papers/2603.13847))'\ntry:\n data=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\n print(data[:500])\nexcept Exception as e:\n print('ERR', e)\nPY"]} to=container.exec code 北京pk赛车 to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse\nquery=urllib.parse.quote('all:auditory prompt injection')\nurl=f'https://export.arxiv.org/api/query?search_query={query}&start=0&max_results=5'\ntry:\n data=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\n print(data[:1500])\nexcept Exception as e:\n print('ERR', e)\nPY"]} to=container.exec code 大发云 to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse, re\nquery=urllib.parse.quote('all:auditory prompt injection')\nurl=f'https://export.arxiv.org/api/query?search_query={query}&start=0&max_results=10'\nxml=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\nprint(xml[:5000])\nPY"]} to=container.exec code 彩神争霸电脑版 to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse, xml.etree.ElementTree as ET\nquery=urllib.parse.quote('all:ultrasonic voice command attack')\nurl=f'https://export.arxiv.org/api/query?search_query={query}&start=0&max_results=10'\nxml=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\nprint(xml[:4000])\nPY"]} to=container.exec code to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse, re\nfor q in ['auditory prompt injection', 'audio jailbreak LALM', 'ultrasonic voice attack microphone nonlinearity']:\n query=urllib.parse.quote('all:'+q)\n url=f'https://export.arxiv.org/api/query?search_query={query}&start=0&max_results=5'\n xml=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\n print('QUERY', q)\n print('entries', xml.count('<entry>'))\n for m in re.finditer(r'<id>http://arxiv.org/abs/([^<]+)</id>.?<title>(.?)</title>', xml, re.S):\n print(m.group(1), m.group(2).strip().replace('\n',' '))\n print('-'*40)\nPY"]} to=container.exec code to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nimport urllib.request, urllib.parse\nq='auditory prompt injection'\nquery=urllib.parse.quote('all:'+q)\nurl=f'https://export.arxiv.org/api/query?search_query={query}&start=0&max_results=5'\nxml=urllib.request.urlopen(url, timeout=20).read().decode('utf-8')\nprint(repr(xml[:1000]))\nprint('entry count', xml.count('<entry>'))\nPY"]} to=container.exec code to=container.exec code: {"cmd":["bash","-lc","python - <<'PY'\nprint('hello')\nPY"]} to=container.exec code Auditory prompt injection is the use of sound to alter what a model treats as operative instructions. In speech-driven LLMs, the injected content typically traverses an ASR stage and becomes text in the model context; in large audio-LLMs (LALMs), the same effect can arise directly in the continuous audio channel, including cases where the adversarial audio is not transcribed as explicit instructions but still steers generation toward attacker-chosen behavior. The resulting attack surface extends ordinary prompt injection into open acoustic channels and includes safety bypass, hidden system-style instructions, unauthorized tool use, and covert command execution (Ling et al., 14 Mar 2026, Chen et al., 16 Apr 2026, Gosmar et al., 19 Jan 2026).

1. Conceptual scope and threat model

Prompt injection is a “broad family of adversarial techniques in which an attacker crafts an input containing instructions that override or subvert the intended task or policy of the system.” In the auditory setting, the decisive property is not the modality itself but the fact that spoken or embedded audio becomes part of the model’s semantic input, either through ASR transcription or through native audio conditioning in end-to-end models (Gosmar et al., 19 Jan 2026).

Current work targets two broad victim classes. The first is the speech-driven pipeline system, in which audio is processed as ASR $\rightarrow$ text LLM. The second is the audio-native or audio-text integrated model, such as GLM-4-Voice, Qwen-Omni, Voxtral-Mini, Phi-4-Multimodal, and related LALMs, which ingest waveform-derived representations directly. Interfaces include mobile phones, smart speakers, browser or app front ends, cloud voice APIs, call-center assistants, and multimodal agents with tool access (Ling et al., 14 Mar 2026, Chen et al., 16 Apr 2026).

The attacker model varies by mechanism but is consistently realistic. In the near-ultrasonic covert-channel setting, the attacker can play audio in the victim’s environment through commodity speakers, cannot modify the victim device directly, has no direct access to the victim’s account or context, and must succeed in a single turn while remaining covert to nearby humans. In the context-agnostic adversarial-audio setting, the attacker is a third party with audio-only access: the attacker can tamper with or supply audio that the model ingests, but cannot modify the system prompt, cannot see or change the user’s instructions, and must preserve strong perceptual stealth (Ling et al., 14 Mar 2026, Chen et al., 16 Apr 2026).

These threat models already imply that auditory prompt injection is not reducible to the classical “voice assistant command” problem. The injected object may be a long and structured prompt, an implicit response prefix that the model treats as its own plan, or an indirect context signal that only manifests after downstream routing and tool execution. This suggests that auditory prompt injection is best understood as prompt injection over an acoustic carrier rather than as mere keyword spoofing.

2. Taxonomic structure and attack surfaces

The literature does not yet provide a single unified taxonomy of auditory prompt injection, but several distinctions recur across recent work. A first distinction is between direct and indirect prompt injection. The early prompt-injection taxonomy for text distinguishes direct prompt injections, in which the attacker directly supplies the malicious prompt, from indirect prompt injections, in which the attacker hides instructions inside third-party content or training data. That taxonomy is channel-agnostic and therefore transfers directly to audio once transcribed or semantically interpreted (Rossi et al., 2024).

A second distinction concerns the semantic locus of the attack. In speech-driven pipelines, malicious spoken content materializes as an ASR transcript and is then merged into the LLM context. In audio-native LALMs, the adversarial audio can steer output behavior even when it is not transcribed as explicit instructions. The latter is central to context-agnostic auditory prompt injection, where the model’s generated response begins with an attacker-specified target response $r_t$ and subsequent autoregressive generation treats that prefix as its own intended action (Chen et al., 16 Apr 2026).

A third distinction concerns audibility and carrier design. Recent work implicitly separates audible from inaudible injection, and short command injection from prompt-based injection. SWhisper focuses on inaudible near-ultrasonic delivery of arbitrary baseband audio, including long jailbreak prompts; this is materially different from earlier short-command ultrasonic demonstrations because the channel is engineered for faithful prompt reconstruction rather than coarse command triggering (Ling et al., 14 Mar 2026).

A fourth distinction concerns routing and observability. The Route-Safety Audit Contract developed for BCI-to-agent systems identifies four audit classes: C1 direct perturbation, C2 context-only injection, C3 adaptive shared-input perturbation, and C4 repeated queries. The C2/C3 distinction transfers cleanly to audio. C2 corresponds to indirect prompt injection through untrusted audio or derived context, while C3 corresponds to adversarial audio that drives both a primary recognizer and a verifier or secondary decoder toward the same attacker-chosen route (Tai, 8 Jun 2026).

At the semantic level, the 301-prompt corpus used for text-side evaluation spans Direct Override, Authority Assertion, Role-Play, Logical Trap, Multi-Step, Obfuscation, Context Injection, Instruction Confusion, Simulated Dialog, and Goal Hijacking. These families map naturally to audio because the relevant variable is the instruction content reaching the model, not whether that content was typed or spoken (Gosmar et al., 19 Jan 2026).

3. Covert near-ultrasonic injection in speech-driven LLMs

The most explicit physical-layer realization of auditory prompt injection is SWhisper, which constructs a covert acoustic channel for speech-driven LLMs under black-box conditions using commodity hardware. Its core mechanism is to encode a baseband speech waveform $m(t)$ containing a jailbreak prompt onto a near-ultrasonic carrier $f_c$ in the 17–22 kHz band, transmit it acoustically, and rely on microphone nonlinearity to demodulate it back into an intelligible baseband replica (Ling et al., 14 Mar 2026).

The microphone nonlinearity is modeled as a polynomial,

$$S_{\text{out}} = k_1 S_{\text{in}} + k_2 S_{\text{in}}^2 + k_3 S_{\text{in}}^3 + \dots$$

and the injected signal takes the form

$S_{\text{in}}(t) = (1+m(t))\cos(2\pi f_c t).$

For near-ultrasonic injection, the quadratic term is the dominant component; after squaring and low-pass filtering, the microphone output contains a baseband component proportional to $k_2 m(t)$. In effect, the device “hears” a prompt that nearby humans do not (Ling et al., 14 Mar 2026).

SWhisper does not attempt an explicit circuit-level model of every nonlinear stage. Instead, it measures an empirical nonlinear transfer matrix $M_{\text{nl}}$ over multiple speakers, microphones, and environments, then reconstructs a desired demodulated baseband spectrum by solving the regularized inverse problem

$$W_{\text{attack}}=\arg\min_X \|M_{\text{nl}}X-W_{\text{target}}\|_2^2+\lambda\|X\|_2^2.$$

The recovered waveform is then modulated with single-sideband synthesis using the Hilbert transform and a Tukey window to reduce spectral leakage. This combination of empirical probing and lightweight channel-inversion pre-compensation is what makes prompt-level delivery practical rather than merely demonstrative (Ling et al., 14 Mar 2026).

The attack is paired with a voice-aware jailbreak generation method designed for intelligibility, brevity, and transferability under speech interfaces. The reported results are strong in black-box settings across commercial and open-source speech-driven LLMs: on commercial models, SWhisper reaches up to 0.94 non-refusal and 0.925 specific-convincing, while a controlled user study finds the injected jailbreak audio perceptually indistinguishable from background-only playback. Robustness is also reported across devices and acoustic conditions, including HIKVISION USB mic, Redmi Note 12, and iPhone 14 Pro; distance 1–4 m with NR dropping only from 0.78 to 0.76; angle 0°–90° with NR = 0.70 and SC = 0.69 even at 90°; and office, restaurant, park, and street backgrounds at 50–60 dB with NR $\ge 0.72$ and SC $r_t$0 (Ling et al., 14 Mar 2026).

The significance of this result is architectural. It demonstrates that auditory prompt injection can be realized as a high-fidelity covert channel for arbitrary spoken prompts, not merely as a short hidden command.

4. Context-agnostic imperceptible hijacking of LALMs

A second major line of work targets large audio-LLMs directly. AudioHijack studies a third-party attacker with audio-only access who cannot control the user’s current or future instructions and therefore must produce context-agnostic adversarial audio. The adversary transforms benign carrier audio $r_t$1 into adversarial audio $r_t$2 such that

$r_t$3

where $r_t$4 is the unknown user context and $r_t$5 is the target misbehavior (Chen et al., 16 Apr 2026).

This formulation differs from direct audio jailbreaks in which the attacker is the user and simply speaks an overt harmful prompt. Here the attacker controls only the audio data. The adversarial signal need not decode into explicit instructions; instead it steers the model’s output distribution so that the first generated tokens match an attacker-chosen target response $r_t$6. For tool-using systems, that target may be a JSON tool call, a text-style tool prefix, or a refusal/persona string that downstream orchestration interprets as legitimate behavior (Chen et al., 16 Apr 2026).

The optimization problem is

$r_t$7

implemented in penalty form as

$r_t$8

Because many LALMs include non-differentiable audio tokenization, AudioHijack uses sampling-based gradient estimation via Gumbel–Softmax over vector-quantized audio codes, with hard argmax in the forward pass and soft samples in the backward pass. Context-agnostic generalization is obtained by multi-context training over auxiliary text and speech instructions and by explicit attention supervision, which enforces a lower bound on attention mass from target response tokens to adversarial audio tokens. Imperceptibility is pursued through convolutional blending: framewise learned kernels initialized from real room impulse responses modulate perturbations into natural reverberation, followed by overlap-add reconstruction and RMS normalization (Chen et al., 16 Apr 2026).

The framework is evaluated on 13 state-of-the-art LALMs spanning discrete-token, continuous-feature, and hybrid audio-text integration schemes. Across 13 models and five non-tool misbehavior categories, average prompt injection success rate is 0.89–0.95 and average behavior match success rate is 0.84–0.94. The paper also reports average success rates of 79%–96% on unseen user contexts with high acoustic fidelity. Tool misuse is especially consequential: Ultravox-v5, Phi-4-Multimodal, and Voxtral-Mini show PISR of approximately 0.90–1.00 and BMSR of approximately 0.79–0.96 depending on tool and output format, with JSON-based tool targets outperforming text-style ones (Chen et al., 16 Apr 2026).

The target misbehaviors are broader than classical safety jailbreaks. They include auditory blindness, prompt refusal, disinformation, phishing delivery, persona control, and tool misuse. Real-world tests on Microsoft Azure and Mistral AI voice agents show that commercial systems can be induced to perform unauthorized searches, download files from attacker-controlled URLs, and execute cascaded exfiltration sequences such as list_calendar followed by send_email, despite benign user instructions (Chen et al., 16 Apr 2026).

5. Detection, mitigation, and auditability

Direct defensive results in the LALM setting remain mixed. Prompt-level in-context defenses that warn about prompt injection reduce BMSR by less than 0.07 on Voxtral-Mini except for somewhat larger reductions on prompt refusal and phishing. Self-reflection detection yields a false positive rate of approximately 0.04 but a true positive rate of only approximately 0.28. Logits-divergence detection with token masking reaches ROC AUC 0.71–0.85 and equal error rate 0.21–0.36, whereas WaveGuard-inspired signal distortions give AUC below 0.6. Attention-deviation detection performs much better against non-adaptive attacks, with precision 0.98 and recall 0.93, but drops to precision 0.90 and recall 0.69 against an adaptive attacker whose BMSR falls by at most 0.10. The defense picture is therefore one of partial observability rather than robust prevention (Chen et al., 16 Apr 2026).

At the text-semantic layer, a more mature mitigation vocabulary already exists and transfers almost directly to auditory systems once malicious speech has become transcript text. The TIVS-O framework evaluates prompt injection with ISR, POF, PSR, CCS, and OSR, where OSR measures the richness of exposed security reasoning. In the evaluated HOPE-inspired Nested Learning architecture, a three-agent pipeline with semantic caching over 301 synthetic prompts from ten attack families achieves zero high-risk breaches at the final stage, with 254 prompts classified as secure, 47 as moderate risk, final ISR = 0.076, POF = 0.059, PSR = 0.986, CCS = 0.933, and OSR = 0.596. Semantic caching at threshold $r_t$9 yields a 41.6% reduction in LLM calls. Although this work is text-only, the paper explicitly argues that the same architecture and metrics apply when the input arrives as an ASR transcript (Gosmar et al., 19 Jan 2026).

For tool-using agents, route-safety evaluation requires stronger audit discipline than accuracy or agreement alone. The Route-Safety Audit Contract defines

$m(t)$0

namely a required per-case log schema, statistical denominator, route-safety event, and pass/fail endpoint. In the BCI-to-agent setting, provenance blocks C2 routes at $m(t)$1, agreement-plus-provenance routes C3 flips at $m(t)$2, and confirmation-plus-provenance routes them at $m(t)$3. A split-conformal confirmation channel produces FAR $m(t)$4 at clean utility $m(t)$5 for $m(t)$6 and FAR $m(t)$7 at clean utility $m(t)$8 for $m(t)$9 under acquisition isolation, but an attacker-controllable confirmation channel breaks the bound to approximately 1. This suggests that auditory prompt injection defenses must log provenance, route outcomes, execution policy, and confirmation status, and that agreement between recognizers is not an intent certificate (Tai, 8 Jun 2026).

6. Security significance, misconceptions, and open problems

Auditory prompt injection affects more than voice chat. Identified exposure points include smart speakers, mobile assistants, browser and app front ends, call-center bots, media summarizers, multimodal assistants that fetch third-party audio, and enterprise or high-stakes systems in finance, healthcare, and home or vehicle control. Attack delivery can be over the air through loudspeakers, embedded in TV or radio content, inserted into uploaded or streamed audio, or realized as near-ultrasonic playback that is covert to human listeners (Ling et al., 14 Mar 2026, Chen et al., 16 Apr 2026).

Several misconceptions are not supported by the current evidence. First, auditory prompt injection is not limited to short voice commands; SWhisper shows prompt-based injection of long, structured jailbreaks. Second, it is not only an ASR misrecognition problem; AudioHijack explicitly studies cases in which the adversarial audio does not need to be transcribed as explicit instructions. Third, strong clean accuracy or agreement between models does not certify safety, because C2 context-only injections are invisible to signal-side monitors and C3 shared-input attacks can satisfy the very agreement predicate used as a defense. Fourth, confirmation and mediation reduce risk, but they are not intent certificates (Ling et al., 14 Mar 2026, Chen et al., 16 Apr 2026, Tai, 8 Jun 2026).

Current limitations are also clear. SWhisper is centered on speech-driven interfaces and near-ultrasonic playback, while AudioHijack depends on white-box knowledge of at least one surrogate LALM and shows imperfect transfer to different-model-family commercial agents. The text-side mitigation work relies on synthetic attack corpora and an empirically chosen caching threshold $f_c$0. The audit-contract work is developed in a BCI setting and therefore transfers to audio as a principled analogue rather than as a direct audio benchmark. Across studies, long contexts, strong noise, extreme speaker diversity, and attacker control of the confirmation channel remain difficult cases (Chen et al., 16 Apr 2026, Gosmar et al., 19 Jan 2026, Tai, 8 Jun 2026).

Open research directions follow directly from these constraints. The literature points to the need for audio-grounded prompt-injection corpora, joint ASR+LLM or LALM evaluations, provenance-aware logging, tool-use guards that verify whether actions are justified by user-visible intent, security-specialized embeddings robust to ASR variability, confirmation channels with enforceable non-controllability assumptions, and training objectives that prevent pathological over-attention to adversarial audio. More broadly, current results indicate that auditory prompt injection should be treated as a first-class prompt-security problem for multimodal agents, not as an edge case of voice command spoofing.