Latent-Space Attacks: Mechanisms & Threats

• Latent-space attacks are adversarial strategies that manipulate internal neural representations, such as latent codes or hidden states, to steer outputs without altering raw inputs.
• They exploit various access routes including direct latent perturbations, input modifications optimized through latent objectives, and inference-time activation interventions.
• Defensive approaches require multi-layered strategies like adversarial training, latent anomaly detection, and integrity checks to secure models across modalities and federated systems.

Latent-space attacks are adversarial procedures that target a model’s internal continuous representations—latent codes, encoder outputs, residual streams, hidden states, KV caches, or other learned bottlenecks—rather than acting only on raw inputs or surface prompts. Across variational autoencoders, image generators, audio-LLMs, language-model residual streams, semantic communication systems, federated learning pipelines, and latent-based multi-agent systems, the shared mechanism is to steer an intermediate representation so that downstream computation behaves as if a different input, instruction, or class were present. Depending on the setting, this can induce targeted outputs, suppress refusal, survive lossy preprocessing, reconstruct private data, or implant persistent backdoor behavior (Creswell et al., 2017, Ziv et al., 29 Dec 2025, Piras et al., 20 May 2026, Xi et al., 3 Dec 2025, Shi et al., 2023, Wang et al., 27 May 2026, Parekh, 1 Apr 2026).

1. Conceptual scope and taxonomy

In the most direct form, a latent-space attack writes to the internal variable itself: a VAE latent $z$, a codec latent, a semantic communication code, or a transformer hidden state. In an indirect form, the attack perturbs the input but optimizes that perturbation only through its effect on the latent representation. The latter is central in encoder-only audio attacks, where a waveform perturbation is learned specifically to make the encoder trajectory approximate a target latent sequence, without accessing the downstream decoder (Ziv et al., 29 Dec 2025). A related pattern appears in image attacks that optimize a generative-model latent and decode back to pixels, thereby replacing unconstrained pixel noise with perturbations that stay on a learned manifold (Shaar et al., 6 Mar 2026, Upadhyay et al., 2020).

A second distinction concerns access. Some attacks are man-in-the-middle modifications of transmitted or stored latents, as in LatentPoison’s additive $\Delta z$ on VAE codes and latent manipulation in semantic communication systems (Creswell et al., 2017, Xi et al., 3 Dec 2025). Others are inference-time activation interventions inside LLMs, where the attack perturbs residual-stream activations or post-attention states to cross from a refusal region into a compliant region, or from a safe state into a jailbreak state (Piras et al., 20 May 2026, Chia et al., 12 Mar 2025). A third class consists of training-time implants: latent backdoors are encoded as learned directions or attractors that are later activated by a trigger, as in backdoor channels hidden in latent geometry and continuous-reasoning backdoors (Eggen et al., 13 May 2026, Parekh, 1 Apr 2026).

Some recent work broadens the notion further by treating latent geometry itself as the vulnerability. “Latent space discontinuities” are described as regions where small latent displacements cause disproportionate output changes, allowing black-box prompt manipulations to steer inference into unstable or poorly aligned regions without direct latent access (Paim et al., 1 Nov 2025). In this view, latent-space attacks include both explicit perturbation of internal variables and indirect induction of harmful latent trajectories through carefully chosen inputs.

2. Attack surfaces and threat models

The recurring attack surface is the bottleneck or interface through which one subsystem controls another. In audio-LLMs, the encoder output is described as the “sole control channel” or “single conduit” through which audio steers the LLM, so matching a target latent trajectory can make the system behave as if a target utterance had been spoken, even when decoder parameters and safety layers are inaccessible (Ziv et al., 29 Dec 2025). CodecAttack exploits an analogous bottleneck at the neural codec latent: lossy codecs preserve what their own representation regards as salient, so perturbations crafted inside that latent subspace survive downstream compression better than generic waveform noise (Roh et al., 19 May 2026).

In semantic communication, the latent transmitted over the channel is itself the protected object. A man-in-the-middle adversary who intercepts $z$ can replace it with $z'$ that decodes to attacker-chosen semantics while remaining statistically consistent with the legitimate latent distribution, making simple distributional checks ineffective (Xi et al., 3 Dec 2025). In latent-based multi-agent systems, the attack surface extends to node-level hidden states and to inter-agent KV-cache handoffs; the latter are especially consequential because they form latent working memory for downstream agents and are not exposed to text inspection (Wang et al., 27 May 2026).

Language-model attacks introduce another geometry-driven threat model. Refusal suppression can be cast as evasion against a linear probe trained to separate refused from answered prompts in latent space. Under that formulation, the attacker need not alter prompts at all; steering the residual stream across the probe’s decision boundary is sufficient to suppress refusal (Piras et al., 20 May 2026). In continuous latent reasoning, the threat model is supply-chain rather than query-time: a poisoned checkpoint contains a learned embedding vector $\phi$ such that a trigger token causes the model’s multi-pass latent trajectory to collapse onto an attacker-chosen answer attractor (Parekh, 1 Apr 2026).

Federated learning demonstrates that the attack surface can be analytic rather than behavioral. Scale-MIA assumes a malicious parameter server that can craft the parameters of existing head layers while still receiving only secure-aggregated updates. The attack then reconstructs latent space representations from the aggregated gradient tensor and decodes them into user inputs, showing that secure aggregation does not prevent privacy loss when the global model’s latent interface is adversarially engineered (Shi et al., 2023).

3. Optimization frameworks and representative constructions

A useful abstraction is to write an encoder $E$, a latent representation $z$, and a downstream generator or predictor $G$. The attack then seeks either direct latent tampering $z' = z + \delta_z$ or an input perturbation $x+\delta_x$ such that $\Delta z$0 lands in an attacker-chosen region. The specific objective depends on whether the goal is targeted generation, refusal evasion, private-data reconstruction, or backdoor activation.

In “Breaking Audio LLMs by Attacking Only the Encoder,” the perturbation is learned in waveform space but driven entirely by an encoder-side latent alignment objective. The attack solves

$\Delta z$1

with a per-frame cosine loss and encoder-only gradients; the downstream decoder is never differentiated (Ziv et al., 29 Dec 2025). CodecAttack instead perturbs the continuous latent $\Delta z$2 of EnCodec, decodes $\Delta z$3, and hardens the perturbation with multi-bitrate straight-through expectation over transformation, so robustness is optimized against the actual compression channel rather than against the target Audio LLM alone (Roh et al., 19 May 2026).

LatentPoison provides the canonical direct-latent formulation. It learns one reusable vector $\Delta z$4 to flip a binary attribute after decoding: $\Delta z$5 with $\Delta z$6 controlling sparsity or dense small-norm change (Creswell et al., 2017). In semantic communication, Test-Time Adaptation Latent Manipulation updates the intercepted latent by gradient descent on a CLIP-based semantic loss, whereas Diffusion-based Re-encoding synthesizes a target-compatible latent using a latent diffusion model trained on the same latent space as the communication codec (Xi et al., 3 Dec 2025).

Language-model activation steering has its own closed-form geometry. “Latent-space Attacks for Refusal Evasion in LLMs” models refusal as a linear decision function $\Delta z$7. Standard ablation is shown to be orthogonal projection onto the refusal boundary, i.e., minimum-confidence evasion. Controlled Latent-space Evasion goes further by projecting activations past the boundary with a chosen margin $\Delta z$8, so the hidden state enters the compliant half-space instead of merely touching the separator (Piras et al., 20 May 2026). REALISTA restricts continuous latent optimization to a simplex over input-dependent directions that are already validated as semantically equivalent rephrasings, then uses projected Langevin dynamics with Gumbel-Softmax-based gradients to search that constrained subspace (Liang et al., 12 May 2026).

The diversity of these constructions is summarized below.

Attack surface	Core mechanism	Representative papers
Direct latent code or bottleneck	Additive $\Delta z$9, codec-latent perturbation, latent semantic editing	(Creswell et al., 2017, Roh et al., 19 May 2026, Xi et al., 3 Dec 2025)
Input perturbation optimized through latent objectives	Waveform perturbation aligned to target encoder trajectory; generative-latent optimization	(Ziv et al., 29 Dec 2025, Shaar et al., 6 Mar 2026, Upadhyay et al., 2020)
Inference-time activation steering	Residual-stream projection, additive hidden-state shifts, safe-to-jailbreak state perturbation	(Piras et al., 20 May 2026, Chia et al., 12 Mar 2025, Liang et al., 12 May 2026)
Training-time latent backdoors	Spiked latent directions, trigger embedding in continuous reasoning, latent-distribution poisoning	(Eggen et al., 13 May 2026, Parekh, 1 Apr 2026, Chacon et al., 2020)
Latent privacy extraction	Closed-form recovery of latent representations from aggregated gradients	(Shi et al., 2023)
Latent communication sabotage	Hidden-state and KV-cache interventions in multi-agent collaboration	(Wang et al., 27 May 2026)

4. Empirical findings across modalities

The empirical record now shows that latent-space attacks are not confined to a single modality or access regime. In audio-LLMs, the encoder-only universal targeted latent-space attack trained on LibriSpeech reaches macro-average targeted ASR values from $z$0 to $z$1 across targets, transfers zero-shot to LibriSpeech test-other, MInDS-14, and Speech Commands, and achieves $z$2 ASR for a random-noise baseline under the same amplitude bound. Relative to an end-to-end baseline over 1,000 iterations, it uses 2.0× lower peak VRAM, is 3.7× faster, and yields 3.7× higher throughput (Ziv et al., 29 Dec 2025). CodecAttack shows a complementary phenomenon under realistic deployment preprocessing: it achieves an average $z$3 target-substring ASR on Opus at moderate bitrates, while a waveform-domain baseline trained with identical EoT hardening never exceeds $z$4 at any bitrate; the same attack transfers without retraining to held-out codecs, reaching up to $z$5 ASR on MP3 and up to $z$6 on AAC-LC (Roh et al., 19 May 2026).

Language-model results are similarly strong, but the objectives vary. Controlled Latent-space Evasion reports an average ASR of $z$7 across 15 instruction-tuned, multimodal, and reasoning models, outperforming multiple refusal-ablation baselines; the additive CLE-A variant is stronger still and retains benign-task performance with only modest changes on MMLU, ARC, and TruthfulQA (Piras et al., 20 May 2026). REALISTA, which constrains latent search to semantically valid directions, reaches ASR@30 values such as $z$8 on Llama-3-3B and $z$9 on Llama-3-8B while keeping semantic coherence error and semantic equivalence error low, and it remains effective against black-box commercial reasoning models where prior realistic attacks are ineffective (Liang et al., 12 May 2026). The black-box discontinuity-based jailbreak study reports successful attacks on 6 of 7 anonymized LLMs within at most five turns, while the remaining model showed instability rather than explicit policy violation; the same work also reports identity correspondence in 11 of 12 sampled diffusion outputs generated from semantically neutral prompts augmented with rare-token noise (Paim et al., 1 Nov 2025).

Backdoor-style latent attacks show that persistence is possible even without explicit prompt engineering at inference. ThoughtSteer achieves at least $z'$0 attack success rate with near-baseline clean accuracy across Coconut and SimCoT models, transfers to held-out reasoning benchmarks at $z'$1–$z'$2 without retraining, survives 25 epochs of clean fine-tuning except under aggressive learning-rate and weight-decay combinations, and evades all five evaluated active defenses (Parekh, 1 Apr 2026). In modern vision backdoors, hidden latent channels implemented as low-rank covariance spikes in a single linear layer yield ASR values typically between about $z'$3 and $z'$4 while preserving clean accuracy within a few percentage points, and standard post-training defenses such as pruning, clipping, parameter noise, fine-tuning, fine-pruning, and Neural Cleanse do not remove the backdoor without unacceptable utility loss (Eggen et al., 13 May 2026).

Earlier latent-code studies already exhibited the basic behavioral effect. LatentPoison flips smile/no-smile labels in a deep VAE with high confidence, while keeping the epsilon-gap confidence discrepancy in ranges such as $z'$5–$z'$6 depending on direction, regularizer, and training regime (Creswell et al., 2017). Latent-space poisoning with a $z'$7-VAE and class-conditioned latent mixing produces out-of-distribution but perceptually plausible adversarial examples that fool adversarially trained classifiers, reporting attack success rates of about $z'$8 on MNIST, $z'$9 on SVHN, and $\phi$0 on CelebA without querying or differentiating through the target classifier (Upadhyay et al., 2020). On the privacy side, Scale-MIA reports high recovery rates and high PSNR at much larger batch sizes than iterative inversion attacks; for AlexNet on CIFAR-10, recovery remains about $\phi$1 at batch size 64 and $\phi$2 at 256, with PSNR around $\phi$3 dB and $\phi$4 dB respectively, while the attack runs in milliseconds rather than minutes (Shi et al., 2023).

Latent-based collaborative systems introduce a further mode of failure: latent-only sabotage without adversarial text. In a multi-agent system with planner, critic, refiner, and judger roles, clean GSM8K accuracy of $\phi$5 drops to $\phi$6 under a RePS edge-level attack on the critic-to-refiner KV handoff, and clean OpenBookQA accuracy of $\phi$7 drops to $\phi$8 under a planner-to-critic edge-level attack, with output-health checks indicating that these effects are not reducible to generic degeneration (Wang et al., 27 May 2026).

5. Detection, defenses, and recurring limitations

Defense proposals concentrate on the same bottlenecks that attacks exploit. Audio work argues for encoder-focused adversarial training, randomized or robust feature extraction, latent anomaly detection, and input-side re-encoding or denoising, but the encoder-only universal audio attack does not empirically benchmark such defenses, leaving the mitigation picture incomplete (Ziv et al., 29 Dec 2025). CodecAttack directly challenges a previously common assumption—that codec compression sanitizes adversarial audio—by showing that compression preserves its own latent subspace and therefore preserves attacks optimized inside that subspace; the paper’s defense suggestions therefore move toward latent-space adversarial training, cross-codec re-synthesis checks, and randomized preprocessing rather than “re-encode to sanitize” (Roh et al., 19 May 2026).

Several lines of work propose explicit latent detection. LatentPoison analyzes deviations from a Gaussian prior in VAE space and suggests density-based detection together with monitoring unusually high classifier confidence on decoded outputs (Creswell et al., 2017). Deep Latent Defence trains encoders on intermediate-layer representations and uses $\phi$9-NN non-conformity in latent space to detect attacks; on MNIST it reports robustness values such as $E$0 against PGD and $E$1 against CW, though a white-box adaptive $E$2 attack that jointly optimizes misclassification and centroid matching still reduces the margin of safety, especially on SVHN (Zizzo et al., 2019). For multi-agent latent attacks, two runtime monitors are evaluated: a direction-aware projection detector and a direction-agnostic layer-profile detector, with the latter reaching high TPR on edge-level KV attacks at low clean FPR while remaining much weaker on node-level hidden-state attacks (Wang et al., 27 May 2026).

Backdoor detection has also been recast as latent distribution analysis. A VCAE-based study of backdoor attacks reports that latent-space entropy rises from $E$3 in a clean model to $E$4 under poisoning, a $E$5 increase, while the KL divergence between clean and backdoored latent distributions reaches $E$6; the authors use pairwise copula plots and decoder samples to expose trigger-induced dependency shifts that persist even when the attacked encoder processes clean data (Chacon et al., 2020). ThoughtSteer, by contrast, finds that unsupervised latent detection can fail on large continuous-reasoning models, while supervised probes achieve AUC $E$7 from the first latent step onward; the same work recommends embedding-row inspection, trajectory probes, Jacobian stability constraints, and Neural Collapse-aware regularization (Parekh, 1 Apr 2026).

Across papers, the limitations are consistent. Many attacks assume gradient access to the attacked encoder, decoder, or residual stream; many provide only single-model evaluations; perceptual or physical robustness is often only partially measured; and cross-architecture transfer is frequently proposed conceptually rather than established experimentally. The black-box discontinuity study is explicit that its results are preliminary, use anonymized models, and rely on relatively small samples, even though it argues for a broad architectural vulnerability (Paim et al., 1 Nov 2025). This suggests that the empirical maturity of latent-space attacks is uneven: some domains now have strong quantitative evidence, while others remain primarily mechanism demonstrations.

6. Broader significance for robustness, alignment, and system design

Latent-space attacks alter the standard boundary between adversarial examples, backdoors, privacy attacks, and interpretability. In alignment research, refusal suppression becomes a geometric evasion problem rather than a prompt-formatting problem: once refused and answered states are linearly or near-linearly separable, steering can be formulated as moving a representation across a decision boundary with controlled margin (Piras et al., 20 May 2026). In mechanistic analyses of jailbreaks, safe and jailbroken states appear as separable latent subspaces or attractor-like regions, and causal perturbations to those states can induce policy-violating behavior even when the visible prompt remains benign (Chia et al., 12 Mar 2025).

The same logic extends beyond standalone models. Semantic communication systems, latent-based multi-agent systems, and continuous latent reasoners reduce or eliminate visible intermediate text, which improves efficiency or flexibility but also relocates attack surface into states that external observers cannot naturally audit (Xi et al., 3 Dec 2025, Wang et al., 27 May 2026, Parekh, 1 Apr 2026). In federated learning, the latent space is the privacy bottleneck rather than the public interface: once the attacker can recover latent representations analytically, secure aggregation no longer guarantees user-level privacy (Shi et al., 2023). In large multimodal systems, reuse of shared encoders and learned codecs creates a monoculture effect in which compromise of a common bottleneck can generalize across many downstream stacks (Ziv et al., 29 Dec 2025, Roh et al., 19 May 2026).

Latent-space attacks also have a constructive counterpart. Counterfactual Attacks reframes realistic counterfactual generation as a latent-space adversarial attack on the representation of an image along a low-dimensional manifold, explicitly linking adversarial optimization with explanation and feature attribution (Goldwasser et al., 21 Apr 2025). Defensive work such as LATPC likewise uses latent-space attacks during training: it identifies safety-critical latent dimensions, applies mask-based refusal feature removal as a training-time adversary, and then calibrates embeddings at inference to reduce over-refusal, reporting a better safety–utility balance than several existing jailbreak defenses (Yi et al., 18 Jan 2025).

The cumulative implication is that robustness evaluation cannot be confined to raw input perturbations or surface-level prompt filters. Learned bottlenecks, residual trajectories, latent communication channels, and hidden-state attractors are now primary security boundaries. This suggests that future defenses will need to combine architectural hardening, latent-space monitoring, adversarial training on intermediate representations, and explicit integrity guarantees for transmitted or reused latents, rather than relying solely on sanitization of visible inputs or moderation of visible outputs.