REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

Abstract: LLMs achieve strong performance across many tasks but remain vulnerable to hallucinations, motivating the need for realistic adversarial prompts that elicit such failures. We formulate hallucination elicitation as a constrained optimization problem, where the goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts. Existing methods remain limited: discrete prompt-based attacks preserve semantic equivalence and coherence but search only over a limited set of prompt variations, while continuous latent-space attacks explore a richer space but often decode into prompts that are no longer valid rephrasings. To address these limitations, we propose REALISTA, a realistic latent-space attack framework. REALISTA constructs an input-dependent dictionary of valid editing directions, each corresponding to a semantically equivalent and coherent rephrasing, and optimizes continuous combinations of these directions in latent space. This design combines the optimization flexibility of continuous attacks with the semantic realism of discrete rephrasing-based attacks. Experiments demonstrate that REALISTA achieves superior or comparable performance to state-of-the-art realistic attacks on open-source LLMs and, crucially, succeeds in attacking large reasoning models under free-form response settings, where prior realistic attacks fail. Code is available at https://github.com/Buyun-Liang/REALISTA.

• The paper proposes a novel latent-space attack that preserves semantic equivalence to reliably elicit hallucinations in LLM responses.
• It leverages an input-specific edit dictionary and simplex-constrained optimization for realistic, continuous adversarial modifications.
• Results show superior attack success rates with minimal semantic errors compared to discrete baselines across multiple LLMs, including GPT-5.

REALISTA: Realistic Latent Adversarial Attacks for LLM Hallucination Elicitation

Introduction and Motivation

REALISTA proposes a new class of attacks on LLMs that are designed to elicit hallucinations using adversarial prompts which remain both semantically equivalent and coherent with respect to the original user intent. The principal problem addressed is the longstanding trade-off in LLM red teaming: discrete adversarial prompt-based attacks preserve semantic intent but are restricted to limited rephrasing candidates and offer low coverage, while continuous, latent-space attacks afford rich exploration but typically decode to prompts that are incoherent or alter meaning, thus losing semantic realism.

This limitation is particularly acute for studying LLM behavior in realistic scenarios: current continuous attacks can easily degrade the surface form or semantics of prompts, causing detection methods or safety guardrails to trivially discard them; but realistic, human-level failures often arise from subtle, semantically faithful rephrasings. REALISTA confronts this gap by constructing attacks in the LLM's latent space using input-conditioned, interpretable editing directions, allowing optimized but realistic adversarial exploration.

Methodology

Latent Space Edit Dictionary Construction

At the core of REALISTA is its construction of an input-specific dictionary of latent edit directions. For each source prompt $\bm{x}_0$, the approach:

• Computes the prompt's latent embedding $\bm{z}_0 = \phi(\bm{x}_0)$ at a selected hidden layer.
• Gathers a diverse set of semantically equivalent rephrasings of $\bm{x}_0$ via dictionary-based or automated methods, leveraging lexical sources such as WordNet.
• Encodes each rephrasing as $\bm{c}^{(i)} = \phi(\bm{x}^{(i)}_{SE})$. Each direction $\bm{z}^{(i)} = \bm{c}^{(i)} - \bm{z}_0$ defines a transformation that preserves meaning but modifies surface form.
• Forms the edit dictionary $$\bm{D}^{(\bm{z}_0)} = [\bm{z}^{(1)}, \bm{z}^{(2)}, ..., \bm{z}^{(n)}]$$.

The adversarial latent is parameterized as a non-negative, $\ell_1$-norm-constrained mixture of these edit vectors: $\bm{z} = \bm{z}_0 + \bm{D}^{(\bm{z}_0)}\bm{\delta}$, with $\bm{\delta} \in \Delta_\varepsilon$, ensuring (empirically) that edits are semantic-preserving and sparse.

Continuous Optimization and Semantic Constraints

The attack is formalized as optimization over the editing strengths $\bm{\delta}$ to maximize a hallucination loss (e.g., target answer scores), while enforcing semantic equivalence (simplex constraint on $\bm{z}_0 = \phi(\bm{x}_0)$0) and coherence (via decoding). Critically, the edit search is conducted in the latent space, yet mapped back to the prompt space using an LLM-based inversion $\bm{z}_0 = \phi(\bm{x}_0)$1. The decoder reconstructs natural language prompts from latent codes, thanks to recent advances in LLM-based self-interpretation and latent decoding [e.g., SelfIE, LARGO].

The adopted optimization algorithm—stochastic projected Langevin dynamics over the latent simplex—incorporates both gradient search (when available via surrogates) and noise injection for exploration. Safeguards ensure only semantically equivalent outputs are retained, by automatic equivalence checkers.

(Figure 1)

Figure 1: Normalized word edit distance across layers between original prompts and their reconstructions. Dots indicate the best-performing layer for each model, and shaded regions show standard deviation over 10,000 bootstrap samples. Reconstruction quality degrades as depth increases.

Selection of the latent layer is critical for ensuring that latent edits can be reliably decoded. Figure 1 shows that shallow layers permit near-perfect reconstructibility, which is essential for maintaining semantic coherence during the attack optimization.

Attack Effectiveness and Comparison

REALISTA is benchmarked on standard open-source and commercial LLMs (Llama-3, Qwen-2.5, and GPT-5 variants) under both open-ended multiple-choice question answering (MCQA) and freestyle reasoning response settings. Against baselines—including SECA (discrete, equivalence-preserving), LARGO (latent, unconstrained), and ICD (template-based)—REALISTA achieves:

• Superior or competitive attack success rates (ASR@30) across all tested models
• Low semantic equivalence error (SEE) and semantic coherence error (SCE) nearly matching the best discrete baselines
• The unique capability of attacking commercial/free-form LLMs where prior realistic attacks fail due to formatting or API access restrictions

Notably, in open-ended MCQA, REALISTA improves ASR by 10-20% on Llama-3 models compared to SECA, without increasing semantic errors. Baseline attacks that relax semantic constraints achieve slightly higher ASR, but at SEE rates exceeding 97%, meaning attacks are not realistic.

Qualitative Analysis

Analysis of activated latent concepts in adversarial prompts reveals that:

• The most frequent edits involve polarity flips ("counterfactual", "inverted"), logical structure modifications (e.g., "conditional", "disjunctive"), or instructional changes (e.g., "imperative", "elaborate"), rather than direct factual modifications.
• Attacks are typically sparse, relying on 1-2 simultaneously active concepts, indicating the sufficiency of localized, concept-orthogonal modifications for effective elicitation of hallucinations in LLMs.

Optimization traces of both the dictionary construction and attack search demonstrate rapid convergence and effective coverage of the adversarial space via continuous exploration.

Strong Numerical Results and Claims

• REALISTA achieves ASR@30 exceeding 90% on Llama-3 models, with SEE < 3.5% and SCE < 3.5%. Baseline latent attacks (LARGO, ICD) can only achieve high ASR at the cost of SEE > 96%.
• REALISTA is the only realistic attack that remains effective against GPT-5-series models in free-form response settings, achieving 20-24% ASR, where SECA and LARGO are inapplicable.
• Human and automated evaluations closely match: for REALISTA, human-assessed SEE is within 0-5% across annotators, confirming that the attacks are natural and semantically faithful.
• The method's stepwise latent optimization reliably remains within a semantic equivalence manifold, empirically supported by the low edit distances and the extensive ablation studies.

Practical and Theoretical Implications

REALISTA's construction addresses a central challenge in LLM safety: attacks must be realistic—with both semantic equivalence and coherence—if they are to inform model alignment and defense. Its latent-based continuous exploration sidesteps the combinatorial limitations of discrete prompt search, supporting richer attack coverage and deeper analysis.

This framework implies that current LLM safety benchmarks relying on fixed prompts or discrete rephrasings may underestimate model susceptibility to realistic attacks; black-box or API-limited models, previously theoretically robust, remain vulnerable to input-realistic red teaming. These findings argue for systematic inclusion of latent-based attack evaluation in LLM deployment and for adversarial training targeting latent anisotropies.

REALISTA further demonstrates the utility of conceptual-edit dictionaries and simplex constraints for preserving semantics in continuous optimization, drawing a link between adversarial robustness in vision (e.g., $\bm{z}_0 = \phi(\bm{x}_0)$2-bounded attacks) and realistic NLP adversariality. The method harnesses recent advances in self-decoding, latent steering, and concept-based interpretability, substantiating the growing consensus that LLM representations encode structured, linearized concept spaces.

Future Directions

• Extension of latent constraints to richer, possibly non-linear or perceptually aligned constraint sets, to further expand coverage of realistic adversarial triggers without sacrificing coherence.
• Integration with automated metric evaluators (e.g., semantic entropy, human-in-the-loop alignment, factual judgment LLM-as-a-judge frameworks) for scalable evaluation pipelines.
• Deployment in adversarial training or continuous model defense pipelines, particularly where white-box access is not available, and especially for commercially deployed, black-box, or instruction-locked LLMs.
• Deeper exploration of the distribution of semantic directions: e.g., are some "concepts" disproportionately leveraged for successful elicitation of hallucinations, and does this generalize across model scales/families?
• Further theoretical study of the observed phenomenon that latent simplex proximity guarantees semantic equivalence under LLM decoder mapping.

Conclusion

REALISTA introduces a principled, continuous approach to realistic adversarial attack construction for LLM hallucination elicitation. Through input-conditioned latent semantic edit dictionaries, effective simplex-constrained stochastic optimization, and latent-to-prompt decoding safeguards, the method redefines the feasible frontier of realistic LLM attack evaluation. Empirically, it achieves both superior attack success and semantic fidelity—across open/free-form and black-box models—where prior methods are either ineffective or unrealistic. This framework is expected to significantly impact safety auditing, adversarial training, and mechanistic interpretability in future large-scale LLM deployments, providing a foundation for scalable, realistic evaluation and robust alignment.

(Figure 1)

Figure 1: Normalized word edit distance across layers between original prompts and their reconstructions. Dots indicate the best-performing layer for each model, and shaded regions show standard deviation over 10,000 bootstrap samples. Reconstruction quality degrades as depth increases.