Evolving and Detecting Multi-Turn Deception using Geometric Signatures

Abstract: Safety defenses for LLMs are typically trained and evaluated on single-turn prompts, yet real attacks often unfold as indirect, multi-turn probing. To defend against this more nuanced form of deception, we present a unified pipeline that generates realistic multi-turn deceptive question sets via multi-objective genetic prompt optimization with co-evolving mutation operators. We validate this dataset through a human study, which also revealed that early generations yielded the most convincing deception and practical constraints such as adherence filtering and ordering effects. Using this data, we were able to detect deceptive attempts to access prohibited information using simple, explainable geometric signals in embedding space coupled with a lightweight feed-forward classifier. Three geometric features (angular coverage, distance ratio, and linearity) augmented with pairwise similarity statistics led to a compact predictive model that achieved consistently high recall (0.89) across base, reworded, and truncated (three-turn) scenarios, with test-time F1 ranging from 0.74-0.86. The results support a central hypothesis that multi-turn deceptive intent leaves a stable geometric footprint that enables lightweight, transparent screening without expensive end-to-end training. We further discuss responsible uses, limitations, and paths toward larger, more diverse human-evaluated datasets. The primary contribution to artificial intelligence is the multi-objective evolutionary framework for prompt generation, and the engineering application is the deployment of a lightweight geometric detection system for LLM safety infrastructure.

• The paper introduces a two-stage pipeline that evolves deceptive multi-turn prompts and detects adversarial intent using explainable geometric features.
• It employs a multi-objective genetic optimization for realistic prompt generation and extracts angular coverage, distance ratio, linearity, and pairwise similarity from embedding spaces.
• The study validates the method with human and LLM evaluations, demonstrating high recall and robustness for operational safety in LLM deployments.

Evolving and Detecting Multi-Turn Deception in LLMs via Geometric Signatures

Motivation and Context

The proliferation of LLMs has brought new challenges in conversational AI safety, particularly in detecting indirect, multi-turn adversarial attempts to elicit prohibited information. Conventional safety systems focus on single-turn prompt detection, primarily using surface-level cues or keyword filters, but fail to address sophisticated attacks spanning multiple conversational turns. The studied paper addresses this gap by proposing a unified pipeline: generation of realistic multi-turn deceptive question sets and detection of deceptive intent using explainable geometric features extracted from embedding space.

Multi-Objective Evolutionary Prompt Generation

The synthetic dataset construction leverages a multi-objective genetic optimization paradigm. Candidate prompt sets and mutation operators are co-evolved to maximize deception quality under policy constraints, specifically ensuring prompts avoid explicit prohibited terms. NSGA-II is utilized for efficient Pareto sorting given multiple objectives: deception degree and adherence to filtering. Mutation operators are LLM-based rewording functions, applied iteratively to diversify and refine candidate question sets, with evaluation metrics quantifying semantic relevance and indirectness.

Figure 1: Iterative steps for generating and optimizing synthetic data via multi-objective selection and LLM-based mutation.

Human-in-the-loop (HITL) assessment validates generated sets, revealing that early generations yield maximally deceptive queries. Over-optimization leads to less subtle deception, and prompt ordering influences perceived intent, underscoring the need for principled stopping criteria and careful human validation.

Geometric Feature-Based Detection Model

Detection eschews end-to-end neural architectures in favor of explainable geometric features. Using MiniLM-L6-v2 sentence embeddings, the pipeline computes:

• Angular coverage: Measures how the question set "surrounds" the target topic, capturing coordinated probing strategies.
• Distance ratio: Quantifies spread relative to the topic embedding, flagging attempts to approach dangerous concepts indirectly.
• Linearity: Detects whether questions follow a linear trajectory in semantic space, informing coherence.
• Pairwise similarity statistics: Further captures inter-question semantic relationships.

A compact feed-forward neural network (MLP, 3 layers) is trained on these features, yielding high interpretability and operational efficiency for deployment in safety-critical systems.

Experimental Validation

Human Subject Evaluation

A two-phase survey instrument collects perceived deceptiveness scores from recruited participants, and is paralleled by LLM (GPT-4o) assessment for surrogate evaluation. The design ensures assessment of both isolated and grouped question deceptiveness, with Likert scales quantifying perceived intent.

Figure 2: Survey interface for rating individual questions on perceived deceptiveness.

Group-level evaluation reveals that aggregating subtly related questions markedly increases perceived deceptiveness, confirming generation effectiveness and the necessity for multi-turn detectors.

Figure 3: Boxplots for the average of 5 individual questions from phase 1 and participant ratings when the 5 questions were viewed together in Phase 2. The three generations of prompt optimizing are shown as well as the control group.

Statistical analysis (Wilcoxon, Kruskal-Wallis) demonstrates significant differences between prompt generations—for instance, the first generation achieved the highest average deceptiveness gap between single and grouped question presentations.

Figure 4: Means, modes and variance of human and LLM average responses per phase.

Notably, LLMs exhibit far less response variance than humans, suggesting their limitations as substitutive evaluators in highly subjective deception detection.

Detection Performance and Robustness

The geometric detector achieves high recall (0.89) across base, reworded, and truncated scenarios (three-turn conversations), with F1 scores ranging from 0.74–0.86 on hold-out test sets. Performance is robust to lexical rewording and turn truncation, demonstrating that geometric signatures capture coordinated intent, not mere surface phrasing.

The system exhibits a cautious bias, with a higher false positive rate for benign sequences, favoring early flagging over missed detection—a necessary property for operational safety infrastructures.

Implications and Future Directions

The demonstrated pipeline achieves two main advances:

• Prompt Generation: Multi-objective prompt evolution, validated via HITL studies, provides nuanced, human-like deceptive conversational datasets, critical for safety benchmark construction and adversarial robustness testing in LLMs.
• Detection: Explainable, geometric feature-based detection avoids opaque end-to-end models, enabling transparency and real-time screening of subtle, coordinated multi-turn attacks.

Practically, this approach can be integrated into LLM safety stacks, improving moderation by screening out conversation-level deception attempts early. Theoretically, geometric analysis complements token-based approaches, providing a robust means to examine conversational patterns in embedding space.

Future directions entail:

• Expansion to larger, more diverse real-world datasets,
• Analysis of temporal and ordering effects,
• Application to broader domains (e.g., cognitive decline, emotion detection),
• Refinement of stopping criteria in prompt evolution,
• Extension of features for higher conversation turns and alternative embedding backbones.

Conclusion

This paper formalizes and validates a novel two-stage pipeline for generating and detecting multi-turn deceptive intent in LLMs. Multi-objective evolutionary prompt optimization produces realistic, human-validated datasets, and deception is reliably flagged using lightweight geometric signatures in embedding space. The method demonstrates operational efficiency, high recall, and robustness to conversational length and lexical variations. Given the dual-use risks, ethical constraints are discussed, and future work includes larger-scale evaluations and broader cross-domain applications.