Multi-Turn Text Interaction (MTI)

• Multi-Turn Text Interaction is defined as a model’s capacity to sustain and utilize accumulated dialogue context for producing coherent, goal-directed responses.
• Evaluation benchmarks like MT-Eval and TurnWiseEval reveal that multi-turn performance often lags behind single-turn performance, highlighting practical challenges in context retention.
• Robust MTI systems leverage explicit state management, memory compression, and turn-wise training to mitigate error propagation and contextual inertia.

Multi-Turn Text Interaction (MTI) denotes the capacity of a LLM to maintain context across multiple dialogue turns and to use that accumulated context to produce coherent, relevant, and goal-directed responses. In recent work, MTI is treated not merely as repeated single-turn prompting, but as a distinct capability that spans instruction following, context memory, planning, multi-turn reasoning, and general conversation, and that increasingly extends to tool use, retrieval, multimodal interaction, and safety-critical monitoring (Zhang et al., 17 Jan 2025, Li et al., 7 Apr 2025).

1. Conceptual foundations

A central claim in the MTI literature is that multi-turn interaction introduces an additional dimension beyond single-turn instruction following. TurnWise states that multi-turn conversation is a distinct capability rather than “single-turn chat repeated several times,” and its evaluation is explicitly designed to compare multi-turn responses against equivalent single-turn settings (Graf et al., 17 Mar 2026). This distinction is also reflected in surveys that separate MTI into task families such as instruction following and conversational engagement, and into enabling capabilities such as memory, planning, and reasoning (Li et al., 7 Apr 2025, Zhang et al., 17 Jan 2025).

One formalization represents a conversation as

$H = \{s, i_0, m_0, \ldots, i_n\},$

where later turns depend not only on new user inputs $i_k$ but also on earlier model outputs $m_k$ (Chen et al., 5 Mar 2026). This formulation captures a defining property of MTI: the model is conditioned on its own prior behavior. In practice, this means that user clarification, constraint addition, correction, and follow-up questioning are inseparable from dialogue history.

MT-Eval operationalizes MTI by deriving four representative interaction patterns from human–LLM conversations: recollection, expansion, refinement, and follow-up (Kwan et al., 2024). These patterns collectively define a large share of the field’s working notion of MTI. Recollection requires the model to remember and apply information from earlier turns; expansion keeps the same topic while adding related sub-questions; refinement modifies or clarifies prior instructions; follow-up makes each new user query depend on the assistant’s previous answer. This typology has become a useful descriptive vocabulary for later MTI work.

2. Canonical interaction patterns and characteristic failure modes

MT-Eval provides one of the clearest empirical demonstrations that MTI is not well predicted by single-turn performance. The benchmark contains 168 dialogue sessions and 1,170 total turns, and compares multi-turn performance against single-turn counterparts wherever possible (Kwan et al., 2024). Its results show substantial degradation when the same information is embedded in dialogue history rather than presented directly. Concrete examples include Llama-2-chat-13B dropping from 7.55 single-turn average to 5.47 multi-turn, and Mixtral-Instruct-8x7B dropping from 8.28 to 6.78. The paper further reports that this degradation is not correlated with the models’ fundamental capabilities, so strong single-turn performance is not a reliable proxy for MTI competence.

Two failure modes recur throughout the literature. The first is distance to relevant content. MT-Eval finds that models degrade as the relevant information becomes farther away in the dialogue, and that inserting distracting turns between the document and the current query harms performance more than inserting them at the front (Kwan et al., 2024). The second is error propagation. In MT-Eval’s analysis of 200 problematic responses, 49.5% were due to noncompliance with earlier instructions, 48% to error propagation, and 2.5% to evaluation errors.

RLSTA sharpens this diagnosis by naming a specific mechanism: Contextual Inertia, defined as a tendency to rigidly adhere to previous reasoning traces even when later turns explicitly correct them (Chen et al., 5 Mar 2026). In the paper’s MT-Add and MT-Refine settings, multi-turn errors are attributed predominantly to misleading context and propagated error, with over 70%–90% of errors falling into those two categories. This suggests that many MTI failures arise not from inability to solve the final task in principle, but from over-commitment to the model’s own earlier responses.

A related line of work frames breakdowns at the user side. Drift-Bench treats many MTI failures as cooperative breakdowns caused by flaws of intention, premise, parameter, or expression (Bao et al., 2 Feb 2026). In this view, MTI is not only about remembering prior turns; it is also about repairing ambiguity, missing parameters, false presuppositions, and vague language through clarification before execution.

3. Evaluation paradigms and benchmark design

The benchmark landscape reflects a shift from short conversational probes to controlled, turn-comparable evaluation. Surveys organize MTI benchmarks around instruction following, reasoning, roleplay, healthcare, education, coding, and adversarial dialogue, while also noting that most benchmarks still focus on 10 or fewer turns (Li et al., 7 Apr 2025).

Benchmark	Setting	Distinctive property
MT-Eval (Kwan et al., 2024)	Text-only conversational ability	168 dialogue sessions, 1,170 total turns, four interaction patterns
TurnWiseEval (Graf et al., 17 Mar 2026)	Paired single-/multi-turn chat evaluation	Up to eight user turns; Absolute and Self comparison modes
TextBindEval (Li et al., 2023)	Multi-turn multimodal instruction following	278 conversations
DialogBen (Huang et al., 2024)	Multi-modal dialogue for text-to-image generation	9957 three-turn conversations; modality switching and coherence metrics
TMDialog-Eva (Lei et al., 29 May 2025)	Long multi-turn multimodal dialogue	329 handwritten samples; Long Conversation averages 8.55 turns
InterMT-Bench (Chen et al., 29 May 2025)	Judge and assistant evaluation in multi-turn multimodal settings	Score evaluation, pair comparison, crucial step recognition

TurnWiseEval is especially important because it isolates multi-turn-specific conversational ability through paired comparison to equivalent single-turn prompts (Graf et al., 17 Mar 2026). Its TurnWiseEval-Absolute mode compares multi-turn responses against GPT-4 (1106-preview) references using a length-controlled win rate judged by GPT-4.1, while TurnWiseEval-Self compares a model’s multi-turn response to its own single-turn response using discrete win rate. Reported gaps include Olmo 3 7B scoring 36.8 on TurnWiseEval-Absolute versus 42.2 on AlpacaEval, and Olmo 3.1 32B scoring 52.4 versus 60.1, both indicating a multi-turn deficit.

Benchmark design in multimodal MTI has emphasized turn structure and output coherence. DialogBen contains 9957 three-turn conversations and evaluates both modality switching accuracy and Generation Coherence VQA (Huang et al., 2024). TextBindEval uses 278 human-refined conversations to test realistic multi-turn multimodal instruction following (Li et al., 2023). TMDialog-Eva contributes 329 handwritten samples with at least three turns, including Long Conversation and Continuous Question categories (Chen et al., 29 May 2025).

Evaluation has also become more trajectory-aware in grounded settings. Drift-Bench introduces the RISE protocol with four axes—Robustness, Intelligence, Safety, and Efficiency—and formalizes metrics such as performance degradation, clarification gain, Safe Action Rate, and Average Interaction Rounds (Bao et al., 2 Feb 2026). In multi-turn composed image retrieval, CIRCLED evaluates not only final retrieval quality but also turn-wise convergence using Hits@10, Final Recall@10, and AUC over Hits@10 across 22,608 sessions (Takeda et al., 26 May 2026). This suggests that MTI evaluation is increasingly concerned with how performance evolves over turns, not only whether the final answer is correct.

4. Modeling strategies for robust MTI

One major modeling strategy is to make conversational state explicit rather than leaving it implicit in prompt concatenation. DELTA does this by decoupling multi-turn Text-to-SQL into two stages: an utterance rewrite model first converts the current contextual utterance into a semantic-completion utterance, and a single-turn parser then maps the rewritten utterance to SQL (Chen et al., 2021). The rewrite model is trained with BART, the parser is RATSQL, and dual learning is used to address rewrite-data sparsity. This design makes context completion interpretable and separates dialogue-history resolution from downstream parsing.

IST-SQL takes a different explicit-state approach by tracking two kinds of interaction states derived from the last predicted SQL query: schema-states and SQL-states (Wang et al., 2020). Schema-states track which SQL keywords are associated with each schema item; SQL-states track which schema items are associated with each SQL keyword. A relational graph neural network updates schema-state representations, and a non-linear layer updates SQL-state representations. On CoSQL, the full model improves question match and interaction match over the corresponding baseline, and ablations show that removing either state type substantially reduces performance.

A second modeling strategy is explicit memory compression and retrieval. ContextQFormer introduces a memory block that stores turn-level [CLS] representations—ViT [CLS] for images and RoBERTa [CLS] for text—in a queue, then retrieves relevant historical information through learnable queries and cross-attention (Lei et al., 29 May 2025). On TMDialog, this context-modeling module improves available rate by about 2%–4% over baselines, with the reported available rate reaching 68.17% versus 66.40% for mPLUG-owl, 62.46% for VisualGLM, and 64.01% for LoRA-only. A plausible implication is that compressed turn-level memory can be more effective for long MTI than raw prompt growth.

A third strategy is to train directly on multi-turn data or on objectives that preserve turn-wise adaptability. TurnWiseData generates conversations of 2 to 8 user turns from single-turn prompts, and experiments with Olmo 3 show that including as little as 10k multi-turn conversations during post-training can lead to a 12% improvement on TurnWiseEval (Graf et al., 17 Mar 2026). The paper also reports that preference tuning with multi-turn data tends to preserve single-turn performance better than supervised fine-tuning.

RLSTA addresses MTI collapse by exploiting the model’s own stronger single-turn solution as a Single-Turn Anchor (Chen et al., 5 Mar 2026). It filters training examples to retain cases where the model succeeds in single-turn form but fails in multi-turn form, then uses GRPO with a reward that combines outcome correctness and anchor alignment. Reported gains include Qwen2.5-3B-Instruct improving from 0.493 to 0.715 on MT-Add math, and Qwen3-4B-Instruct-2507 improving from 0.772 to 0.903. The paper further reports cross-domain transfer from math to code and effectiveness without external verifiers. This suggests that MTI training can benefit from anchoring multi-turn behavior to stable single-turn competence.

5. Tool-integrated, multimodal, and retrieval-oriented MTI

Tool use turns MTI into a closed-loop interaction with an external environment. Interactive-T2S formalizes Text-to-SQL as a repeated thought–action–observation loop in which the LLM chooses among four tools—SearchColumn, SearchValue, FindShortestPath, ExecuteSQL—and updates its reasoning after each observation (Xiong et al., 2024). The method is designed for large and schema-heavy databases and reports 54.56 EX on BIRD-Dev without oracle knowledge, while using only two full-process exemplars. MTIR-SQL extends this execution-aware paradigm with Multi-turn Tool-Integrated Reasoning and GRPO-Filter, interleaving SQL generation with database execution feedback at each step. The reported headline results are 64.4% on BIRD Dev and 84.6% execution accuracy on SPIDER Dev with a 4B model (Xu et al., 29 Oct 2025).

Multimodal MTI has developed rapidly around interleaved image–text conversation. TextBind generates multi-turn multimodal instruction-response conversations from image-caption pairs in an almost annotation-free way, producing 25,629 conversations with an average of 3.36 turns, 2.46 images, and 285.90 words per conversation (Li et al., 2023). Its accompanying MIM architecture is language-model-centric and supports arbitrarily interleaved image-text input and output; among the three image-generation variants explored, the Language Description as Medium variant performs best. DialogGen addresses a related problem in multi-turn text-to-image generation by aligning an off-the-shelf MLLM with an off-the-shelf T2I model through drawing prompt alignment, careful training data curation, and error correction (Huang et al., 2024). On DialogBen, DialogGen achieves 97.24% average modality-switching accuracy, and DialogGen-X reaches 0.6514 on Coherence VQA and 0.7559 in human evaluation.

MTI has also been extended to longer-context multimodal dialogue and motion-language interaction. TMDialog contributes over 1.5 million instruction-tuning samples and an evaluation set whose Long Conversation category averages 8.55 turns (Lei et al., 29 May 2025). VIM pairs a unified motion-LLM with Inter-MT2, a dataset containing 82.7K multi-turn interactive motion instructions and 153K interactive motion samples, with 4 to 8 turns per sample (Park et al., 2024). In both cases, the multi-turn setting is not merely conversational packaging; it defines the control interface for refinement, editing, reasoning, and grounded generation.

Retrieval-oriented MTI uses interaction to progressively sharpen an information need. In text-based person search, CTGI introduces an inference-time MTI module that identifies an anchor image among top-$K$ retrieval candidates, asks diagnostic questions, synthesizes a refined query, and reranks with

$$\hat{S}_{q,v} = \lambda S_{q,v} + (1-\lambda)S_{\hat{q},v}.$$

On RSTPReid, MTI alone improves over MTG-only baselines, and MTG + MTI gives the best reported results for both IRRA and RDE backbones (Xie et al., 16 Sep 2025). In multi-turn composed image retrieval, CIRCLED constructs 22,608 sessions across nine subsets, enforces turn-wise progress toward the target image, and extends evaluation beyond fashion into general domains (Takeda et al., 26 May 2026).

6. Safety, alignment, and open problems

Safety research has increasingly reframed MTI as a trajectory-level control problem. TurnGate treats hidden malicious intent as distributed across multiple benign-looking turns and formalizes the defender’s observation at turn $t$ as

$$x_t = \left((q_1,r_1), \ldots, (q_{t-1},r_{t-1}), q_t, \tilde r_t\right),$$

where $\tilde r_t$ is the candidate response before delivery (Shen et al., 7 May 2026). It defines the earliest harm-enabling turn

$$t^*(\tau,g)=\min\bigl(\{t : \mathrm{Suff}(x_t,g)=1\}\cup\{\infty\}\bigr),$$

and trains a response-aware, turn-level monitor on MTID, which contains 8,000 harmful and 8,000 benign dialogues. Compared to Reweighted-SFT, TurnGate improves Harmful Score from 0.479 to 0.602, raises exact-turn accuracy from 34.3% to 41.4%, and reduces miss rate from 37.9% to 17.7. In online evaluation it yields the lowest attack success rate, at about 0.26 for attacker iteration budget $i=1$ and staying below about 0.68 even at $i_k$0.

STAR advances a complementary state-space view of multi-turn safety failure (Li et al., 15 Mar 2026). Rather than treating dialogue history as a passive transcript, it models history as a state-transition operator and studies how aligned models cross a safety boundary under structured trajectories. Reported Safety Failure Rates under STAR reach 94.5% for GPT-4o, 74.0% for Claude 3.5 Sonnet, 96.1% for Gemini 2.0-Flash, 89.0% for LLaMA-3-8B-IT, and 85.5% for LLaMA-3-70B-IT. Mechanistic analysis further reports monotonic drift away from refusal-related representations and abrupt phase transitions induced by role-conditioned context. This suggests that MTI safety is fundamentally dynamic and state-dependent.

Drift-Bench shows that failure is not limited to adversarial intent; it also arises when user inputs violate cooperative assumptions (Bao et al., 2 Feb 2026). The benchmark reports around a 40% drop overall in some settings under cooperative breakdowns and identifies a Clarification Paradox: clarification often helps in state-oriented, white-box environments but can hurt in service-oriented, black-box environments because of context overload, parsing fragility, and clarification-induced collapse. The same work reports that agents often act prematurely rather than defer for clarification, in roughly 70% of cases in some high-risk scenarios.

Alignment work has begun to incorporate human preference directly over multi-turn trajectories. InterMT is presented as the first human-preference dataset for multi-turn multimodal interaction, with 15,604 seed prompts, 52.6k multi-turn dialogue instances, and 32,459 human-labeled preference annotations across nine sub-dimensions spanning local and global judgments (Chen et al., 29 May 2025). The paper emphasizes human oversight, including professional quality control review and researcher auditing of 20% of the data, and reports that AI annotators achieve only about 60% agreement locally and perform much worse globally. This finding aligns with survey conclusions that MTI still requires better context management, stronger reasoning across turns, more realistic training data, calibrated LLM-based evaluators, and more robust safety and fairness analysis (Li et al., 7 Apr 2025, Zhang et al., 17 Jan 2025).