---
title: Multi-Turn Text Interaction (MTI)
url: https://www.emergentmind.com/topics/multi-turn-text-interaction-mti
type: topic
---

# Multi-Turn Text Interaction (MTI)

Multi-Turn Text Interaction (MTI) denotes the capacity of a language model to maintain context across multiple dialogue turns and to use that accumulated context to produce coherent, relevant, and goal-directed responses. In recent work, MTI is treated not merely as repeated single-turn prompting, but as a distinct capability that spans instruction following, context memory, planning, multi-turn reasoning, and general conversation, and that increasingly extends to tool use, retrieval, multimodal interaction, and safety-critical monitoring [2501.09959][2504.04717].

## 1. Conceptual foundations

A central claim in the MTI literature is that multi-turn interaction introduces an additional dimension beyond single-turn instruction following. TurnWise states that multi-turn conversation is a distinct capability rather than “single-turn chat repeated several times,” and its evaluation is explicitly designed to compare multi-turn responses against equivalent single-turn settings [2603.16759]. This distinction is also reflected in surveys that separate MTI into task families such as instruction following and conversational engagement, and into enabling capabilities such as memory, planning, and reasoning [2504.04717][2501.09959].

One formalization represents a conversation as
$$
H = \{s, i_0, m_0, \ldots, i_n\},
$$
where later turns depend not only on new user inputs $i_k$ but also on earlier model outputs $m_k$ [2603.04783]. This formulation captures a defining property of MTI: the model is conditioned on its own prior behavior. In practice, this means that user clarification, constraint addition, correction, and follow-up questioning are inseparable from dialogue history.

MT-Eval operationalizes MTI by deriving four representative interaction patterns from human–LLM conversations: recollection, expansion, refinement, and follow-up [2401.16745]. These patterns collectively define a large share of the field’s working notion of MTI. Recollection requires the model to remember and apply information from earlier turns; expansion keeps the same topic while adding related sub-questions; refinement modifies or clarifies prior instructions; follow-up makes each new user query depend on the assistant’s previous answer. This typology has become a useful descriptive vocabulary for later MTI work.

## 2. Canonical interaction patterns and characteristic failure modes

MT-Eval provides one of the clearest empirical demonstrations that MTI is not well predicted by single-turn performance. The benchmark contains 168 dialogue sessions and 1,170 total turns, and compares multi-turn performance against single-turn counterparts wherever possible [2401.16745]. Its results show substantial degradation when the same information is embedded in dialogue history rather than presented directly. Concrete examples include Llama-2-chat-13B dropping from 7.55 single-turn average to 5.47 multi-turn, and Mixtral-Instruct-8x7B dropping from 8.28 to 6.78. The paper further reports that this degradation is not correlated with the models’ fundamental capabilities, so strong single-turn performance is not a reliable proxy for MTI competence.

Two failure modes recur throughout the literature. The first is distance to relevant content. MT-Eval finds that models degrade as the relevant information becomes farther away in the dialogue, and that inserting distracting turns between the document and the current query harms performance more than inserting them at the front [2401.16745]. The second is error propagation. In MT-Eval’s analysis of 200 problematic responses, 49.5% were due to noncompliance with earlier instructions, 48% to error propagation, and 2.5% to evaluation errors.

RLSTA sharpens this diagnosis by naming a specific mechanism: **Contextual Inertia**, defined as a tendency to rigidly adhere to previous reasoning traces even when later turns explicitly correct them [2603.04783]. In the paper’s MT-Add and MT-Refine settings, multi-turn errors are attributed predominantly to misleading context and propagated error, with over 70%–90% of errors falling into those two categories. This suggests that many MTI failures arise not from inability to solve the final task in principle, but from over-commitment to the model’s own earlier responses.

A related line of work frames breakdowns at the user side. Drift-Bench treats many MTI failures as **cooperative breakdowns** caused by flaws of intention, premise, parameter, or expression [2602.02455]. In this view, MTI is not only about remembering prior turns; it is also about repairing ambiguity, missing parameters, false presuppositions, and vague language through clarification before execution.

## 3. Evaluation paradigms and benchmark design

The benchmark landscape reflects a shift from short conversational probes to controlled, turn-comparable evaluation. Surveys organize MTI benchmarks around instruction following, reasoning, roleplay, healthcare, education, coding, and adversarial dialogue, while also noting that most benchmarks still focus on 10 or fewer turns [2504.04717].

| Benchmark | Setting | Distinctive property |
|---|---|---|
| MT-Eval [2401.16745] | Text-only conversational ability | 168 dialogue sessions, 1,170 total turns, four interaction patterns |
| TurnWiseEval [2603.16759] | Paired single-/multi-turn chat evaluation | Up to eight user turns; Absolute and Self comparison modes |
| TextBindEval [2309.08637] | Multi-turn multimodal instruction following | 278 conversations |
| DialogBen [2403.08857] | Multi-modal dialogue for text-to-image generation | 9957 three-turn conversations; modality switching and coherence metrics |
| TMDialog-Eva [2505.23121] | Long multi-turn multimodal dialogue | 329 handwritten samples; Long Conversation averages 8.55 turns |
| InterMT-Bench [2505.23950] | Judge and assistant evaluation in multi-turn multimodal settings | Score evaluation, pair comparison, crucial step recognition |

TurnWiseEval is especially important because it isolates multi-turn-specific conversational ability through paired comparison to equivalent single-turn prompts [2603.16759]. Its **TurnWiseEval-Absolute** mode compares multi-turn responses against GPT-4 (1106-preview) references using a length-controlled win rate judged by GPT-4.1, while **TurnWiseEval-Self** compares a model’s multi-turn response to its own single-turn response using discrete win rate. Reported gaps include Olmo 3 7B scoring 36.8 on TurnWiseEval-Absolute versus 42.2 on AlpacaEval, and Olmo 3.1 32B scoring 52.4 versus 60.1, both indicating a multi-turn deficit.

Benchmark design in multimodal MTI has emphasized turn structure and output coherence. DialogBen contains 9957 three-turn conversations and evaluates both modality switching accuracy and Generation Coherence VQA [2403.08857]. TextBindEval uses 278 human-refined conversations to test realistic multi-turn multimodal instruction following [2309.08637]. TMDialog-Eva contributes 329 handwritten samples with at least three turns, including Long Conversation and Continuous Question categories [2505.23950].

Evaluation has also become more trajectory-aware in grounded settings. Drift-Bench introduces the RISE protocol with four axes—Robustness, Intelligence, Safety, and Efficiency—and formalizes metrics such as performance degradation, clarification gain, Safe Action Rate, and Average Interaction Rounds [2602.02455]. In multi-turn composed image retrieval, CIRCLED evaluates not only final retrieval quality but also turn-wise convergence using Hits@10, Final Recall@10, and AUC over Hits@10 across 22,608 sessions [2605.26734]. This suggests that MTI evaluation is increasingly concerned with *how* performance evolves over turns, not only whether the final answer is correct.

## 4. Modeling strategies for robust MTI

One major modeling strategy is to make conversational state explicit rather than leaving it implicit in prompt concatenation. DELTA does this by decoupling multi-turn Text-to-SQL into two stages: an utterance rewrite model first converts the current contextual utterance into a semantic-completion utterance, and a single-turn parser then maps the rewritten utterance to SQL [2106.02282]. The rewrite model is trained with BART, the parser is RATSQL, and dual learning is used to address rewrite-data sparsity. This design makes context completion interpretable and separates dialogue-history resolution from downstream parsing.

IST-SQL takes a different explicit-state approach by tracking two kinds of interaction states derived from the last predicted SQL query: **schema-states** and **SQL-states** [2012.04995]. Schema-states track which SQL keywords are associated with each schema item; SQL-states track which schema items are associated with each SQL keyword. A relational graph neural network updates schema-state representations, and a non-linear layer updates SQL-state representations. On CoSQL, the full model improves question match and interaction match over the corresponding baseline, and ablations show that removing either state type substantially reduces performance.

A second modeling strategy is explicit memory compression and retrieval. ContextQFormer introduces a memory block that stores turn-level [CLS] representations—ViT [CLS] for images and RoBERTa [CLS] for text—in a queue, then retrieves relevant historical information through learnable queries and cross-attention [2505.23121]. On TMDialog, this context-modeling module improves available rate by about 2%–4% over baselines, with the reported available rate reaching 68.17% versus 66.40% for mPLUG-owl, 62.46% for VisualGLM, and 64.01% for LoRA-only. A plausible implication is that compressed turn-level memory can be more effective for long MTI than raw prompt growth.

A third strategy is to train directly on multi-turn data or on objectives that preserve turn-wise adaptability. TurnWiseData generates conversations of 2 to 8 user turns from single-turn prompts, and experiments with Olmo 3 show that including as little as 10k multi-turn conversations during post-training can lead to a 12% improvement on TurnWiseEval [2603.16759]. The paper also reports that preference tuning with multi-turn data tends to preserve single-turn performance better than supervised fine-tuning.

RLSTA addresses MTI collapse by exploiting the model’s own stronger single-turn solution as a **Single-Turn Anchor** [2603.04783]. It filters training examples to retain cases where the model succeeds in single-turn form but fails in multi-turn form, then uses GRPO with a reward that combines outcome correctness and anchor alignment. Reported gains include Qwen2.5-3B-Instruct improving from 0.493 to 0.715 on MT-Add math, and Qwen3-4B-Instruct-2507 improving from 0.772 to 0.903. The paper further reports cross-domain transfer from math to code and effectiveness without external verifiers. This suggests that MTI training can benefit from anchoring multi-turn behavior to stable single-turn competence.

## 5. Tool-integrated, multimodal, and retrieval-oriented MTI

Tool use turns MTI into a closed-loop interaction with an external environment. Interactive-T2S formalizes Text-to-SQL as a repeated thought–action–observation loop in which the LLM chooses among four tools—SearchColumn, SearchValue, FindShortestPath, ExecuteSQL—and updates its reasoning after each observation [2408.11062]. The method is designed for large and schema-heavy databases and reports 54.56 EX on BIRD-Dev without oracle knowledge, while using only two full-process exemplars. MTIR-SQL extends this execution-aware paradigm with Multi-turn Tool-Integrated Reasoning and GRPO-Filter, interleaving SQL generation with database execution feedback at each step. The reported headline results are 64.4% on BIRD Dev and 84.6% execution accuracy on SPIDER Dev with a 4B model [2510.25510].

Multimodal MTI has developed rapidly around interleaved image–text conversation. TextBind generates multi-turn multimodal instruction-response conversations from image-caption pairs in an almost annotation-free way, producing 25,629 conversations with an average of 3.36 turns, 2.46 images, and 285.90 words per conversation [2309.08637]. Its accompanying MIM architecture is language-model-centric and supports arbitrarily interleaved image-text input and output; among the three image-generation variants explored, the **Language Description as Medium** variant performs best. DialogGen addresses a related problem in multi-turn text-to-image generation by aligning an off-the-shelf MLLM with an off-the-shelf T2I model through drawing prompt alignment, careful training data curation, and error correction [2403.08857]. On DialogBen, DialogGen achieves 97.24% average modality-switching accuracy, and DialogGen-X reaches 0.6514 on Coherence VQA and 0.7559 in human evaluation.

MTI has also been extended to longer-context multimodal dialogue and motion-language interaction. TMDialog contributes over 1.5 million instruction-tuning samples and an evaluation set whose Long Conversation category averages 8.55 turns [2505.23121]. VIM pairs a unified motion-language model with Inter-MT2, a dataset containing 82.7K multi-turn interactive motion instructions and 153K interactive motion samples, with 4 to 8 turns per sample [2410.05628]. In both cases, the multi-turn setting is not merely conversational packaging; it defines the control interface for refinement, editing, reasoning, and grounded generation.

Retrieval-oriented MTI uses interaction to progressively sharpen an information need. In text-based person search, CTGI introduces an inference-time MTI module that identifies an anchor image among top-$K$ retrieval candidates, asks diagnostic questions, synthesizes a refined query, and reranks with
$$
\hat{S}_{q,v} = \lambda S_{q,v} + (1-\lambda)S_{\hat{q},v}.
$$
On RSTPReid, MTI alone improves over MTG-only baselines, and MTG + MTI gives the best reported results for both IRRA and RDE backbones [2509.12662]. In multi-turn composed image retrieval, CIRCLED constructs 22,608 sessions across nine subsets, enforces turn-wise progress toward the target image, and extends evaluation beyond fashion into general domains [2605.26734].

## 6. Safety, alignment, and open problems

Safety research has increasingly reframed MTI as a trajectory-level control problem. TurnGate treats hidden malicious intent as distributed across multiple benign-looking turns and formalizes the defender’s observation at turn $t$ as
$$
x_t = \left((q_1,r_1), \ldots, (q_{t-1},r_{t-1}), q_t, \tilde r_t\right),
$$
where $\tilde r_t$ is the candidate response before delivery [2605.05630]. It defines the **earliest harm-enabling turn**
$$
t^*(\tau,g)=\min\bigl(\{t : \mathrm{Suff}(x_t,g)=1\}\cup\{\infty\}\bigr),
$$
and trains a response-aware, turn-level monitor on MTID, which contains 8,000 harmful and 8,000 benign dialogues. Compared to Reweighted-SFT, TurnGate improves Harmful Score from 0.479 to 0.602, raises exact-turn accuracy from 34.3% to 41.4%, and reduces miss rate from 37.9% to 17.7. In online evaluation it yields the lowest attack success rate, at about 0.26 for attacker iteration budget $i=1$ and staying below about 0.68 even at $i=5$.

STAR advances a complementary state-space view of multi-turn safety failure [2603.15684]. Rather than treating dialogue history as a passive transcript, it models history as a state-transition operator and studies how aligned models cross a safety boundary under structured trajectories. Reported Safety Failure Rates under STAR reach 94.5% for GPT-4o, 74.0% for Claude 3.5 Sonnet, 96.1% for Gemini 2.0-Flash, 89.0% for LLaMA-3-8B-IT, and 85.5% for LLaMA-3-70B-IT. Mechanistic analysis further reports monotonic drift away from refusal-related representations and abrupt phase transitions induced by role-conditioned context. This suggests that MTI safety is fundamentally dynamic and state-dependent.

Drift-Bench shows that failure is not limited to adversarial intent; it also arises when user inputs violate cooperative assumptions [2602.02455]. The benchmark reports around a 40% drop overall in some settings under cooperative breakdowns and identifies a **Clarification Paradox**: clarification often helps in state-oriented, white-box environments but can hurt in service-oriented, black-box environments because of context overload, parsing fragility, and clarification-induced collapse. The same work reports that agents often act prematurely rather than defer for clarification, in roughly 70% of cases in some high-risk scenarios.

Alignment work has begun to incorporate human preference directly over multi-turn trajectories. InterMT is presented as the first human-preference dataset for multi-turn multimodal interaction, with 15,604 seed prompts, 52.6k multi-turn dialogue instances, and 32,459 human-labeled preference annotations across nine sub-dimensions spanning local and global judgments [2505.23950]. The paper emphasizes human oversight, including professional quality control review and researcher auditing of 20% of the data, and reports that AI annotators achieve only about 60% agreement locally and perform much worse globally. This finding aligns with survey conclusions that MTI still requires better context management, stronger reasoning across turns, more realistic training data, calibrated LLM-based evaluators, and more robust safety and fairness analysis [2504.04717][2501.09959].

Source: https://www.emergentmind.com/topics/multi-turn-text-interaction-mti