---
title: Multi-Turn Intent Dataset (MTID)
url: https://www.emergentmind.com/topics/multi-turn-intent-dataset-mtid
type: topic
---

# Multi-Turn Intent Dataset (MTID)

Searching arXiv for the referenced MTID-related papers to ground the article in current literature.
A **Multi-Turn Intent Dataset (MTID)** is a dataset family in which the basic object of analysis is not an isolated utterance but a temporally ordered interaction whose intent must be inferred from context, evolution, and, in some cases, grounded execution or multimodal evidence. Recent work uses the concept in several closely related ways: as multi-turn safety corpora that track adversarial intent drift and earliest harm-enabling turns, as execution-grounded agent trajectories driven by structured user intents, as multilingual customer-service sessions labeled for final-turn or turn-level intent, as multimodal conversation benchmarks with out-of-scope or deceptive intent labels, and as next-query corpora in which intent is represented implicitly by the user’s forthcoming turn [2602.16935], [2605.05630], [2606.11520], [2411.12307], [2403.10943], [2605.23668], [2604.12700].

## 1. Conceptual Scope

Several recent papers do not introduce a dataset literally named “MTID,” but effectively define or operate on such a resource. In DeepContext, the term is used conceptually for a large proprietary corpus of **437,058 conversational sequences** targeting **multi-turn adversarial intent drift**, plus a **1,010-conversation** evaluation benchmark with **210 multiturn jailbreaks** and **800 benign conversations** [2602.16935]. In TurnGate, **MTID** is an explicit dataset name and denotes **branching attack rollouts, matched benign hard negatives, and annotations of the earliest harm-enabling turns** in chemistry and cybersecurity dialogues [2605.05630]. In ISETrace, the same concept appears as a dataset of execution-grounded OS-agent trajectories built from **43,956 unique structured intents** and **23,132 complete trajectories** [2606.11520]. In multilingual e-commerce intent classification, both LARA and C-LARA operate on proprietary multi-turn customer-service sessions whose final query is labeled from a large intent taxonomy, while MINT-E provides a synthetic multilingual multi-turn e-commerce corpus with **381** intents [2403.16504], [2411.12307], [2411.14252].

Across these works, “intent” is not uniform. In some datasets it is a binary safety variable such as **safe vs unsafe** or **benign vs malicious** [2602.16935], [2605.05630]. In others it is a structured task state, such as a schema-grounded dialogue state in MultiWOZ or SGD, a role- and task-conditioned OS-agent objective, or a canonical customer-service label from a hierarchical taxonomy [2606.11520], [2510.17940], [2411.12307]. In still others it is implicit: NQP-Bench treats the ground-truth **next user query** as the operational realization of evolving intent, while MISID models hidden strategic intent through deception, role, and fact-grounded causal reasoning over long multimodal interactions [2605.23668], [2604.12700].

This breadth suggests that MTID is best understood as a *dataset design principle* rather than a single benchmark name. The common denominator is that intent is temporally distributed and cannot be reliably recovered from a single utterance alone.

## 2. Canonical Data Structures and Supervision

A recurrent formalization treats a conversation as an ordered sequence of user and assistant turns. DeepContext writes the history at turn \(t\) as
\[
\mathcal{H}_t = \{(u_1, r_1), (u_2, r_2), \dots, u_t\},
\]
while ISETrace defines a trajectory as
\[
\tau = \{(u_t, a_t, e_t)\}_{t=1}^{T},
\]
where \(e_t\) is environment feedback from real tool execution [2602.16935], [2606.11520]. LARA and related MTIC work represent a session as
\[
\mathcal{Q} = \{q_i\}_{i=1}^n,
\]
with context
\[
\mathcal{C} = \{q_i\}_{i=1}^{n-1},
\]
and the prediction target is the intent of the final query \(q_n\) [2403.16504], [2411.12307]. OnePred instead uses
\[
\mathcal{C}_T = \{(q_t, r_t)\}_{t=1}^{T}
\]
and predicts \(q_{T+1}\), so the label is not a predefined intent class but the next free-form query itself [2605.23668].

These structural choices induce different supervision granularities.

| MTID form | Representative resource | Supervision |
|---|---|---|
| Safety trajectory | DeepContext, TurnGate MTID | Binary safety or earliest harm-enabling turn |
| Intent-class session | LARA, C-LARA, MINT-E | Final-turn or per-turn intent labels |
| Execution-grounded trajectory | ISETrace | Structured intent metadata, tool outcomes, completion |
| Multimodal conversational intent | MIntRec2.0, MISID | Intent/OOS or deception-role-causal labels |
| Implicit intent trajectory | NQP-Bench | Ground-truth next user query |

The label schema varies accordingly. DeepContext trains a **binary safety classification** objective with per-turn predictions
\[
P(\text{harm}_k = 1 \mid \mathcal{H}_t) = \sigma(\mathrm{MLP}_k(R_t)),
\]
and a **BCE with Focal Loss**
\[
\mathcal{L} = - (1 - p_t)^\gamma \log(p_t), \quad \gamma = 2.0
\]
[2602.16935]. TurnGate instead annotates the earliest turn
\[
t^*(\tau, g) = \min\bigl(\{t : \mathrm{Suff}(x_t,g)=1\} \cup \{\infty\}\bigr),
\]
which turns MTID into a turn-level intervention benchmark [2605.05630]. MIntRec2.0 uses **30** in-scope intent classes plus an **OOS** label over **15,040** utterances in **1,245** dialogues, while MISID uses a two-tier scheme covering role, emotion, subjectivity, deception, modality inconsistency, and hidden-intent reasoning over interactions of **154–555 turns** [2403.10943], [2604.12700].

A central distinction is whether supervision is conversation-level, turn-level, or trajectory-level. Datasets such as LARA’s multi-turn test sets label only the **last query** in a session, whereas DeepContext, TurnGate, and ISETrace require temporal supervision that is meaningful at intermediate turns [2403.16504], [2602.16935], [2605.05630], [2606.11520].

## 3. Construction Pipelines

MTID construction now spans at least four major paradigms: aggregation of heterogeneous safety corpora, simulation from structured intents, pseudo-labeling from industrial logs, and multimodal annotation over naturally occurring interaction streams.

DeepContext constructs a proprietary training corpus of **437,058 conversational sequences**, with approximately **20%** labeled malicious, by combining benign interactions, malicious exploits, and both single-turn and multi-turn trajectories. Public ingredients include **LLMail**, **HH-RLHF**, **XGuard**, **PRODIGy**, **MultiWOZ**, **WikiQA**, **SafeDial**, **CoSafe**, and **DEF CON**, then augmenting them with synthetic examples [2602.16935]. TurnGate’s MTID uses a different pipeline: starting from **400 harmful** and **400 benign** seeds, it runs up to **20** rollouts per seed to produce **8,000 harmful** and **8,000 benign** dialogues, with seed-level **70% / 15% / 15%** train/validation/test splits and post-verification of the closure turn \(t^*\) [2605.05630].

ISETrace begins from a structured intent space rather than existing dialogues. Its **Intent \(\rightarrow\) Simulate \(\rightarrow\) Execute** pipeline first samples roughly **50,000** intents from a 4D space over persona, domains, tasks, and complexity, deduplicates to **43,956 unique structured intents**, then uses a role-locked user simulator and live OS execution to retain **23,132 complete trajectories** averaging **8.12 user turns**, **68.24 total dialogue turns**, and **29.26 tool calls** [2606.11520]. The resulting data is not merely conversational but execution-grounded, with authentic failure-recovery behavior.

Industrial MTIC work relies more heavily on pseudo-labeling. C-LARA samples **70k online chat logs**, applies LLM labeling with retrieval-augmented prompting and a three-way self-consistency filter, discards about **12%** inconsistent sessions, and retains roughly **61k pseudo-labeled multi-turn sessions** per experiment setting [2411.12307]. Chain-of-Intent and MINT-E take another route: they estimate intent transition statistics from real logs, then combine HMM-based intent chains with LLM self-play to generate **20k sessions**, **44k questions**, and **381** intents in a multilingual e-commerce corpus [2411.14252].

Interactive safety benchmarks use yet another strategy. CarryOnBench starts from **398** seemingly harmful queries with benign underlying intents and generates **5,970 conversations** with **1,866 different conversation flows** of **4–12 turns**, while MISID annotates **3,962 utterances** from strategic deception games totaling **9.15 hours** and averaging **374.7** turns per game [2604.27093], [2604.12700]. MIntRec2.0, by contrast, builds a multimodal multi-party benchmark from television dialogue, with **1,245 dialogues**, **15,040 samples**, **9,304** in-scope utterances, and **5,736** OOS utterances [2403.10943].

These pipelines reveal a methodological split. Some MTIDs are primarily *curated mixtures* of existing resources; others are *synthetic but structured*; still others are *execution-grounded* or *multimodal observational corpora*. This suggests that “MTID” denotes a family of data-generation problems rather than a single annotation recipe.

## 4. Tasks and Modeling Paradigms Enabled by MTIDs

MTIDs support several distinct learning problems. In safety monitoring, DeepContext models intent as a recurrent latent state:
\[
h_t = \text{RNN}(h_{t-1}, e_t), \qquad
R_t = [\phi(h_t); e_t], \qquad
y_t = \text{MLP}(R_t),
\]
so that multi-turn risk is inferred from both the current turn and accumulated history [2602.16935]. TurnGate converts the same general problem into a turn-level stopping policy over
\[
x_t = (h_{t-1}, q_t, \tilde{r}_t),
\]
where the model must decide **Pass** or **Block** relative to the annotated closure point \(t^*\) [2605.05630].

In multilingual intent classification, LARA, C-LARA, and related work formulate MTIC as predicting the intent of the final query \(q_n\) from the context \(\mathcal{C}\), often with hierarchical labels and large taxonomies ranging from **237** to **481** intents per market [2403.16504], [2411.12307]. Caro extends this setup to OOD detection with multi-turn context, treating each sample as \(x=(h,u)\) and learning robust representations via a multi-view information bottleneck [2305.03237]. MINT-CL further combines hierarchical intent classification with response-ranking supervision, optimizing
\[
\mathcal{L} = \mathcal{L}_{\text{intent}} + \lambda \mathcal{L}_{\text{contrastive}}, \quad \lambda=0.3
\]
on synthetic multilingual multi-turn e-commerce data [2411.14252].

Execution-grounded agent datasets define intent differently. ISETrace treats a trajectory as the realization of a structured user objective through user turns, agent actions, and environment feedback, making the main task not classification but successful long-horizon execution under changing dialogue context [2606.11520]. IntentDial represents yet another paradigm: multi-turn dialogue is mapped to a path in a manually constructed intent graph, and intent identification becomes reinforcement learning over graph states \(s_t=(e_t,e_r,c_i)\) conditioned on a context embedding \(c_i\) [2310.11818].

Some MTIDs make intent implicit. OnePred’s NQP-Bench predicts the next user query \(q_{T+1}\) from \(\mathcal{C}_T\), arguing that the model must capture the user’s “evolving intent trajectory across topics, unresolved needs, and interest shifts.” Its recursive intent memory updates
\[
m_t,\; \hat{q}_t^{(1)}, \ldots, \hat{q}_t^{(N)} = f_\theta(m_{t-1}, o_t)
\]
and thereby compresses multi-turn history into a bounded latent “intent chain” [2605.23668]. The LiC work on intent mismatch provides a complementary theoretical lens, decomposing response quality into **task execution** and **intent inference**:
\[
P_\theta(R \mid C_t) \approx \sum_{I_t \in \mathcal{I}} P_\theta(R \mid I_t)\, P_\theta(I_t \mid C_t),
\]
and motivating a Mediator-Assistant architecture that rewrites ambiguous histories into explicit instructions \(\hat{U} \approx I_t\) before execution [2602.07338].

Multimodal MTIDs broaden the modeling space further. MIntRec2.0 supports multimodal fusion and OOS detection at the utterance level within multi-party conversation [2403.10943]. MISID adds long-context multimodal fact extraction and explicit evidence-chain reasoning:
\[
\mathcal{T}_c = \arg\max_{\mathbf{z}} P_{\Psi} \big( \mathbf{z} \mid \mathcal{F}_t, \mathcal{C}_{ret}, \mathbf{p}_{align} \big),
\qquad
\hat{\mathbf{Y}} = \arg\max_{\mathbf{y}} P_{\Psi} \big( \mathbf{y} \mid \mathcal{T}_c, \mathcal{F}_t, \mathcal{C}_{ret}, \mathbf{p}_{sys} \big),
\]
which turns hidden intent inference into long-range causal reasoning [2604.12700].

## 5. Evaluation Regimes and Empirical Findings

Because MTIDs encode temporal intent structure, their evaluations differ sharply from standard single-turn classification. DeepContext reports **F1=0.84**, **Recall=0.83**, **Precision=0.86**, and **MTTD=4.24** on a **1,010**-conversation multi-turn jailbreak benchmark, outperforming Llama-Prompt-Guard-2-86M (**0.67**) and Granite-Guardian-3.3-8B (**0.67**), while maintaining sub-20ms inference overhead [2602.16935]. TurnGate, evaluated on MTID, is measured not only by benign pass-through but also by exact closure-time detection, early-block utility, and \(\ell_1\) distance between predicted intervention time \(\eta_\pi\) and gold closure \(t^*\), and is reported to substantially outperform prompt-based and guardrail baselines [2605.05630].

In agent settings, ISETrace shows that multi-turn, execution-grounded data materially changes downstream capability. Fine-tuning Qwen3-8B on ISETrace raises ClawEval **pass@1** from **19.3%** to **37.7%**, with completion increasing from **0.367** to **0.533** and robustness from **0.925** to **0.959**. An ablation that truncates trajectories to the first user turn lowers pass@1 to **28.1%**, indicating that Stage 2 multi-turn simulation supplies a large share of the gain [2606.11520].

In multilingual customer-service MTIC, LARA reports that retrieval-augmented prompting improves average accuracy by **3.67%** over state-of-the-art single-turn intent classifiers, while the later Symbol Tuning and C-LARA work reports **5.09%** higher classification accuracy, **40%** lower annotation cost, and average accuracy up to **66.35%** for the pseudo-labeling framework [2403.16504], [2411.12307]. MINT-CL, trained on MINT-E, shows that combining multi-turn synthetic data with contrastive response ranking yields the best overall average accuracy among its variants, reaching **61.55%** on five-market MTIC evaluation, with stronger gains in English-market averages [2411.14252].

Implicit-intent MTIDs also produce measurable efficiency gains. OnePred reports that recursive memory reduces per-turn token consumption by up to **22×** relative to full-history prompting while outperforming full-history and current-turn baselines on all three NQP-Bench subsets; on NQP-Wild, the RL-trained memory model reaches a judge score of **46.00** versus **44.35** for full-history [2605.23668]. In state-tracking settings, diversity-aware retrieval under matched token budgets produces strong Joint Goal Accuracy on standard de facto MTIDs: LDRA–GRPO reaches **89.35** JGA on MultiWOZ 2.4 and **92.01** on SGD [2510.17940].

Interactive benchmarks quantify phenomena that single-turn benchmarks cannot. CarryOnBench shows that at turn one, models fulfill only **10.5–37.6%** of the user’s benign information need, rising to **25.1–72.1%** when benign intent is stated upfront, and that with benign clarifications in multi-turn conversations **13 of 14** models approach or exceed this single-turn baseline [2604.27093]. MIntRec2.0 shows that multimodal intent recognition with natural OOS turns remains difficult: despite gains from multimodal fusion, ChatGPT trails humans by large margins, and the dataset’s **5,736** OOS utterances make open-world evaluation central rather than optional [2403.10943]. Caro, operating on multi-turn OOD intent data, improves **F1-OOD** by over **29%** compared to the previous best method [2305.03237]. MISID similarly exposes a large gap between state recognition and strategic hidden-intent reasoning; FRACTAM raises hidden intent inference from **34.22%** to **42.79%** on average over text-only LLM baselines [2604.12700].

## 6. Limitations, Misconceptions, and Open Directions

A recurring limitation is the absence of a shared public schema. DeepContext’s MTID-like corpus is proprietary; LARA’s and C-LARA’s industrial datasets are proprietary; several papers explicitly state that they do not release a formally named MTID even though they construct one conceptually [2602.16935], [2403.16504], [2411.12307]. This fragmentation complicates reproducibility, cross-paper comparison, and standardization of train/validation/test protocols. Even in released resources, supervision is heterogeneous: some datasets label only the final turn, some label every turn, some annotate binary safety, some annotate structured intents, and some operationalize intent through the next query or execution trace [2605.23668], [2606.11520], [2403.10943].

Another limitation is coverage. ISETrace is restricted to macOS/Linux shell tasks and does not test Windows, GUI automation, or browser automation; MISID is domain-specific to strategic deception games; CarryOnBench simulates user clarifications rather than sampling real user logs; DeepContext notes that future attack forms are not yet included; MTID for TurnGate is currently limited to chemistry and cybersecurity [2606.11520], [2604.12700], [2604.27093], [2602.16935], [2605.05630].

A common misconception is that MTID problems are solved simply by adding more context. Multiple papers argue against this. DeepContext reports that stateless LLM baselines often perform better with a **sliding window** than with full concatenated context because long benign preambles dilute adversarial signal; LDRA finds that **retrieval diversity, rather than longer prompts**, improves multi-turn intent understanding under fixed token budgets; the LiC paper argues that scaling model size or training alone cannot eliminate the structural ambiguity in \(C_t\) because the core problem is an **intent alignment gap** rather than pure execution weakness [2602.16935], [2510.17940], [2602.07338]. This suggests that MTID research is shifting from raw context accumulation toward *state abstraction*, *retrieval selection*, *response awareness*, and *causal evidence tracking*.

Open directions already appear in the literature. Several papers point toward richer per-turn labels, community-standard schemas, and open-source benchmarks [2602.16935], [2605.05630]. Others indicate the need for better multilingual generation in low-resource settings, broader domain transfer, and stronger multimodal grounding [2411.14252], [2604.12700], [2403.10943]. A plausible implication is that future MTIDs will increasingly combine three properties that currently tend to appear separately: explicit temporal annotations, grounded execution or environment feedback, and fine-grained evidence chains over long contexts.

Source: https://www.emergentmind.com/topics/multi-turn-intent-dataset-mtid