State-Update Multi-turn Dialogue Strategy

Updated 12 July 2026

State-update multi-turn dialogue strategy is an approach that incrementally revises a compact conversation state, addressing inefficiencies and information loss in raw dialogue history.
It employs diverse representations—such as structured symbolic states, natural language reformulations, and latent vectors—to tailor state updates for tasks like tracking, generation, and safety control.
Empirical findings demonstrate that these methods enhance long-horizon accuracy, reduce token consumption and inference time, and improve robustness across various dialogue applications.

State-update multi-turn dialogue strategy denotes a family of dialogue methods that treat conversation as an evolving process in which each turn revises a compact representation of what is currently relevant, instead of repeatedly consuming an ever-growing raw transcript. In the literature, this maintained representation may be a structured dialogue state of domain-slot-value triples, a rewritten self-contained utterance, a standalone reformulated query, a set of supporting sentences, an explicit recalled evidence set, a dependency-based thread structure, a latent dialogue-state vector, or a belief over hidden user state. The updated state is then used for downstream tracking, generation, planning, routing, safety control, or personalization (Chen et al., 2020, Su et al., 2019, Rastogi et al., 2019, Liu, 22 Sep 2025, Pang et al., 14 May 2026, Hu et al., 28 Feb 2025, Luo et al., 23 May 2026).

1. Motivations and problem formulation

A recurring motivation is that naïve multi-turn processing by raw history accumulation is both semantically fragile and computationally inefficient. In long-horizon prompting, models exhibit a recency bias and a “forgetting phenomenon,” with performance degrading as the number of turns grows and with early evidence being used less effectively than late evidence. The same line of work also identifies linear history concatenation as a direct source of rising token cost and inference time (Liu, 22 Sep 2025).

Other strands identify a more structural problem: the current utterance often does not contain enough information on its own. In ordinary conversation, coreference and information omission are frequent, so a downstream model may fail simply because the utterance is underspecified. One study reports, in a preliminary study on 2,000 Chinese conversations, coreference in 33.5%, omission in 52.4%, and neither in 29.7%, motivating explicit rewriting before downstream interpretation (Su et al., 2019).

In retrieval-based response selection, the difficulty is framed differently but points to the same underlying issue. Binary history–response classification can learn semantic relatedness without learning “what should come next,” because the dialogue is collapsed into a single long document and temporal dependencies between utterances are weakly modeled. This motivates explicit training signals for utterance order, adjacency, and inconsistency detection (Whang et al., 2020).

Task-oriented dialogue state tracking (DST) exposes another limitation of uniform context handling. One line of work argues that using one fixed dialogue history for all slots is suboptimal because different slots require different dialogue contents at different turns; a fixed history can therefore be insufficient for some slots and redundant for others (Guo et al., 2022). In multi-domain settings, contextual understanding is further complicated by heterogeneous schemas across domain-specific agents, quadratic growth in schema transitions, and the need to resolve referring expressions without modifying each downstream spoken language understanding subsystem (Rastogi et al., 2019).

The same state-update logic has also been extended beyond task completion. In safety research, multi-turn jailbreaking is described as contextual drift in an evolving conversation, which single-turn prompt filters do not address well. In personalization and counseling, the central claim is that dialogue should be treated as sequential decision-making under partial observability, because the relevant user condition is hidden, evolving, and action-conditioned (Hu et al., 28 Feb 2025, Luo et al., 23 May 2026).

2. Forms of dialogue state representation

There is no single canonical state representation. The surveyed literature instead uses several distinct but related forms, each chosen to support a particular downstream objective.

Representation type	Characteristic form	Representative papers
Structured symbolic state	Linearized domain-slot-value sequence or slot-conditioned belief state	(Chen et al., 2020, Guo et al., 2022)
Natural-language reformulation	Rewritten utterance or standalone query	(Su et al., 2019, Rastogi et al., 2019)
Explicit textual memory	Supporting sentences or recalled history turns	(Liu, 22 Sep 2025, Pang et al., 14 May 2026)
Structural context decomposition	Dependency-based threads or self-contained sub-dialogues	(Jia et al., 2020)
Latent dynamical state	User-state embeddings, hidden dialogue state, or belief over hidden user state	(Cheng et al., 2022, Hu et al., 28 Feb 2025, Luo et al., 23 May 2026)

In task-oriented DST, dialogue state is typically defined as a compact representation of the user goal in dialogue history, often as a set of domain-slot-value triples such as hotel(price=cheap) or taxi(destination=cambridge). One sequence-generation formulation linearizes the full state with markup tags, preserving inter-slot relations within a single structured output rather than predicting slots independently (Chen et al., 2020). A related slot-wise model represents the previous state as a concatenation of slot and value markers, then conditions turn-level selection and generation on that explicit prior state (Guo et al., 2022).

Natural-language reformulation methods treat the rewritten text itself as the updated state. One rewriting framework learns $p(R \mid H, U_n)$ , where the current utterance is rewritten into a self-contained utterance $R$ that recovers omitted or coreferred content (Su et al., 2019). Another approach formalizes contextual query reformulation as learning $y_t = f_\theta(x_t)$ , where $x_t$ is a window of previous user and system turns and $y_t$ is a standalone user query suitable for an existing downstream agent (Rastogi et al., 2019).

A third family makes state explicitly textual and cumulative. In prompt-based long-context reasoning, the state is the set of previously selected supporting sentences, carried forward turn by turn via XML tags and a “Previously selected” field (Liu, 22 Sep 2025). In self-recall reasoning, the state is the recursively constructed recalled set $H_k = H_{k-1} \cup \{o_k\}$ , where the model emits <HIS> tags to copy relevant past utterances into its reasoning trace before producing the answer (Pang et al., 14 May 2026).

Latent state models move away from directly observable memory. Multi-turn emotional support models construct per-round user-state embeddings $\mathbf{u}_i$ from system utterance, user utterance, emotion cause, and NRC VAD-based emotion embeddings, then stack them as $\mathbf{U}_t=[\mathbf{u}_1;\mathbf{u}_2;\dots;\mathbf{u}_{t-1}]$ (Cheng et al., 2022). Safety steering models define a hidden dialogue state $x_k$ in a learned state-space system (Hu et al., 28 Feb 2025). PUMA represents personalization through an approximate belief $q_\phi(s_t)$ over hidden user state, explicitly separating semantic memory from latent user state (Luo et al., 23 May 2026).

3. Mechanisms of state update across turns

In explicit DST, state update is typically incremental. One tracker compresses earlier information into the $R$ 0-th dialogue state $R$ 1 and combines it with the current $R$ 2 dialogue turns $R$ 3, so the model updates belief state from recent context plus carried-over prior state rather than rereading full history. Its architecture is coarse-to-fine: a context encoder, a state sketch decoder, a sketch encoder, and a sketch-guided state decoder (Chen et al., 2020).

Slot-wise update refines this idea by making history selection conditional on the slot. DiCoS-DST first applies a State Update Predictor to decide which slots should be updated and lets all other slots inherit their previous values. For slots selected for update, it scores each historical turn from three perspectives: explicit connection to the slot name (SN-DH), relevance to the current turn dialogue (CT-DH), and Implicit Mention Oriented Reasoning (IMOR). Only the top- $R$ 4 selected dialogue contents are then passed to the State Generator, explicitly minimizing distracting information before value prediction (Guo et al., 2022).

Text-to-text reformulation realizes state update through compression and rewriting. Utterance rewriting uses the pipeline $R$ 5, so the dialogue history is compressed into a single enriched utterance that resolves references and fills in omitted content (Su et al., 2019). Contextual query reformulation uses a pointer-generator network to rewrite context-dependent input into a standalone query, allowing a production system to preserve multi-turn context without modifying downstream domain-specific agents (Rastogi et al., 2019).

Some methods update state by explicit evidence accumulation rather than full rewriting. State Reconstruction and History Reminder reconstruct each new turn from the current passage plus previously selected supporting sentences, and instruct the model that “Each response should contain ALL supporting sentences (previous + new ones).” The state is therefore the accumulated supporting evidence, not the full transcript (Liu, 22 Sep 2025). Self-Recall Thinking performs a comparable update internally: it analyzes the query, recalls selected historical turns with <HIS> tags, integrates them into the reasoning chain, and answers from the recalled set $R$ 6 (Pang et al., 14 May 2026).

Structural decomposition offers a different update mechanism. A dialogue dependency parser can transform history into threads by predicting “reply-to” relations, removing low-confidence edges, and extracting root-to-leaf paths as self-contained sub-dialogues. Each new candidate response is then matched against thread-level representations rather than a flat context (Jia et al., 2020). In response selection, Utterance Manipulation Strategies update the model’s implicit notion of dialogue state through three self-supervised tasks—Insertion, Deletion, and Search—which respectively teach utterance position, inconsistency detection, and temporal adjacency (Whang et al., 2020).

4. Learning paradigms for state-update behavior

The learning signal for state update varies widely across the literature. A prominent formulation is dual learning. In dual-DST, DST is cast as sequence generation and coupled with a dual utterance generator in an iterative framework. The primal tracker maps dialogue context to state sequence, the dual model maps turn state plus delexicalized system utterance to delexicalized user utterance, and both are updated through reconstruction errors and reward signals without labeled data. External-knowledge rewards from pretrained LLMs $R$ 7 and $R$ 8 are combined with BLEU-based reconstruction rewards as $R$ 9 and $y_t = f_\theta(x_t)$ 0 (Chen et al., 2020).

Supervised reformulation methods often rely on copying-oriented objectives. Contextual query reformulation uses a pointer-generator network together with an Entity-Copy Auxiliary Objective, motivated by the observation that different gold rewrites for the same context often contain the same set of entities even when token order differs. The auxiliary classifier predicts which entity tokens should be copied, while inference remains based on the main rewriting model (Rastogi et al., 2019).

A different route is self-supervision. Utterance Manipulation Strategies require no additional annotation: the held-out utterance for insertion, the random utterance for deletion, and the shuffled predecessor search target are all generated automatically from dialogue data. The final training objective is the sum of response-selection loss and auxiliary UMS losses with equal weight (Whang et al., 2020).

Strategy induction and reuse make state-update behavior more explicit at the discourse level. IDEAS first induces original strategies from human-machine dialogues, abstracts them into a high-level strategy pool, and then, at each new turn, uses a ranker and user simulator to select a strategy and generate the next instruction conditioned on current dialogue history. A reflection module enforces correctness and coherence by rejecting instructions that contradict prior answers or are already answerable from existing answers (Ou et al., 2024).

Self-recall reasoning combines staged supervision with reinforcement learning. SRT first performs supervised fine-tuning so the model can emit recall-based reasoning chains with <HIS> tags, then applies GRPO using a composite reward $y_t = f_\theta(x_t)$ 1. The recall term is a scaled Jaccard score over predicted and gold recall sets, directly rewarding minimal sufficient evidence selection (Pang et al., 14 May 2026).

5. Planning, control, and action selection over updated state

Once state is maintained explicitly, many systems use it not only for understanding but for long-horizon decision-making. In knowledge-grounded conversation, one framework defines dialogue strategy as a distribution over knowledge items,

$y_t = f_\theta(x_t)$ 2

so knowledge selection depends jointly on the partner’s last utterance and prior context. The strategy is optimized with policy gradient under a compound reward that combines informativeness and coherence, while the decoder is kept fixed to preserve linguistic quality (Bao et al., 2019).

In emotional support dialogue, MultiESC explicitly scores a candidate strategy by

$y_t = f_\theta(x_t)$ 3

where $y_t = f_\theta(x_t)$ 4 is history-based strategy prediction and $y_t = f_\theta(x_t)$ 5 is a lookahead heuristic estimating future user feedback over a bounded horizon $y_t = f_\theta(x_t)$ 6 with beam search over top- $y_t = f_\theta(x_t)$ 7 future strategy sequences. The same model maintains a user-state history $y_t = f_\theta(x_t)$ 8 and conditions both planning and generation on it (Cheng et al., 2022).

State-aware policy learning in open-domain dialogue also uses future-state-dependent reward. WeaSuL models dialogue as trajectories of state-action pairs, weakly labels simulated $y_t = f_\theta(x_t)$ 9 pairs into VeryHigh, High, and Low reward buckets with three quality modules—Semantic Relevant, Semantic Coherence, and Consistent Flow—and further shapes reward with the next state through

$x_t$ 0

This makes the learned policy depend not only on whether the current reply is locally good, but on whether it leads to a better next state (Khandelwal, 2021).

Clinical dialogue makes the sequential-control interpretation explicit. DoctorAgent-RL formulates consultation as an MDP whose action space is $x_t$ 1. The doctor observes history $x_t$ 2, asks one question or terminates with a diagnosis, receives a patient response that updates the state, and is optimized by GRPO under rewards for diagnostic accuracy, information acquisition efficiency, and protocol compliance. A Dynamic Turn Budget Training Strategy randomly assigns a turn budget from 2 to 10 and reminds the model of the remaining turns after each interaction (Feng et al., 26 May 2025).

Routing and negotiation extend state update from content selection to model or policy selection. DialRouter treats multi-turn LLM routing as a sequential decision problem with state $x_t$ 3, action $x_t$ 4 as the choice of LLM, offline MCTS for long-horizon trajectory discovery, and retrieval-based future state approximation during policy learning (Sun et al., 13 Apr 2026). GameTalk optimizes full-conversation strategic utility rather than isolated turns, adapting GRPO, DPO, and STaR through branching rollouts from a shared root conversation so that reward depends on the whole interaction trajectory (Vendrell et al., 22 Jan 2026).

Two recent lines generalize state update into formal control or model-based planning. Safety steering models dialogue as a latent dynamical system,

$x_t$ 5

and uses a neural barrier function to define a safe set $x_t$ 6, thereby enforcing invariant safety under contextual drift (Hu et al., 28 Feb 2025). PUMA models personalization as partial observability, learns $x_t$ 7 and $x_t$ 8, maintains a belief $x_t$ 9, and selects actions by minimizing expected free energy $y_t$ 0, explicitly balancing epistemic and pragmatic objectives (Luo et al., 23 May 2026). MAPO, finally, addresses long-horizon subjective dialogue with dense process feedback, Monte Carlo returns, and a mixed advantage estimator that combines turn-level normalized return advantage with batch-level normalized immediate-reward advantage, remaining critic-free throughout (Zhang et al., 6 Mar 2026).

6. Empirical findings across applications

Reported results indicate that state-update methods improve either long-horizon accuracy, efficiency, or robustness, and often all three.

Area	Representative reported result	Paper
Task-oriented DST	Dual-DST reaches 49.88% joint accuracy on MultiWOZ 2.1; its coarse-to-fine tracker reaches 48.79%, versus 45.60% for TRADE	(Chen et al., 2020)
Slot-wise history selection for DST	DiCoS-DST reports 61.02% joint goal accuracy on MultiWOZ 2.1 and 61.13% on MultiWOZ 2.2	(Guo et al., 2022)
Utterance rewriting	Intent precision improves from 80.77% to 89.91%; chit-chat CPS improves from 6.3 to 7.7	(Su et al., 2019)
Query reformulation	InCar ResF1 All improves from 33.6 to 36.1; on the hardest internal subset $y_t$ 1, Entity F1 reaches 77.1	(Rastogi et al., 2019)
Prompt-based state reconstruction	On HotpotQA, core information filtering improves by 32.6%, QA score by 14.1%, inference time drops by 73.1%, and token consumption by 59.4%	(Liu, 22 Sep 2025)
Self-recall reasoning	The abstract reports 4.7% F1 improvement and 14.7% end-to-end latency reduction	(Pang et al., 14 May 2026)

The state-update idea also improves robustness and strategic accuracy in settings where local matching is insufficient. UMS improves retrieval-based response selection across Ubuntu, Douban, E-Commerce, and Kakao, and in adversarial evaluation the average drop in $y_t$ 2 is about 48% for UMS versus about 58% for baselines, indicating stronger temporal-consistency modeling (Whang et al., 2020).

In emotionally grounded and personalized dialogue, the gains are tied directly to state tracking quality. MultiESC reports 42.01 strategy accuracy and 34.01 weighted-F1, versus 31.61 accuracy for MISC, and is preferred by human evaluators on fluency, empathy, identification of the user’s situation, suggestion quality, and overall support (Cheng et al., 2022). On CAMI, PUMA reports Curr-Acc = 0.689 and Next-Acc = 0.717, whereas the long-prompt baseline reaches Curr-Acc = 0.667 and Next-Acc = 0.231, which is a particularly sharp difference on next-state prediction (Luo et al., 23 May 2026).

A common empirical pattern is that explicit state update is especially valuable when annotation is scarce or dependencies are long-range. Dual learning improves more than pseudo labeling when only 20% labeled data is available in DST (Chen et al., 2020), while SRT is strongest on SRQA, the benchmark specifically designed for long-range dependency (Pang et al., 14 May 2026).

7. Limitations, distinctions, and open questions

A common misconception is to equate state update with symbolic belief tracking alone. The literature shows a much broader design space: state may be a symbolic sequence, a natural-language rewrite, a cumulative evidence set, an explicit recall chain, a latent dynamical vector, or a belief distribution over hidden user state (Chen et al., 2020, Rastogi et al., 2019, Liu, 22 Sep 2025, Hu et al., 28 Feb 2025). What unifies these approaches is not the representational format but the commitment to turn-by-turn revision of a compact decision-relevant state.

The surveyed methods also expose recurring limitations. In utterance rewriting, information completion remains harder than coreference resolution, and positive samples are harder than negative ones (Su et al., 2019). Prompt-based state reconstruction is manually crafted, depends on correct <info> extraction, assumes that relevant evidence can be represented as supporting sentences, and is evaluated primarily on QA-style tasks (Liu, 22 Sep 2025). Self-recall reasoning still deteriorates with dialogue length: as dialogue length increases from 8 to 32 turns, Missing Recall rises from 23.5% to 35.9%, Wrong Recall rises from 15.0% to 19.2%, and Over Recall remains around 5–6% (Pang et al., 14 May 2026).

In emotionally sensitive dialogue, improved state-update planning does not eliminate broader safety and personalization concerns. MultiESC is reported to remain often general and repetitive compared with human supporters, may lack personalized, deep understanding or highly specific suggestions, and is explicitly positioned for peer support rather than professional psychotherapy; robust crisis detection is still needed (Cheng et al., 2022). PUMA’s main evaluation uses dataset-specific discrete state annotations and simulator-based dynamics, so its dynamic results should not be interpreted as direct clinical effectiveness (Luo et al., 23 May 2026).

Safety-oriented state update introduces its own trade-off. Neural barrier function steering yields stronger defenses against multi-turn jailbreaks, but the threshold $y_t$ 3 controls a clear safety-helpfulness frontier: smaller $y_t$ 4 gives less aggressive filtering and better helpfulness, while larger $y_t$ 5 yields stronger safety steering with more helpfulness degradation (Hu et al., 28 Feb 2025). Structural decomposition methods also illustrate that not every state-update method is a full dialogue manager: Thread-Encoder is thread-aware and dependency-driven, but it remains fundamentally a response matching model rather than an explicit symbolic state tracker (Jia et al., 2020).

Taken together, these results suggest that state-update multi-turn dialogue strategy is best understood as a general architectural principle rather than a single algorithmic recipe. The central research questions now concern which state representation is appropriate for which task, how dense or delayed the supervisory signal should be, and how to balance compactness, interpretability, robustness, and long-horizon control in evolving dialogue.