Multi-Strategy Utterance Generation

• Multi-strategy utterance generation is a design principle that models an utterance as the outcome of multiple control signals such as strategy-response pairs, dialogue acts, and planning tokens.
• It integrates various paradigms from emotional support and negotiation to task-oriented dialogue and rewriting, employing architectures like planner-realizer splits and editing-based methods.
• Explicit supervision and control signals enhance performance on metrics like BLEU, ROUGE, and BERTScore while highlighting challenges in strategy diversity and sequence accuracy.

Multi-strategy utterance generation denotes a family of generation problems in which an utterance is not treated as the realization of a single undifferentiated decision, but as the product of multiple communicative, stylistic, editing, or expressive choices. In current research usage, the term is explicit in emotional support conversation, where a single supporter turn may contain multiple ordered strategy-response pairs (Zhu et al., 20 Apr 2026), and in related work on emotional support that redefines the target as a full supporter turn rather than a single strategy-utterance pair (Bai et al., 21 May 2025). Closely related formulations appear in negotiation, where high-level strategy is decoupled from surface realization (He et al., 2018), in task-oriented NLG, where sentence planning and stylistic parameters are explicitly controlled (Reed et al., 2018, Oraby et al., 2018), in multi-turn dialogue rewriting, where utterances are generated through structured editing operations (Huang et al., 2020, Cao et al., 20 Mar 2025), and in dialogue speech generation, where speaker identity, turn-level expression, and contextual appropriateness are separated into different control channels (Su et al., 9 Apr 2026).

1. Conceptual scope and historical development

The clearest explicit formulation of multi-strategy utterance generation comes from emotional support conversation. One ESConv analysis identifies the “Consecutive Use of multiple Strategies (CUS)” and reports that 2,525 out of 13,767 supporter responses use more than one strategy, or roughly 18.3% (Bai et al., 21 May 2025). A related reformulation of ESC reports that 17.7% of supporter utterances contain two or more strategies and models each supporter turn as one or more ordered strategy-response pairs rather than a single strategy-response pair (Zhu et al., 20 Apr 2026). These results establish that, at least in ESC, the single-strategy-per-turn assumption is empirically incomplete.

Older dialogue and NLG work reached similar design conclusions without using the same label. In negotiation, “Decoupling Strategy and Generation in Negotiation Dialogues” argues that end-to-end systems entangle high-level decision making with surface realization, and replaces direct response generation with a factorization through coarse dialogue acts such as propose(price=50), counter, accept, reject, and inquire (He et al., 2018). In task-oriented generation, “Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?” treats sentence scoping, distributive aggregation, and discourse contrast as explicit planning operations that determine how multiple propositions are packaged into one or more utterances (Reed et al., 2018). “Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators” similarly frames style as a bundle of aggregation and pragmatic-marker decisions rather than a single latent label (Oraby et al., 2018).

A related line of work treats multi-strategy generation as structured rewriting rather than direct response production. “Improving Multi-turn Dialogue Modelling with Utterance ReWriter” formulates rewriting as recovery of omitted and coreferred information before downstream generation (Su et al., 2019), while “SARG” and “EO-IUR” decompose rewriting into multiple editing operations such as copy, delete, replace, and insert (Huang et al., 2020, Cao et al., 20 Mar 2025). This suggests that the phrase “multi-strategy” covers several neighboring ideas: multi-act response composition, explicit planning variables, editing-operation guidance, and multimodal turn-level control.

2. Representational units and formalizations

Across the literature, the main difference is not whether strategies exist, but what counts as a strategy-bearing unit. In ESC, the unit is an ordered sequence of strategy-response pairs within one supporter turn. In negotiation, it is a coarse dialogue act chosen before realization. In rewriting, it is an editing operation tied to specific source tokens. In dialogue speech generation, it is a set of turn-level expressive attributes layered on top of a stable speaker representation.

Paradigm	Strategy unit	Representative papers
Emotional support turn composition	Ordered strategy-response pairs within one utterance	(Zhu et al., 20 Apr 2026, Bai et al., 21 May 2025)
Negotiation planning	Coarse dialogue acts such as propose(price=50)	(He et al., 2018)
Sentence planning and style control	period, distribute, contrast, and stylistic parameter vectors	(Reed et al., 2018, Oraby et al., 2018)
Incomplete utterance rewriting	KEEP/DELETE/CHANGE or NA/RP/NW/IN editing operations	(Huang et al., 2020, Cao et al., 20 Mar 2025)
Dialogue speech generation	Speaker-level captions plus CoT turn attributes	(Su et al., 9 Apr 2026)

The most explicit turn-level formalization appears in ESC. One formulation defines the dialogue context as

$$U=\{u_1^{usr}, u_1^{sys}, u_2^{usr}, u_2^{sys}, \ldots, u_t^{usr}\},$$

and decomposes the supporter utterance as

$$u_t^{sys} = \{r_{t,1}^{sys}, \ldots, r_{t,N_t}^{sys}\},$$

where each segment $r_{t,i}^{sys}$ is associated with a support strategy $s_{t,i}^{sys}$ (Zhu et al., 20 Apr 2026). A related reformulation defines the target supporter response as a sequence

$$r^{(i)} = \{ur_1^{(i)}, ur_2^{(i)}, \ldots, ur_{m_i}^{(i)}\},$$

with the final utterance in the context constrained to come from the seeker (Bai et al., 21 May 2025). In both cases, the prediction object is the full supporter turn, not an isolated local act.

In negotiation, the canonical decomposition is

$$p(x_t \mid h_t) = \sum_{z_t} p(z_t \mid h_t)\, p(x_t \mid z_t, h_t),$$

where $z_t$ is a coarse dialogue act and $x_t$ is the realized utterance (He et al., 2018). In rewriting, the corresponding formalizations are $p(R \mid (H, U_n))$ for utterance rewriting (Su et al., 2019) and $P(\mathcal{T}\mid \mathcal{D})$ for incomplete utterance rewriting (Cao et al., 20 Mar 2025). These formulations differ operationally, but they share a structural principle: the generator is constrained by an intermediate representation that narrows the space of plausible utterances.

3. Architectural patterns

One major pattern is the planner-realizer split. In negotiation, the strategy module predicts a coarse act from dialogue history, partner behavior, item context, and price variables, while a retrieval-based realization module produces natural language conditioned on that act and the current context (He et al., 2018). This architecture makes it possible to swap in supervised, reinforcement-learned, or rule-based policies without changing the utterance generator. A related, more fine-grained variant appears in task-oriented stylistic generation, where the strongest model conditions on a context vector of 36 stylistic parameters and exposes that vector to the decoder through an added encoder state under attention (Oraby et al., 2018).

A second pattern is explicit within-turn sequencing. In ESC, “All-in-One” serializes the full strategy-response sequence in one decoding pass, whereas “One-by-One” generates one strategy-response pair at a time and adds a binary stop flag (Zhu et al., 20 Apr 2026). In the redefined ESConv task, three supervised variants occupy different points in this design space: MCC predicts the entire strategy sequence as one class, MTL jointly predicts response tokens and strategy labels during decoding, and CP uses a special planning token [SN] together with a bag-of-words objective so that each utterance-level segment is planned before realization (Bai et al., 21 May 2025). These architectures treat a supporter turn as a structured object rather than a flat text span.

A third pattern is editing-based generation. “SARG” augments the source utterance with dummy tokens, predicts KEEP, DELETE, and CHANGE operations in parallel, and runs autoregressive generation only on positions tagged CHANGE (Huang et al., 2020). “EO-IUR” adds a token-level heterogeneous dialogue graph and a multitask sequence labeling module with labels NA, RP, NW, and IN, then uses the resulting label probabilities to rescale decoder cross-attention toward editing-relevant tokens (Cao et al., 20 Mar 2025). In these systems, strategy is not a dialogue act but an edit program.

A fourth pattern is multi-path intention and context fusion. “InferEM” separately encodes the whole dialogue, the actual last utterance, and a predicted “virtual” last utterance, then fuses them through a multi-head attention based intention fusion module before empathetic response generation (Lv et al., 2022). “Multi-party Response Generation with Relation Disentanglement” is adjacent to this line: it infers latent relations among utterances through a deep graph random process and uses them to guide structure-aware response generation in multi-party dialogue (Dai et al., 2024). This suggests that multi-strategy generation often depends on disentangling multiple latent sources of control even when those sources are not named as strategies.

A fifth pattern is multimodal control at different temporal scales. “CapTalk” conditions speech generation on speaker-level captions, dialogue context, target text, a CoT control sequence in dialogue, an utterance-level speaker embedding, and hierarchical latent variables that separate stable timbre from context-adaptive expression (Su et al., 9 Apr 2026). Here the strategy space is not rhetorical or dialog-act based, but expressive: <emotion><tone><pitch><energy><speed> explicitly plans how the current turn should sound.

4. Learning signals, supervision, and control mechanisms

The literature repeatedly shows that multi-strategy generation benefits from explicit supervision rather than purely latent emergence. In sentence planning, adding simple control tokens for period, distribute, or contrast makes strategy realization reliable, whereas unsupervised end-to-end models often collapse onto common surface patterns (Reed et al., 2018). In personality-based stylistic generation, coarse personality-token supervision helps less than explicit parameter-level conditioning, indicating that strategy variables work better when they are close to the realization operations they are meant to control (Oraby et al., 2018).

Supervision also appears as auxiliary structure rather than only direct labels. “EO-IUR” uses multitask learning so that edit-label prediction directly biases cross-attention during generation (Cao et al., 20 Mar 2025). “Utterance Rewriting with Contrastive Learning in Multi-turn Dialogue” adds Words Detection, sentence-level contrastive learning for intent consistency, and probability contrastive learning across stochastic views (Wang et al., 2022). “Do Response Selection Models Really Know What’s Next?” is not a generation paper, but its insertion, deletion, and search tasks were designed to teach utterance order and dialogue coherence, which suggests a transferable auxiliary supervision strategy for generators that otherwise optimize only semantic relatedness (Whang et al., 2020).

In ESC, recent work combines structured reasoning with reinforcement learning. One reasoning-augmented formulation prepends a four-node chain—Context, Cognition, Emotion, Support Plan—inside > tags, and then generates structured strategy-response outputs inside <answer> tags (Zhu et al., 20 Apr 2026). The same work applies GRPO with rewards based on strategy-sequence Levenshtein Ratio and, for iterative decoding, correct stop-flag prediction. This is a strong form of explicit control: the model is trained not only to write a supportive utterance, but also to align its internal plan and output structure with the reference sequence of strategies.

Control can also be inserted at inference time rather than through task labels alone. “Improved Goal Oriented Dialogue via Utterance Generation and Look Ahead” generates successive user utterances from unlabeled dialogue data and uses those generated continuations as additional evidence for intent prediction, including counterfactual look-ahead under conflicting intents (Ben-David et al., 2021). Although this work focuses on understanding rather than response generation, it demonstrates a broader principle: generated utterances can function as internal strategic probes, not only as user-facing outputs.

5. Evaluation regimes and empirical findings

The strongest empirical evidence for explicit multi-strategy utterance generation currently comes from ESC. In the multi-strategy ESC formulation, the best strategy-prediction numbers are obtained by One-by-One + reasoning + RL, which reaches EMR 33.53, LR 37.97, and BERTScore 21.11 at the utterance level, while All-in-One + reasoning + RL attains the strongest overlap metrics, including BLEU-4 4.07 and ROUGE-L 20.71 (Zhu et al., 20 Apr 2026). At the dialogue level, One-by-One + reasoning + RL reaches success rate 40.00, compared with 13.85 for the single-strategy baseline. The same study reports that human references contain 18.9% multi-strategy utterances, while the best models still generate such utterances only 8.4% or 7.7% of the time, showing substantial remaining underproduction (Zhu et al., 20 Apr 2026).

The ESConv reformulation based on CUS yields a complementary picture. Among supervised models, CP is best on exact and approximate strategy-sequence match with EMR 19.11 and LR 24.9 on $$u_t^{sys} = \{r_{t,1}^{sys}, \ldots, r_{t,N_t}^{sys}\},$$0, while MTL produces by far the largest variety of distinct strategy sequences, 754, but low sequence accuracy (Bai et al., 21 May 2025). Human evaluation produces a different ranking: DeepSeek-R1 and GPT-4o receive the highest scores on Fluency, Identification, Comforting, Suggestion, and Overall, and in that setup they outperform both supervised systems and human supporters. The same paper reports that LLMs frequently ask questions and provide suggestions, contrary to earlier claims that they systematically avoid those behaviors (Bai et al., 21 May 2025). A plausible implication is that exact reference strategy order and perceived supportive quality are related but non-identical targets.

Task-oriented sentence-planning experiments show even sharper gains from explicit control. For sentence scoping, period accuracy rises from 0.216 without supervision to 0.995 with a period-count token. For distributive aggregation, accuracy rises from 0.29 to 0.99, and generalization on the held-out high/high case rises from 0.00 to 0.98 under binary supervision (Reed et al., 2018). These are unusually direct demonstrations that explicit planning variables can convert a weakly emergent behavior into a controllable one.

Personality-conditioned task-oriented generation yields a similar conclusion. The model conditioned on a 36-dimensional stylistic context vector achieves BLEU 0.3766, NIST 5.3437, METEOR 0.3964, and ROUGE-L 0.5255, while substantially reducing semantic errors relative to a weaker personality-token baseline (Oraby et al., 2018). The paper’s semantic error analysis—deletions, repetitions, substitutions—shows that coarse style tags can damage content fidelity, whereas explicit strategy supervision can preserve both style and meaning.

In rewriting, the gains arise from structured editing rather than discourse-act composition. “Utterance ReWriter” reports that its best model, T-Ptr-$$u_t^{sys} = \{r_{t,1}^{sys}, \ldots, r_{t,N_t}^{sys}\},$$1, reaches BLEU-4 52.47, ROUGE-L 77.70, EM 55.84 on positive samples, and EM 98.14 on negative samples, while also improving a task-oriented chatbot’s intent precision from 80.77% to 89.91% and a chit-chat system’s average conversation-turns-per-session from 6.3 to 7.7 (Su et al., 2019). “SARG” attains the best results on six of seven metrics on Restoration-200k and is roughly 10x faster than T-Ptr-$$u_t^{sys} = \{r_{t,1}^{sys}, \ldots, r_{t,N_t}^{sys}\},$$2 in inference time, because only short edited spans are generated autoregressively (Huang et al., 2020). “EO-IUR” further improves this line, reaching EM 80.8 and BLEU-4 90.6 on TASK, and reducing redundant-token generation from 12.75% without edit guidance to 8.25% with edit guidance (Cao et al., 20 Mar 2025).

Multimodal evidence extends the same pattern to speech. CapTalk achieves 73.73 AVG on InstructTTSEval-ZH in single-utterance voice design and, in dialogue, obtains CoT controllability rates of 0.7675 for Emotion and Tone, 0.8400 for Pitch, 0.8550 for Energy, and 0.8675 for Speed (Su et al., 9 Apr 2026). In dialogue evaluation, the full CoT-conditioned model is preferred to a no-CoT variant on contextual coherence by both Gemini and human judges, indicating that turn-level explicit control can improve not only prosodic variation but also perceived discourse appropriateness.

6. Limits, misconceptions, and open directions

A recurrent misconception is that any latent structural factor learned by a model should count as a human-interpretable strategy. Work on latent relation modeling in multi-party dialogue cautions against this identification: latent relation types can guide generation, but they are not the same as explicit human-defined strategies unless the training setup names or constrains them (Dai et al., 2024). The same caution applies to style, planning, and editing variables more broadly.

Another recurring issue is evaluation mismatch. In the ESConv reformulation, CP has the best EMR and LR but the weakest human ratings among supervised models, while MTL has poor exact strategy-sequence match yet stronger human judgments (Bai et al., 21 May 2025). This indicates that multiple strategy sequences may be valid for the same context. Exact match is therefore informative for structural imitation, but it is not a sufficient measure of utterance quality. The underproduction of multi-strategy utterances in ESC, despite improved formulations, reinforces this point: current models remain biased toward simpler outputs because single-strategy examples still dominate the data (Zhu et al., 20 Apr 2026).

Architectural trade-offs also remain domain-specific. Negotiation-style act decoupling provides interpretability and policy modularity, but coarse acts lose nuance and retrieval-based realization can limit novelty (He et al., 2018). Rewriting-based systems are efficient when source-target overlap is high, but their edit formulations are less natural for tasks that require extensive paraphrasing or global restructuring (Huang et al., 2020, Cao et al., 20 Mar 2025). Sentence-planning tokens work well for operations such as period or distribute, but richer semantic supervision can perform worse than simpler binary signals, as seen in distributive aggregation (Reed et al., 2018). In speech generation, CapTalk offers explicit control over how a turn is spoken, but not over what semantic strategy the turn should contain (Su et al., 9 Apr 2026).

Taken together, these works suggest that multi-strategy utterance generation is not a single method class but a design principle. The shared principle is to expose intermediate control variables—dialogue acts, strategy sequences, planning tokens, editing operations, intention signals, or expressive CoT attributes—so that utterance generation becomes compositional, inspectable, and tunable. The strongest empirical results consistently arise when those variables are explicit enough to guide decoding, yet structured enough to remain compatible with natural language realization (He et al., 2018, Zhu et al., 20 Apr 2026, Cao et al., 20 Mar 2025).