Direct Response Generation

• Direct response generation is a conditional paradigm that maps inputs (e.g., dialogue acts, review attributes) directly to responses without intermediate intent recognition.
• Architectural variations such as seq2seq, hierarchical models, and LLM-based prompting offer flexible control mechanisms across diverse applications like open-domain dialogue and customer review response.
• Empirical studies show that these methods can achieve higher specificity, diversity, and engagement compared to traditional pipelines despite challenges in evaluation and controllability.

Searching arXiv for the cited papers and closely related work on direct response generation. Direct response generation is a family of conditional generation paradigms in which a model maps an input representation directly to a response utterance. In the dialogue-systems literature, this can mean mapping a raw user utterance, optionally with dialogue history, directly to an agent reply without any explicit intermediate intent-classification component; in controllable NLG, it can mean mapping a meaning representation consisting of a dialogue act and semantic attributes directly to a surface utterance without hand-crafted templates or multiple pipeline steps (Bolshinsky et al., 5 Sep 2025, Ramirez et al., 2023). Across app-review automation, open-domain dialogue, target-guided transition generation, and service assistants, the shared objective is to learn $p_\theta(y \mid x)$ or a closely related conditional distribution while coping with the one-to-many structure of valid replies, the need for control over response properties, and the difficulty of evaluating response quality beyond word overlap (Gao et al., 2020, Xu et al., 2019).

1. Definition and formal problem setting

In the “single-step” formulation for service assistants, direct response generation trains a single model $p_\theta(y \mid x)$ end-to-end to maximize the conditional likelihood of an agent response $y$ given user input $x$, rather than factorizing response generation through an explicit intent variable $c$ (Bolshinsky et al., 5 Sep 2025). In dialogue-act-conditioned NLG, the same idea is expressed as a direct mapping from a meaning representation $x=(d,a)$, where $d$ is a specified dialogue act and $a$ is a bag of semantic attributes, to an utterance $y=f(d,a)$ (Ramirez et al., 2023).

Several works make explicit that the response space is intrinsically one-to-many. Gao et al. formulate short-text conversation as generating a set of responses $\{r_1,\dots,r_K\}$ for a single post and note that a given post is often associated with multiple replies simultaneously in real applications; their machinery can then be collapsed to a single direct reply by taking $p_\theta(y \mid x)$0 or selecting the top latent word and decoding once (Gao et al., 2018). “Neural Response Generation with Meta-Words” formalizes this same one-to-many relation through structured response attributes, or meta-words, that specify what kind of response to generate (Xu et al., 2019).

The conditional signal provided to a direct response generator varies by task. RRGen conditions on a review, keyword-topic tags, and review-side attributes such as app category, review length, user rating, and inferred sentiment score (Gao et al., 2020). CODA conditions on dialogue context, a target sentence, and a generated commonsense bridging path (Gupta et al., 2022). EM pre-training for multi-party response generation conditions on dialogue history together with an addressee label, observed during downstream training and latent during pre-training (Li et al., 2023). These variations preserve the same underlying principle: response generation is performed directly from the model’s conditioned input rather than from manually authored templates.

2. Architectural families

The literature does not use a single canonical architecture for direct response generation. Instead, it presents a set of conditioning and control mechanisms layered onto seq2seq, Transformer, or LLM backbones.

Family	Core conditioning signal	Representative work
Attentional seq2seq	Review attributes and keyword-topic tags	RRGen (Gao et al., 2020)
Hierarchical recurrent encoder-decoder	Word-level and utterance-level attention over context	HRAN (Xing et al., 2017)
GPT-2 target-guided generation	Commonsense bridging path plus target sentence	CODA (Gupta et al., 2022)
Encoder-decoder with latent structure	Latent word, addressee label, or meta-word memory	(Gao et al., 2018, Li et al., 2023, Xu et al., 2019)
LLM prompting and ranking	Few-shot prompts, evaluator scores, or RAG context	(Ramirez et al., 2023, Sakaeda et al., 2022, Azov et al., 2024)

RRGen extends a standard attentional Seq2Seq architecture by explicitly injecting an A-component of high-level attributes and a K-component of keyword/topic tags. Its review encoder uses token embeddings enriched with keyword-topic signals and a bidirectional GRU; its relation-learning module embeds each attribute with small MLPs and concatenates them with the encoder’s final state; and its decoder is a unidirectional GRU initialized with the fused context and equipped with attention over encoder states (Gao et al., 2020).

HRAN addresses multi-turn response generation with a three-layer encoder-decoder in which a word-level bidirectional GRU produces hidden states per utterance, an utterance-level GRU processes utterance summaries in reverse order, and hierarchical attention attends both within utterances and among utterances at each decoding step (Xing et al., 2017). This design explicitly models the fact that words and utterances in a context are differentially important.

CODA uses a two-stage GPT-2-based framework. A Knowledge Path Generator first produces a multi-hop commonsense path between a source concept in the dialogue context and a target concept in the target sentence; a Commonsense Response Generator then conditions on the generated path, the target sentence, and the dialogue context to decode the transition response (Gupta et al., 2022). The explicit path serves as a controllable intermediate structure, but the output remains a directly generated response.

Other models introduce different forms of latent or structured control. Gao et al. use a latent variable $p_\theta(y \mid x)$1 drawn from the vocabulary as a semantic anchor for response generation (Gao et al., 2018). The EM-based multi-party model builds $p_\theta(y \mid x)$2 on BART and injects addressee information through learned addressee embeddings in the encoder input (Li et al., 2023). Meta-word generation augments seq2seq with a Goal-Tracking Memory Network that embeds response attributes, tracks which attributes have been satisfied, and reads out residual goals to guide the decoder (Xu et al., 2019).

LLM-era systems expand the design space further. Ramirez et al. use few-shot prompting plus overgenerate-and-rank for dialogue-act realization (Ramirez et al., 2023). The generator-evaluator architecture of “Generate, Evaluate, and Select” uses a T5 generator and a BERT evaluator that scores candidate responses and selects the best one (Sakaeda et al., 2022). SCRABLE couples retrieval-augmented generation, an LLM-as-a-Judge, and self-optimizing prompts for customer review responses (Azov et al., 2024). This suggests that direct response generation is best understood as a problem formulation rather than a single model class.

3. Objectives, decoding, and control mechanisms

A large portion of the literature trains direct response generators with standard autoregressive maximum likelihood or cross-entropy objectives. RRGen maximizes $p_\theta(y \mid x)$3 over review-response pairs and decodes with beam search of size $p_\theta(y \mid x)$4 (Gao et al., 2020). HRAN minimizes the negative log-likelihood of the target response conditioned on the multi-turn context (Xing et al., 2017). The fine-tuned T5 direct-response model for service assistants also uses standard token-level cross-entropy, optimized by AdamW via teacher forcing (Bolshinsky et al., 5 Sep 2025). CODA separately trains the path generator and the response generator with negative log-likelihood objectives and combines them as $p_\theta(y \mid x)$5 (Gupta et al., 2022).

Not all systems rely exclusively on MLE. Gao et al. formulate response generation with a latent-word policy $p_\theta(y \mid x)$6 and a response generator $p_\theta(y \mid x)$7, then fine-tune with REINFORCE using token-level F1-based rewards defined against a bag of human replies (Gao et al., 2018). The EM pre-training approach for multi-party dialogue treats the addressee as a discrete latent variable, performs hard-EM by selecting the highest-posterior addressee label, and updates the generator on a high-confidence subset of pseudo-labeled examples (Li et al., 2023). In spoken-dialog response generation, automatic conversation evaluators are incorporated both by reranking beam candidates and by augmenting the training loss with an evaluator-based discriminator term weighted by $p_\theta(y \mid x)$8 (Yi et al., 2019).

Control enters direct response generation through multiple channels. RRGen conditions on user rating, review length, app category, inferred sentiment, and topic tags (Gao et al., 2020). CODA controls response direction through a specified target sentence and a generated commonsense path (Gupta et al., 2022). Dialogue-act NLG controls the realized act through few-shot prompt styles and ranking functions that combine DA accuracy and semantic accuracy (Ramirez et al., 2023). Meta-word generation controls length, dialogue act, copy ratio, specificity, and other response attributes through interpretable attribute records tracked by memory (Xu et al., 2019). In multi-party dialogue, the addressee must be specified before the response is generated, making addressee conditioning central to the task formulation (Li et al., 2023).

Candidate generation and selection also vary substantially. RRGen and the evaluator-guided spoken-dialog system use beam search (Gao et al., 2020, Yi et al., 2019). Gao et al. sample latent words from clustered candidate pools to encourage diversity across multiple responses (Gao et al., 2018). The diversity-aware T5+BERT system generates seven candidates by combining greedy search, beam search, and repeated top-50 multinomial sampling, or forty-nine candidates in the combined DADE setting, and then chooses $p_\theta(y \mid x)$9 according to the evaluator (Sakaeda et al., 2022). Ramirez et al. overgenerate $y$0 candidates from few-shot prompts and rank them with functions such as $y$1 or the hard-DA variant $y$2 (Ramirez et al., 2023). SCRABLE iteratively regenerates responses under revised prompts until the average judge score reaches a threshold of $y$3 or the maximum number of iterations is reached (Azov et al., 2024).

4. Evaluation methodology and empirical findings

Evaluation in direct response generation is notably heterogeneous. Corpus-level BLEU-4 is used in RRGen, where it measures 1–4 gram overlap between a single human reference response and the model output (Gao et al., 2020). HRAN reports perplexity on development and test sets together with side-by-side human judgments (Xing et al., 2017). Service-assistant work evaluates BLEU, ROUGE-L, BERTScore, Task Success Rate, and human ratings of fluency, relevance, and helpfulness (Bolshinsky et al., 5 Sep 2025). Diversity-aware systems report BLEU-n, distinct-1/2, and human judgments of engagingness or semantic distinctness (Gao et al., 2018, Sakaeda et al., 2022). CODA introduces TARGET-COHERENCE, a reference-less trained metric $y$4, after observing that BLEU, ROUGE, METEOR, and BERTScore correlate poorly with human judgment for target-guided transitions (Gupta et al., 2022).

The empirical record shows that direct response generation can perform strongly when task-specific conditioning is designed carefully. On 58 apps and 309,246 review-response pairs, RRGen achieves BLEU-4 $y$5, compared with $y$6 for attentional NMT, $y$7 for NNGen nearest-neighbor retrieval, and $y$8 for random response selection; the reported improvement over NMT is approximately $y$9 with $x$0 and large effect $x$1 (Gao et al., 2020). In multi-turn dialogue, HRAN reduces dev/test perplexity to $x$2, outperforming S2SA, HRED, and VHRED, and also wins significantly more human pairwise comparisons than it loses against each baseline (Xing et al., 2017).

Target-guided direct generation also benefits from explicit control. CODA reaches TARGET-COHERENCE $x$3 on Otters-ID, versus $x$4 for GPT-2, $x$5 for Multigen, and $x$6 for GPT2-Fudge; on Otters-OOD, CODA reaches $x$7 versus a best baseline of approximately $x$8 (Gupta et al., 2022). Ablations show that removing paths drops TARGET-COHERENCE by more than 4 points, removing DailyDialog data drops it by approximately 1 point, and replacing relation-annotated paths with entities only loses about 3 points (Gupta et al., 2022).

Few-shot controllable generation achieves very high act fidelity. Ramirez et al. report that several prompt settings achieve perfect DA accuracy and near perfect semantic accuracy of $x$9, outperforming few-shot fine-tuning (Ramirez et al., 2023). With instruction-tuned models plus a definitional prompt, reported results include ViGGO $c$0, $c$1, $c$2; TV $c$3, $c$4, $c$5; and Laptop $c$6, $c$7, $c$8 (Ramirez et al., 2023).

Evaluator-guided generation also shows sizable gains. In spoken-dialog response generation, reranking and evaluator-based fine-tuning raise BLEU-4 on the Alexa Prize test set from $c$9 for the baseline seq2seq model to $x=(d,a)$0 for the combined $x=(d,a)$1 system, while human coherence and engagement scores increase from $x=(d,a)$2 to $x=(d,a)$3 (Yi et al., 2019). In the T5+BERT generate-evaluate-select system, pairwise human evaluation on 2,000 held-out single-turn contexts shows that $x=(d,a)$4 beats greedy decoding in $x=(d,a)$5 of cases and loses in $x=(d,a)$6, indicating the practical value of selecting among diverse candidates (Sakaeda et al., 2022).

For customer review response generation with LLMs, SCRABLE reports an LLM-judge average score of $x=(d,a)$7 on the test set for the LLM-optimized prompt, versus $x=(d,a)$8 for the base prompt and $x=(d,a)$9 for a human-optimized prompt; human evaluation increases from $d$0 to $d$1, a reported $d$2 improvement (Azov et al., 2024). In service assistants, direct response generation with fine-tuned T5 is close to a fully fine-tuned intent-first pipeline: on BiToD, direct generation reaches BERTScore/ROUGE-L/BLEU of $d$3 against $d$4 for the two-step system, with Task Success Rate $d$5 versus $d$6; on Bitext, the corresponding Task Success Rates are $d$7 and $d$8 (Bolshinsky et al., 5 Sep 2025). This directly challenges the assumption that explicit intent recognition is always necessary for high-quality service responses.

5. Major application domains

App-review and customer-review response generation is one of the clearest industrial settings for direct response generation. Hassan et al. are cited as finding that responding to a review increases the chances of a user updating their given rating by up to six times compared to not responding, motivating automation (Gao et al., 2020). RRGen is explicitly designed for app-store review response generation, while SCRABLE uses retrieval-augmented generation over app manuals, FAQs, and past responses to produce customer-review replies and iteratively improve prompts (Gao et al., 2020, Azov et al., 2024).

Open-domain dialogue remains the most diverse research domain. HRAN addresses multi-turn chat by focusing on hierarchical attention over context (Xing et al., 2017). Gao et al. study short-text conversation and the generation of multiple diverse replies from a single post, then describe how the same machinery can be adapted to produce a single direct reply (Gao et al., 2018). The generate-evaluate-select system focuses on single-turn Twitter dialogue and explicitly targets engagingness through evaluator-guided selection (Sakaeda et al., 2022). The spoken-dialog evaluator work similarly concentrates on coherence and engagement at each turn (Yi et al., 2019).

Controlled transition generation forms a separate subarea. CODA aims to move a conversation smoothly from a dialogue context toward a target sentence by generating a commonsense bridging path and conditioning generation on that path (Gupta et al., 2022). Dialogue-act-conditioned NLG treats direct response generation as a mapping from semantic intent specifications to utterances in domains such as ViGGO, Laptop, and TV (Ramirez et al., 2023). A plausible implication is that direct response generation encompasses both response selection from conversational context and structured realization from semantic control variables.

Multi-party and service-assistant settings introduce additional task structure. In multi-party dialogue, the addressee of a response utterance should be specified before it is generated, and the EM pre-training framework addresses the lack of annotated addressee labels during pre-training by inferring them iteratively (Li et al., 2023). In service assistants, direct generation is compared directly with intent-first pipelines on BiToD and Bitext Customer Service, covering domains such as hotel booking, restaurant search, ride-hailing, and synthetic QA pairs spanning 27 intents (Bolshinsky et al., 5 Sep 2025).

Several papers also state that the framework can transfer beyond the original task. RRGen notes that the same supervised framework applies to other single-turn feedback-response tasks, including help-desk FAQs and product Q&A, by retraining on task-specific pairs and attributes (Gao et al., 2020). The service-assistant study recommends starting with direct response generation in open-ended or rapidly evolving domains and considering retrieval augmentation or hybrid fallbacks in higher-stakes settings (Bolshinsky et al., 5 Sep 2025).

6. Limitations, controversies, and open directions

A recurring limitation is that many direct response generators are optimized for narrowly defined interaction structures. RRGen only handles the single-reply scenario, described as $d$9 of app-store dialogues, and extending it to multi-turn interaction would require a dialogue-state encoder (Gao et al., 2020). The target-guided work depends on a small benchmark and compensates by re-purposing DailyDialog through SRL-based target extraction and filtering (Gupta et al., 2022). The multi-party setting shows that direct generation may require latent structural inference, here in the form of addressee estimation, before a useful response can be produced (Li et al., 2023).

Another controversy concerns evaluation. CODA explicitly reports that BLEU, ROUGE, METEOR, and BERTScore hover near zero or even negative correlation with human judgment on target-guided response evaluation, motivating TARGET-COHERENCE with correlation $a$0 (Gupta et al., 2022). SCRABLE similarly replaces embedding-based similarity with an LLM-as-a-Judge rubric and reports Pearson correlation $a$1 with human scores on the test set, versus approximately $a$2 for the cited comparator (Azov et al., 2024). These findings make clear that overlap-based metrics are often inadequate for response-generation tasks where multiple semantically acceptable outputs exist.

Direct response generation also raises controllability and reliability issues. The service-assistant study identifies slot omission in structured tasks, hallucination risk under domain shift, and reduced controllability because auditing which intent was recognized is impossible in pure direct generation (Bolshinsky et al., 5 Sep 2025). The diversity-aware generator-evaluator model notes that collecting separate evaluation data for each generator type is costly and that scoring dozens of candidates adds inference cost (Sakaeda et al., 2022). The spoken-dialog evaluator work likewise depends on external evaluators and reranking infrastructure (Yi et al., 2019).

Task-specific noise sources remain important. RRGen notes that SentiStrength may mislabel user sentiment and proposes joint sentiment learning as a future extension; it also relies on generic placeholders such as “<email>” and “<url>” that must be post-processed with an app-specific lookup table (Gao et al., 2020). Dialogue-act-conditioned prompting reports that DA-specified response quality can lag due to skewed DA distributions and imperfect DA classifiers in related setups (Sakaeda et al., 2022), while the few-shot DA study itself depends on a high-accuracy DA classifier and domain-specific slot-error scripts for ranking (Ramirez et al., 2023).

Open directions are stated explicitly across the literature. RRGen suggests enriching context with review-post time, device information, or user-history embeddings and extending the model to multi-turn interaction (Gao et al., 2020). CODA indicates that commonsense paths and augmentation are both essential, implying that richer knowledge planning may remain valuable for controlled dialogue (Gupta et al., 2022). The service-assistant comparison recommends hybrid architectures in high-stakes domains, retaining explicit intent labels for critical flows while using direct response generation for non-critical FAQs (Bolshinsky et al., 5 Sep 2025). Taken together, these works suggest that the main frontier is not whether direct response generation is possible, but how much explicit structure, retrieval, planning, or post-hoc evaluation should be coupled to end-to-end generation in a given domain.