Conversational Infill

• Conversational infill is a conditional generation task that reconstructs missing dialogue segments using observed context and pragmatic reasoning.
• It spans methodologies from text infilling and latent modeling to real-time voice-agent responses and implicit conversational implicature resolution.
• Architectural paradigms like Transformer decoders, ILM techniques, and frame-guided systems enable coherent, context-aware completions under latency constraints.

Searching arXiv for recent and foundational papers relevant to conversational infill and closely related formulations. Conversational infill is a family of conditional generation and inference problems in which missing material in dialogue is reconstructed from partially observed conversational context. In its broadest technical sense, it includes generating missing words, phrases, turns, or multi-turn spans from surrounding dialogue, as well as inferring unstated meaning conveyed pragmatically rather than literally. A foundational formulation comes from general text infilling, where arbitrary portions of text are deleted and a model must generate content so that the final output is complete, semantically coherent, and meaningful (Zhu et al., 2019). In dialogue, this formulation maps directly to missing spans or turns whose locations are known but whose content and length are not (Zhu et al., 2019). Related work also treats conversational implicature as a form of pragmatic completion, where a listener fills in an intended yes/no meaning that is not explicitly stated, and studies real-time voice-agent settings in which a lightweight model must produce fluent conversational material while incorporating delayed knowledge from a backend model (Kim et al., 2023). Together, these lines of work establish conversational infill as a bridge between span-level text generation, dialogue understanding, pragmatic reasoning, and latency-constrained conversational systems.

1. Formal task formulations

The most general formalization derives from text infilling. Given a text “template” where arbitrary portions have been deleted or redacted, the task is to generate text that fills the blanks so that the output is complete, semantically coherent, and meaningful (Zhu et al., 2019). The formulation allows an arbitrary number of blanks, arbitrary unknown blank lengths, and full surrounding context, including both left and right context around each blank (Zhu et al., 2019). In segment notation, the original text is divided into contiguous segments $S=(S_1,\dots,S_n)$, some of which are blank segments collected in $\mathcal{B}$, and the model generates each missing segment as a token sequence beginning with <bob> and typically ending with <eob> (Zhu et al., 2019). The conditional probability for blank $i\in\mathcal{B}$ is factorized autoregressively as

$$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$

with total loss given by the sum of per-blank negative log-likelihoods (Zhu et al., 2019).

For conversational infill, this maps directly to dialogue structure. The known portion is which utterance or span is missing; the unknown portion is how many words or sentences correspond to the missing turn and what their content is (Zhu et al., 2019). This supports infilling at multiple granularities: missing words inside a turn, missing phrases, whole missing turns, or even multi-turn gaps (Zhu et al., 2019). A related formulation appears in sentence-infilling work, where a paragraph $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$ contains one missing sentence $s^{(k)}_{m_k}$, and the model must generate $\hat s^{(k)}_{m_k}$ that makes the full paragraph coherent (Huang et al., 2019). The same structure transfers to dialogue by treating utterances as sentence-level units (Huang et al., 2019).

A second formulation concerns conversational implicature. Here the “missing” object is not a deleted span but an unstated intended meaning. The task is operationalized as binary classification: given a short dialogue and a responder’s utterance, the model must infer the implicated “yes” or “no” meaning (Kim et al., 2023). In this setting, the infill is a pragmatic proposition recovered from literal content, contextual background knowledge, and the assumption that speakers follow Grice’s Cooperative Principle and Maxims (Kim et al., 2023). Thus conversational infill includes both textual reconstruction and interpretation of underspecified conversational meaning.

A third formulation arises in real-time voice systems. Conversational infill there denotes a task in which a small on-device model produces conversational phrases conditioned on a time-indexed stream of external inputs $E=(e_1,\dots,e_n)$, where each $e_i$ is either a backend knowledge chunk $l_i\in L$ or a silence token $\mathcal{B}$0, and emits a phrase sequence $\mathcal{B}$1 with one phrase per external input (Srinivas et al., 10 Nov 2025). When $\mathcal{B}$2 is a knowledge chunk, the paper requires an entailment relation

$\mathcal{B}$3

so that the phrase is grounded in streamed backend content (Srinivas et al., 10 Nov 2025). This version of conversational infill is defined not by reconstructing deleted text, but by producing contextually appropriate dialogue while integrating delayed knowledge.

2. Architectural paradigms

The most direct architecture for general infilling is a self-attention decoder with template attention. In the formulation of "Text Infilling" (Zhu et al., 2019), the best-performing model is a Transformer-based decoder with 6 blocks, 8 heads, and $\mathcal{B}$4, trained to generate one blank at a time while attending to the full template sequence containing observed tokens and placeholders (Zhu et al., 2019). Each decoder layer combines self-attention over previously generated tokens in the current blank and attention over template tokens, enabling bidirectional contextual conditioning because the template includes both left and right context (Zhu et al., 2019). Blanks are filled sequentially, with the template updated after each infill using either ground truth during training or generated text during inference (Zhu et al., 2019).

A key design in this architecture is segment-aware position encoding. Because standard absolute positions are unstable when blank lengths are unknown, each token receives a two-dimensional index $\mathcal{B}$5, mapped into a unique scalar position by

$\mathcal{B}$6

followed by standard sinusoidal positional encoding (Zhu et al., 2019). This allows the model to represent arbitrary-length missing spans within a fixed segment structure. For conversation, seg_id can correspond to turn or utterance index, and offset_id to token position within the utterance (Zhu et al., 2019).

A contrasting decoder-only approach is infilling by language modeling. "Enabling LLMs to Fill in the Blanks" (Donahue et al., 2020) shows that an autoregressive LLM can be adapted to infilling by training on a concatenation of masked text $\mathcal{B}$7, a separator token sep, and the concatenated missing spans $\mathcal{B}$8, separated by answer tokens (Donahue et al., 2020). The model therefore learns $\mathcal{B}$9 using an off-the-shelf GPT-2 with only a few additional special tokens: blank tokens, answer, and sep (Donahue et al., 2020). This formulation supports multiple blanks and variable-length spans without changing the model architecture (Donahue et al., 2020). Because the incomplete text $i\in\mathcal{B}$0 appears before the missing spans $i\in\mathcal{B}$1, the autoregressive model conditions generation on both left and right context in the masked template (Donahue et al., 2020).

Another architecture decomposes infilling into representation learning, discourse planning, and generation. "INSET: Sentence Infilling with INter-SEntential Transformer" (Huang et al., 2019) uses a denoising sentence autoencoder $i\in\mathcal{B}$2, with BERT as encoder and GPT-2 as decoder, and a sentence-level transformer $i\in\mathcal{B}$3 operating over sentence embeddings (Huang et al., 2019). For a missing sentence at position $i\in\mathcal{B}$4, the model uses a zero vector placeholder at that position and predicts a sentence embedding, then decodes it into text: $i\in\mathcal{B}$5 The sentence-level transformer is trained with cosine loss

$i\in\mathcal{B}$6

where $i\in\mathcal{B}$7 is the true sentence embedding (Huang et al., 2019). This architecture cleanly separates local utterance modeling from higher-level discourse planning and transfers naturally to utterance infilling in dialogue (Huang et al., 2019).

A further controlled-generation architecture is frame-guided infilling. "InFillmore: Frame-Guided Language Generation with Bidirectional Context" (Ou et al., 2021) extends infilling with explicit FrameNet control. Using ILM-style input formatting with [blank] and [sep], the model conditions generation on both context and a sequence of frame ID tokens such as [Request], [Food], or [Commerce_buy] (Ou et al., 2021). The paper introduces fine-tuning with frames (FFL) and disjunctive lexically constrained decoding (LCD), enabling either soft semantic control through control tokens or hard control through lexical units associated with frames (Ou et al., 2021). In conversational settings, these frames can act as intent or dialogue-act constraints on missing turns (Ou et al., 2021).

Real-time voice agents motivate yet another architecture. "ConvFill: Model Collaboration for Responsive Conversational Voice Agents" (Srinivas et al., 10 Nov 2025) uses a 360M-parameter on-device model based on SmolLM2-360M-Instruct, extended with <|knowledge|> and <|sil|> tokens (Srinivas et al., 10 Nov 2025). It consumes the current user utterance plus an interleaving of backend knowledge chunks or silence tokens and its own previously generated phrases: $s^{(k)}_{m_k}$0 The model is trained with standard language-model cross-entropy over the last conversational phrase in the sequence (Srinivas et al., 10 Nov 2025). This architecture addresses a different infill problem—responsive phrase-level generation under delayed knowledge—but still treats conversational output as a sequence of context-conditioned infills.

3. Training regimes and synthetic supervision

A recurring feature of conversational infill research is the absence of naturally annotated infill corpora. As a result, most systems rely on synthetic supervision constructed by masking spans in existing corpora. In "Text Infilling" (Zhu et al., 2019), supervised data are built by masking out spans from Yelp reviews, Grimm’s Fairy Tales, and NBA game reports with varying mask rates and blank counts (Zhu et al., 2019). Three regimes are used: random masking over 30–50% of tokens on Yelp with 1 or 2 blanks; long-content infilling where only anchor words such as a noun and a verb, or team names and numerical stats, are kept; and a preposition/article infilling task where all such function words are masked and exactly three blanks are created (Zhu et al., 2019). The training objective is the sum of cross-entropy losses over all blank tokens, optimized with Adam using $i\in\mathcal{B}$8, $i\in\mathcal{B}$9, $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$0, batch size 200, and 150 epochs (Zhu et al., 2019).

ILM-based training similarly creates synthetic supervision by masking spans from complete documents. "Enabling LLMs to Fill in the Blanks" (Donahue et al., 2020) uses a granularity-aware masking function that traverses a hierarchy from documents to paragraphs, sentences, n-grams, and words, masking subtrees with about 3% probability to obtain an overall marginal token mask rate of about 15% (Donahue et al., 2020). Different blank tokens correspond to different granularities, enabling the model to learn length distributions associated with span type (Donahue et al., 2020). The resulting training sequence is always the masked context followed by the concatenated missing spans (Donahue et al., 2020).

INSET uses a two-stage training strategy rather than direct span masking. First, the sentence autoencoder is trained as a denoising model with 15% random token masking at the encoder input and negative log-likelihood reconstruction at the decoder (Huang et al., 2019). Then sentence embeddings are precomputed, and the sentence-level transformer is trained to predict the embedding of the missing sentence in a 7-sentence window, always masking the 4th sentence (Huang et al., 2019). This stage isolates discourse planning from surface realization (Huang et al., 2019).

Conversational ellipsis work introduces direct supervision for completion. "Filling Conversation Ellipsis for Better Social Dialog Understanding" (Zhang et al., 2019) constructs a dataset of 2,258 user utterances from Alexa Prize social conversations, with 1,124 manually completed elliptical utterances and 204 complete utterances modified into elliptical versions (Zhang et al., 2019). A pointer-generator sequence-to-sequence model with 2-layer LSTM encoder and decoder, hidden size 500, dropout 0.3, and OpenNMT training is then used to map elliptical utterances to completed ones (Zhang et al., 2019). This is one of the few settings in which conversational infill supervision is directly human-authored.

Another synthetic regime appears in ConvFill. Because existing conversational corpora are filled with simple “um/uh” fillers and lack the desired structure for responsive voice agents, "ConvFill" (Srinivas et al., 10 Nov 2025) generates 5,997 synthetic multi-domain conversations with GPT-4o across advice, assistant queries, education, event planning, customer service, and medical conversation (Srinivas et al., 10 Nov 2025). Each conversation contains 8–12 turns, and each turn stores a user utterance, a list of assistant phrases, and a parallel list of “thoughts or silence” that serve as backend knowledge chunks (Srinivas et al., 10 Nov 2025). A DeBERTaV3-base MNLI classifier is then used to verify that each knowledge chunk entails its associated conversational phrase (Srinivas et al., 10 Nov 2025).

Training data can also be generated through explicit reasoning over information gaps. "Bridging Information Gaps with Comprehensive Answers" (Liu et al., 24 Feb 2025) constructs synthetic follow-up questions by first producing a hypothetical comprehensive answer to an initial question, then identifying information present in the comprehensive answer but not in the initial answer, and finally generating follow-up questions that are answerable by the comprehensive answer but not by the initial answer (Liu et al., 24 Feb 2025). Although this work studies question generation rather than direct span reconstruction, it is relevant because it treats conversational progress as targeted infilling of information gaps (Liu et al., 24 Feb 2025).

4. Pragmatics, ellipsis, and discourse-level completion

Conversational infill is not limited to reconstructing missing text; it also includes recovering missing meaning and resolving under-specification. "Is the Pope Catholic? Applying Chain-of-Thought Reasoning to Understanding Conversational Implicatures" (Kim et al., 2023) explicitly frames implicature as a missing piece that the listener must infer. The paper adopts Grice’s Cooperative Principle and the maxims of Quality, Quantity, Relevance, and Manner, and shows that LLMs improve substantially on implicature interpretation when prompted to reason step by step about literal content, apparent maxim violations, background knowledge, and the intended yes/no answer (Kim et al., 2023). On BIG-bench implicature (492 items), accuracy rises from 0.68 with zero-shot prompting to 0.91 with chain-of-thought prompting; on the LUDWIG dataset (600 items), accuracy rises from 0.63 to 0.87, surpassing the reported average human accuracy of 86.2% on LUDWIG (Kim et al., 2023). This establishes a pragmatic dimension of conversational infill: the reconstruction of implied communicative content rather than literal spans.

A related but more controlled benchmark is "DRInQ: Evaluating Conversational Implicature with Controlled Context Variation" (Arai et al., 22 May 2026). Each instance is a triple $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$1, where $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$2 is a fixed question utterance, $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$3 is a context, and $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$4 is a set of candidate implied comments with exactly one correct answer (Arai et al., 22 May 2026). By holding $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$5 fixed and varying only $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$6, DRInQ isolates pragmatic contextualization. The benchmark is built with 300 base questions, at least 5 distinct intents per question, and 3 context variants per intent, followed by human validation retaining 819 instances with at least 4-of-5 annotator agreement and a 400-item hard subset (Arai et al., 22 May 2026). On the hard subset, the best model reaches 0.67 accuracy, GPT-4o reaches $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$7, and human annotators reach $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$8 (Arai et al., 22 May 2026). The paper reports a generation–inference asymmetry: models can generate plausible contexts for intended implicatures but are less reliable at recovering the intended implicature from context alone (Arai et al., 22 May 2026). This suggests that pragmatic infill remains difficult even when fluent conversational generation appears strong.

Ellipsis is another discourse-level completion problem. "Filling Conversation Ellipsis for Better Social Dialog Understanding" (Zhang et al., 2019) estimates that about 50% of 2,000 sampled Alexa Prize social-conversation utterances contain some degree of ellipsis (Zhang et al., 2019). The paper uses a pointer-generator completion model to rewrite elliptical utterances into completed forms and then combines downstream predictions from the original and completed utterances through a hybrid selection mechanism (Zhang et al., 2019). For dialog act prediction, the hybrid system improves F1 from 79.65 to 80.90, a gain of about 1.3 points; for semantic role labeling, it improves F1 from 88.39 to 90.87, a gain of about 2.5 points (Zhang et al., 2019). The central conclusion is that automatic completion is useful but noisy, so the completed utterance should be treated as a complementary signal rather than a replacement for the original (Zhang et al., 2019).

A different route to conversational completion uses interaction itself as infill. "Insert-expansions for Tool-enabled Conversational Agents" (Göldi et al., 2023) adapts the conversation-analytic notion of insert-expansion: an inserted mini-sequence between the user’s request and the system’s final answer that clarifies intent or scopes the response (Göldi et al., 2023). The paper implements “user-as-tool” operations such as clarify_intent, scope_response, and enhance_appeal in a LangChain-style GPT-3.5-turbo-0301 agent (Göldi et al., 2023). In recommendation scenarios, these insert-expansions improve perceived control, and in a grounded recommendation scenario they improve both control and naturalness (Göldi et al., 2023). Although this is not infilling in the masked-span sense, it treats intermediate user queries as conversational infill that supplies missing parameters for a preferred final response.

Knowledge-graph conversational QA provides another non-generative perspective. "Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion" (Christmann et al., 2019) addresses follow-up questions that omit entities or predicates. The method maintains a context subgraph $$p\big(S'_i \mid S_{\text{template}_i}\big) = \prod_{j=0}^{o_i} p\big(x'_{(i,j)} \mid x'_{(i,0)}, \dots, x'_{(i,j-1)}, S_{\text{template}_i}\big),$$9, computes lexical matching $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$0, graph proximity $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$1, and KG prior $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$2, and combines them into a frontier score

$p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$3

then expands the graph judiciously and ranks candidate answers by proximity both to the frontier and to prior Q/A entities (Christmann et al., 2019). The paper treats conversational infill as completing omitted entities, relations, and constraints in graph space rather than by rewriting the user utterance (Christmann et al., 2019).

5. Semantic control, style, and local conversational phenomena

Conversational infill often requires more than coherence; it may require controllable semantics, style transfer, grammatical repair, or natural placement of minimal discourse material such as backchannels and fillers. "InFillmore" (Ou et al., 2021) addresses semantic control directly through FrameNet frames. In story-style sentence infilling, frame-guided generation improves frame recall from about 0.165–0.169 for unguided ILM to as high as 0.640 for FFL, with perfect recall up to 0.259, while maintaining plausibility in human evaluation (Ou et al., 2021). Because frames such as Request, Giving, Emotion_directed, and Contacting align closely with dialogue acts, the same mechanism can be used to steer missing turns toward desired communicative functions (Ou et al., 2021).

Controlled rewriting through masking also bears on conversational infill. "Mask and Infill: Applying Masked LLM to Sentiment Transfer" (Wu et al., 2019) casts sentiment transfer as masking sentiment-bearing spans and infilling them with an Attribute Conditional MLM conditioned on target sentiment (Wu et al., 2019). The reconstruction loss is

$p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$4

and an additional discriminative loss

$p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$5

is combined via

$p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$6

The method achieves strong gains on Yelp and Amazon review sentiment transfer, with human ratings on Yelp of 4.2 grammaticality, 4.0 content preservation, and 4.4 attribute match for AC-MLM-SS + fusion, compared with 3.4, 3.5, and 3.6 for DeleteAndRetrieval (Wu et al., 2019). A plausible implication is that span-selective, attribute-conditioned infilling offers a concrete template for dialogue editing tasks such as politeness rewriting or tone-controlled response completion.

At the micro-conversational level, backchannels and fillers are themselves a form of local infill. "Investigating the Representation of Backchannels and Fillers in Fine-tuned LLMs" (Wang et al., 24 Sep 2025) studies whether fine-tuning improves LM representations of items such as “yeah”, “uh-huh”, “right”, “okay”, “uh”, and “um”, as well as Japanese aizuchi and hesitation markers (Wang et al., 24 Sep 2025). Using clustered contextual embeddings and silhouette scores, the paper finds that fine-tuning generally increases representational structure for such items. It also shows dramatic changes in generation behavior: for English LLaMA‑3, backchannel/filler frequency rises from 4.29% to 18.61% after fine-tuning, diversity rises from 73 to 83 unique BC/F types, and BC/F perplexity drops from 197.67 to 5.30 (Wang et al., 24 Sep 2025). The paper argues that backchannels and fillers should be treated as central conversational material, not stripped from the data (Wang et al., 24 Sep 2025). This is directly relevant to conversational infill, because systems that need to insert “uh-huh”, “right”, or “um” at the appropriate time require both good contextual representations and training data in which such phenomena are preserved.

The survey "Fillers in Spoken Language Understanding" (Dinkar et al., 2023) adds psycholinguistic and systems context. It characterizes fillers as highly frequent disfluencies such as “uh”, “um”, “er”, and “ah”, notes that natural conversations contain about 5–10% disfluencies, and reviews evidence that listeners treat fillers as informative cues rather than pure noise (Dinkar et al., 2023). Fillers signal planning difficulty, affect lexical prediction, and can modulate interpretation of certainty and stance (Dinkar et al., 2023). This suggests that local conversational infill by fillers and hesitations is not merely ornamental; it can serve discourse management and listener modeling functions.

6. Empirical performance and system-level implications

Across span-level infilling tasks, self-attention with bidirectional context consistently outperforms weaker baselines. In "Text Infilling" (Zhu et al., 2019), the self-attention model beats seq2seq and GAN baselines on Yelp random masking and on long-content infilling in Grimm’s Fairy Tales and NBA reports (Zhu et al., 2019). On Grimm, self-attention reaches perplexity 9.65 and human score 2.664, compared with 10.41/1.991 for seq2seq and 11.78/1.338 for GAN; on NBA, it reaches 6.54/2.273, compared with 10.30/1.909 for seq2seq and 7.25/1.818 for GAN (Zhu et al., 2019). Performance degrades as mask rate and number of blanks increase, confirming that unknown-length multi-span completion is intrinsically difficult (Zhu et al., 2019).

ILM demonstrates that decoder-only LLMs can perform infilling competitively with minimal changes. In "Enabling LLMs to Fill in the Blanks" (Donahue et al., 2020), sentence-level perplexity on short stories is 15.6 for ILM, matching LM-All and improving over a plain LM at 18.3 and a reverse LM at 27.1 (Donahue et al., 2020). In human evaluation, annotators fail to identify ILM-generated sentences as machine-generated 45% of the time, compared with 41% for the LM baseline, 29% for the self-attention baseline from Zhu et al., and 20% for BERT; the effective upper bound is 80% because one of five sentences must always be chosen (Donahue et al., 2020). This indicates that efficient decoder-only infilling can reach humanlike local plausibility in story contexts (Donahue et al., 2020).

At the sentence level, INSET yields gains over a self-attention baseline on both TripAdvisor and Recipe datasets. On TripAdvisor without keywords, INSET improves BLEU‑1 from 16.34 to 21.25, BLEU‑4 from 0.54 to 0.96, NIST‑4 from 0.54 to 1.23, and METEOR from 5.85 to 7.04 (Huang et al., 2019). Human raters also prefer INSET over the baseline on coherence, fluency, and informativeness (Huang et al., 2019). This supports the claim that explicit discourse-level planning helps bridge surrounding context at the missing-sentence level.

For semantic control, InFillmore shows that frame guidance improves both fidelity and predictive ease. In a one-blank setting, infill token perplexity drops from 12.85 for ILM to 6.19 for A-FFL and 5.05 for A-FFL-ordered (Ou et al., 2021). Human evaluation of relative plausibility also favors frame-guided models when using richer frame sets (Ou et al., 2021). This suggests that symbolic semantic control can be layered onto bidirectional infilling without large losses in naturalness.

The strongest system-level results appear in real-time conversational agents. ConvFill is specifically evaluated on latency and factual usefulness under backend delay. On a 250-question NaturalQuestions subset, standalone small models such as SmolLM2-360M, Qwen3-0.6B, and Gemma3-270M achieve accuracies of 0.10, 0.12, and 0.06, respectively, whereas ConvFill with backend models reaches 0.46 with GPT‑5, 0.52 with Claude 4.5, and 0.49 with Gemini‑2.5‑Pro (Srinivas et al., 10 Nov 2025). Time-to-first-token remains around $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$7 seconds for ConvFill+backend, comparable to standalone small models and far below backend-only latencies such as $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$8 seconds for Claude 4.5 and $p^{(k)}=(s^{(k)}_1,\dots,s^{(k)}_{M_k})$9 seconds for Gemini‑2.5‑Pro (Srinivas et al., 10 Nov 2025). The paper therefore reports accuracy improvements of 36–42 percentage points over a standalone 360M model while consistently retaining sub-200ms response latency (Srinivas et al., 10 Nov 2025). Contradictions in turn-level entailment remain relatively low, between 5.3% and 7.7%, though many outputs are neutral rather than entailed (Srinivas et al., 10 Nov 2025). A plausible implication is that conversational infill can decouple responsiveness from backend reasoning quality, but grounding remains an open issue.

On the dialogue-understanding side, ellipsis completion improves downstream task performance, but only when combined carefully with the original utterance. The Hybrid-EL-CMP framework shows that automatic conversational completion is best treated as noisy infill that augments, rather than replaces, the raw input (Zhang et al., 2019). On pragmatic interpretation benchmarks, chain-of-thought prompting grounded in Gricean reasoning can produce large gains, yet DRInQ shows that even strong models remain well below human performance on context-sensitive implicature inference (Kim et al., 2023).

7. Limitations and unresolved questions

Several limitations recur across the literature. First, most infilling models lack explicit discourse structure. The self-attention infiller in "Text Infilling" (Zhu et al., 2019) treats text as a flat sequence of segments and does not model speaker roles, dialogue acts, or dialogue state (Zhu et al., 2019). INSET’s sentence-level transformer likewise models inter-sentential structure implicitly rather than through explicit conversational variables (Huang et al., 2019). InFillmore adds symbolic frames, but frame parsers may degrade on colloquial dialogue, and LCD can become overly rigid when too many frames are enforced (Ou et al., 2021).

Second, many results are based on non-dialogue corpora. Text infilling, ILM, INSET, and InFillmore are demonstrated mainly on reviews, stories, recipes, lyrics, abstracts, or ROCStories (Zhu et al., 2019). Conversational transfer is often argued rather than empirically established. A plausible implication is that adaptation to dialogue requires new corpora with turn structure, speaker identity, and pragmatic phenomena preserved.

Third, automatic completion and pragmatic inference are both brittle. In ellipsis completion, pointer-generator outputs may misrepresent user intent, which is why hybrid models outperform completion-only systems (Zhang et al., 2019). In implicature understanding, even when prompts improve benchmark scores, the task is often reduced to binary yes/no decisions or multiple-choice selection (Kim et al., 2023). DRInQ further shows that models tend to over-attribute negative intent, over-fixate on incidental contextual details, and generate stronger implicatures than humans endorse (Arai et al., 22 May 2026).

Fourth, synthetic data are both enabling and constraining. ConvFill depends entirely on synthetic dialogues and six domains (Srinivas et al., 10 Nov 2025). The comprehensive-answer follow-up question work depends on GPT‑4o for generating rich synthetic supervision (Liu et al., 24 Feb 2025). DRInQ’s contexts and candidate interpretations are partly LLM-generated, even though human filtering is used (Arai et al., 22 May 2026). This suggests that conversational infill pipelines often inherit the biases and coverage of the data-generation model.

Fifth, evaluation remains fragmented. Span reconstruction work uses perplexity, BLEU, NIST, METEOR, or human detectability (Zhu et al., 2019). Pragmatic work uses accuracy against consensus labels (Kim et al., 2023). Real-time systems use latency, QA accuracy, and entailment proportions (Srinivas et al., 10 Nov 2025). Backchannel research uses silhouette scores, BC/F-specific perplexity, BERTScore, and BLEU (Wang et al., 24 Sep 2025). These metrics are not interchangeable, and many do not fully capture whether a completion is interactionally appropriate. This suggests that conversational infill lacks a unified evaluation framework spanning literal reconstruction, pragmatic interpretation, timing, and social appropriateness.

Finally, different strands of the literature define “conversational infill” at different levels. In one line, it is span reconstruction in dialogue-like text (Zhu et al., 2019). In another, it is pragmatic completion of implied meaning (Kim et al., 2023). In another, it is real-time phrase generation that fills backend latency with grounded or filler dialogue (Srinivas et al., 10 Nov 2025). These notions are compatible, but they target different computational bottlenecks: unknown text spans, missing implicit meaning, or missing timely content. A plausible implication is that future work may need modular systems combining all three: structural infilling for omitted text, pragmatic reasoning for unstated meaning, and latency-aware phrase planning for spoken interaction.