IterChat: Iterative Conversational Paradigm

Updated 7 July 2026

IterChat is a family of iterative conversational systems that update explicit intermediate states—such as structured memos and dialogue embeddings—to maintain context.
It leverages varied representations, including MemoChat’s topic-organized memos, ChatIR’s dialogue-conditioned queries, InterChat’s visual descriptors, and PuppetChat’s action contexts.
IterChat addresses one-shot interaction limitations by enabling incremental refinement through instruction tuning, multi-round questioning, and multimodal engagement.

Searching arXiv for the cited papers to ground the article in the current record. IterChat denotes an iterative conversational paradigm in which a system does not treat interaction as a single-shot query-response event, but instead updates an intermediate conversational state and uses that state to determine the next move. In the cited literature, that intermediate state takes several forms: a structured memo for long-range open-domain conversation in MemoChat, a dialogue-conditioned query representation for image retrieval in ChatIR, a structured prompt built from direct manipulation and natural language in InterChat, and a reciprocity-aware, history-aware action context in PuppetChat (Lu et al., 2023, Levy et al., 2023, Chen et al., 6 Mar 2025, Wang et al., 23 Feb 2026). This suggests that “IterChat” is best understood as a family of iterative interaction patterns rather than a single fixed architecture.

1. Problem setting and conceptual scope

The motivating problem across these systems is that one-off interaction is often too weak for tasks that require memory, clarification, multimodal reference, or continuity. MemoChat frames long-range conversation as generative question answering over prior conversation history $\mathbf{x}^h$ and the current user query $\mathbf{x}^q$ , and argues that a plain LLM with a limited context window quickly loses access to earlier content. The paper further argues that simply expanding the window is not a complete solution, because consistency failures mostly come from forgetting or mis-recalling earlier facts, especially when topics change several times and later questions refer back to much earlier material (Lu et al., 2023).

ChatIR identifies an analogous limitation in image retrieval. Traditional retrieval is described as single-shot: the user gives one text query, the system returns ranked images, and if the result is wrong the user manually reformulates the query. The paper argues that a single caption often does not fully capture the search intent, so ChatIR reframes retrieval as a multi-round conversation in which follow-up questions progressively reduce ambiguity and narrow the candidate set (Levy et al., 2023).

InterChat extends the iterative framing to generative visual analytics. Its central claim is that pure natural-language interaction is flexible but often too ambiguous and verbose for complex exploratory analysis, especially when users are reasoning about visual patterns already on the screen. The paper therefore treats multimodal interaction—direct manipulation plus language—as necessary for expressing analytical and interaction intents precisely during progressive exploratory analysis (Chen et al., 6 Mar 2025).

PuppetChat applies the same broad logic to dyadic messaging. It argues that existing instant messaging systems support mostly single-direction reactions, limited expressive depth, weak continuity across turns, and only weak ties to personal history and shared memories. The resulting design treats chat as an ongoing sequence of mutually responsive actions, each optionally grounded in personal narrative context and preserved as a revisitable trace (Wang et al., 23 Feb 2026).

2. Iterative loop structures

The systems instantiate iterative chat through different update loops, but all treat each turn as input to a subsequent inference step.

System	Intermediate state	Iterative loop
MemoChat	structured on-the-fly memo	memorization–retrieval–response
ChatIR	dialogue history $D_i$	question generation and reranking
InterChat	structured prompt with descriptors and links	manipulation, linking, visualization generation
PuppetChat	action context plus micronarratives	recommendation, response, replay

MemoChat formalizes long-range conversation as a decomposition:

$\mathbf{x}^m = f(\mathbf{x}^h), \qquad \mathbf{x}^{h'} = g(\mathbf{x}^q,\mathbf{x}^m),$

followed by response generation:

$\hat{Y}=\arg\max_Y p(Y \mid \mathbf{x}^q, \mathbf{x}^{h'}).$

Here $f$ builds a structured memo from the history, and $g$ retrieves query-relevant evidence from that memo. The paper explicitly identifies this as an iterative “memorization-retrieval-response” cycle for maintaining consistency over long conversations (Lu et al., 2023).

ChatIR formalizes the dialogue state as

$D_i := (C, Q_1, A_1, \ldots, Q_i, A_i),$

with an LLM questioner $G$ generating the next question $Q_{i+1}$ from the full dialogue and a retriever $\mathbf{x}^q$ 0 encoding the same history into a query embedding for image ranking by cosine similarity. The paper emphasizes that the whole history is kept, not just the latest question-answer pair (Levy et al., 2023).

InterChat implements a closed loop in which direct manipulations are translated into descriptors, language references are linked to those descriptors, and the resulting structured prompt is passed to a visualization agent that generates D3.js code and an explanation. The rendered visualization then remains interactive, so subsequent direct manipulation or text becomes part of the next prompt state (Chen et al., 6 Mar 2025).

PuppetChat organizes interaction as action-plus-caption exchange. A user sends an action, the partner sees it as a prompt, the partner responds with a complementary or mirrored action, and the exchange can be retained or replayed if desired. The paper describes this as a system intended to keep the thread going rather than terminate it at each message (Wang et al., 23 Feb 2026).

3. Memory, retrieval, and grounded state representations

A defining property of IterChat systems is the use of explicit intermediate representations instead of raw history alone. MemoChat’s memo is a structured on-the-fly memory organized by topic, with JSON-like fields such as topic, summary, and dialogues; in the memo-writing task it also includes span positions start and end for topic boundaries. Retrieval is performed over memo entries’ topics and summaries rather than over the full transcript, and the retrieval task includes an explicit NOTO (“None of the others”) option for genuinely new topics (Lu et al., 2023).

ChatIR uses a different form of state. The dialogue history is encoded into a shared embedding space, and images are ranked by cosine similarity between the dialogue embedding and image embeddings. The retriever is based on BLIP image/text encoders fine-tuned for dialogue-based retrieval, with the dialogue sequence concatenated using $\mathbf{x}^q$ 1 and $\mathbf{x}^q$ 2 tokens so self-attention can aggregate the entire sequence (Levy et al., 2023).

InterChat makes intermediate state semantically interpretable. Each direct manipulation is converted into a concise descriptor by $\mathbf{x}^q$ 3, including data-related element selection, axis-range selection, and free drawing. For free drawing, the system captures a screenshot of the canvas and a vision LLM produces a textual description of the sketch, such as “an arrow indicating a steady upward trend.” A second agent, $\mathbf{x}^q$ 4, then links trigger phrases in the user’s text—such as “this trend,” “those bars,” or “the selected data”—to the manipulation descriptors using order matching, content matching, and flexible matching (Chen et al., 6 Mar 2025).

PuppetChat uses a structured action library rather than a memo or prompt graph. Actions are stored as objects with attributes including name, description, keywords, emotion, interaction role, embedding array, and reactionCandidates. Micronarratives are generated from three sources: the user’s personal story, the selected action, and the surrounding conversational context. The paper states that keywords in the personal narrative act as persona anchors, helping maintain a stable voice across turns while adapting to action and context (Wang et al., 23 Feb 2026).

These systems differ in representation, but the shared pattern is explicit state construction. This suggests that iterative chat, in these papers, depends less on appending more tokens to a context window than on building a task-specific representation that can be searched, linked, or reused.

4. Instruction tuning, prompting, and multi-agent orchestration

MemoChat’s main methodological novelty is instruction-tuning LLMs to carry out each stage of the loop reliably. It reconstructs about 10k instruction examples from TopiOCQA, DialogSum, and Alpaca-GPT4. Memo Writing instruction tuning teaches the model to identify all possible topics, determine the span for each topic, summarize each sub-dialogue, and output everything in JSON format, using 1,790 examples from TopiOCQA and 1,256 examples from DialogSum. Memo Retrieval instruction tuning uses 1,719 examples from TopiOCQA and 1,935 examples from DialogSum; the query is created from several sequential sentences, candidate topics come from the conversation, and NOTO is inserted with 10% probability. Chat w/ Memo instruction tuning uses 1,602 examples from DialogSum and 1,698 examples from Alpaca-GPT4. The paper emphasizes instruction templates with a task introduction, a task-specific input body, and a detailed task explanation placed at the end (Lu et al., 2023).

ChatIR separates question generation from retrieval. The questioner module can be instantiated with few-shot LLM prompting, including FLAN-T5-XXL, FLAN-ALPACA-XL, FLAN-ALPACA-XXL, and ChatGPT, while the retriever is trained with a Recall@K surrogate loss. Optimization details reported for the retriever include AdamW, learning rate $\mathbf{x}^q$ 5, exponential decay $\mathbf{x}^q$ 6 down to $\mathbf{x}^q$ 7, batch size 512, 36 epochs, and about 114 seconds per epoch on four NVIDIA A100 nodes. An ablation over masking strategies found that masking captions during training gives the best result, because it forces the model to use the dialogue rather than over-rely on the caption (Levy et al., 2023).

InterChat is explicitly a multi-agent LLM pipeline. $\mathbf{x}^q$ 8 generates manipulation descriptors from direct manipulation, $\mathbf{x}^q$ 9 links language trigger phrases to those descriptors, and $D_i$ 0 generates D3.js code from a structured prompt containing dataset schema, task context, inferred interaction descriptors, user text and trigger links, and chain-of-thought generation instructions. The paper states that the generation is decomposed into chart and field selection, code generation, layout refinement, and code verification (Chen et al., 6 Mar 2025).

PuppetChat combines heuristics, embeddings, and LLM annotations in its recommendation pipeline. Its global action score

$D_i$ 1

uses fixed weights $D_i$ 2, $D_i$ 3, and $D_i$ 4. Text-based scoring combines keyword extraction with negation handling, emotional alignment, and embedding-based fuzzy matching. Context-based scoring uses the partner’s most recent action to query complementary responses, with matching candidates receiving a +5 bonus, and responsive actions receiving a +1 bonus when the conversational state indicates a reply is appropriate (Wang et al., 23 Feb 2026).

5. Evaluation paradigms and empirical findings

The empirical literature on IterChat is heterogeneous because the target tasks differ substantially. MemoChat introduces MT-Bench+, an expert-annotated benchmark built by expanding the original two-turn MT-Bench questions into four-turn versions, sampling 2–4 questions from every 3–4 of those four-turn conversations to form a long chat stream of about 12–15 turns, and appending a long-range question at the end. MT-Bench+ contains 18 questions each for retrospection, continuation, and conjunction, for 54 total. Downstream answers are judged by GPT-4 on a 1–100 faithfulness scale. The paper reports that MemoChat substantially improves consistency over prior memory-based baselines and gives much better average GPT-4 consistency scores than MPC and MemoryBank, roughly 7 to 19.5 points higher on average. It also notes that conjunction remains the hardest scenario (Lu et al., 2023).

ChatIR evaluates iterative retrieval on VisDial with a corpus of 50,000 unseen COCO images, reporting Hits@10 and Average Target Rank. Using ChatGPT as questioner, BLIP2 as answerer, and a trained dialogue retriever, the system reaches 78.3% Hit@10 after 5 dialogue rounds and 81.3% after 10 rounds. The reported single-shot text-to-image retrieval baseline is about 64%, while human-asked questions are about 75% after 5 rounds. The paper further reports that ChatGPT questioners perform best overall, RL-based visual-dialog questioners are among the worst, and stronger questioners show a near-monotonic decrease in average target rank across rounds (Levy et al., 2023).

InterChat combines usage scenarios, a controlled user study, and expert feedback. In the user study, 10 participants from the visualization community, each completing 10 tasks under multimodal and language-only conditions, showed task completion time reductions of 18% for simple tasks and 15% for complex tasks, with average retries dropping from 0.79 to 0.32. The paper also reports that multimodal interaction improved visualization correctness and intent accuracy especially for complex tasks, while intent accuracy for simple tasks was slightly lower with multimodal interaction. In a failure analysis of 100 task executions, there were 26 failed intent interpretations, 4 failed visualization generations, and 12 functional but imperfect visualizations (Chen et al., 6 Mar 2025).

PuppetChat evaluates iterative, reciprocal messaging through a 10-day field study with 11 dyads and 22 participants. Logged usage includes an average of 18.0 action-button presses, 14.5 actions sent, about 80% trigger-to-send conversion, 8.2 messages with micronarratives, about 57% of sent actions carrying captions, 13.2 edit operations, and 9.7 preview opens. The authors report a 16-item BASIC scale with subscale reliability $D_i$ 5 from .78 to .86 and full-scale reliability .93. Mean ratings on a 7-point scale were 5.3 for Co-presence and Affective Intimacy, 5.1 for Reciprocal Interaction, 5.2 for Personalized Micronarratives, 5.4 for Agency and Persistence, and 5.26 overall; SUS was 86.59 and RTLX was 31.72 (Wang et al., 23 Feb 2026).

6. Misconceptions, limitations, and research implications

One recurring misconception is that iterative chat is merely “chat with memory” in a vague sense. MemoChat explicitly rejects that reduction: its contribution is not generic memory attachment but a specifically trained memorization–retrieval–response cycle in which the model learns to write structured memos, retrieve relevant memo entries, and answer using those retrieved memories. The paper also states that MemoChat is better than earlier memory systems because it does not just track persona or isolated events; it tries to summarize all important facts in the conversation history in a topic-organized memo (Lu et al., 2023).

A second misconception is that more interaction modalities are always better. InterChat’s user study found that multimodal interaction improved efficiency and correctness especially for complex tasks, but for simple one-shot requests participants often preferred plain language because multimodal interaction felt unnecessary, and intent accuracy for simple tasks was slightly lower under multimodal interaction. The paper also identifies limitations in latency, hallucinations, limited computational reasoning, narrow interaction modality scope, and missing output verification (Chen et al., 6 Mar 2025).

A third misconception is that iterative questioning alone guarantees better retrieval. ChatIR reports strong improvements, but it also identifies clear failure modes: redundant or repetitive questions, questioners that fail to use dialogue history, poorly informative answers, and ambiguity that remains underdetermined. The paper further notes a limitation in that the questioner currently uses only the dialogue history; a better version might also inspect current retrieval candidates, ranking uncertainty, and discriminative differences among top candidates (Levy et al., 2023).

A fourth misconception is that narrative grounding automatically produces relational authenticity. PuppetChat’s study reports positive results for social presence, self-disclosure, continuity, and shared-memory formation, yet the weakest BASIC item was Connection with $D_i$ 6, indicating that AI-generated micronarratives did not always strongly connect to actual past experiences. The paper is therefore explicit that editability, user control, and optional persistence are central design requirements, not secondary interface features (Wang et al., 23 Feb 2026).

Taken together, the literature suggests several durable design principles for IterChat. A plausible implication is that strong iterative systems tend to combine explicit state construction, targeted next-step inference, user-visible grounding, and mechanisms for progressive refinement. In the surveyed works, those mechanisms appear as structured memo compression and retrieval, dialogue-conditioned clarification, contextual interaction linking, reciprocity-aware action recommendation, editable micronarratives, and optional replay. The resulting picture is not of a single universal architecture, but of a research program organized around the claim that conversation becomes more effective when each turn is interpreted as part of an accumulating, task-relevant process rather than as an isolated prompt.