Improving Multi-turn Dialogue Consistency with Self-Recall Thinking

Abstract: LLM based multi-turn dialogue systems often struggle to track dependencies across non-adjacent turns, undermining both consistency and scalability. As conversations lengthen, essential information becomes sparse and is buried in irrelevant context, while processing the entire dialogue history incurs severe efficiency bottlenecks. Existing solutions either rely on high latency external memory or lose fine-grained details through iterative summarization. In this paper, we propose Self-Recall Thinking (SRT), a framework designed to address long-range contextual dependency and sparse informative signals in multi-turn dialogue. SRT identifies helpful historical turns and uses them to generate contextually appropriate responses, enabling the model to selectively recall and reason over context during inference. This process yields an endogenous reasoning process that integrates interpretable recall steps without external modules. SRT incorporates: (1) Dependency Construction: Generating and converting it into self-recall chains; (2)Capability Initialization: Training to enable reasoning chains with recall tokens capability; (3)Reasoning Improvement: Refining accuracy via verifiable rewards to optimize recall and reasoning for correct answers. Experiments on multiple datasets demonstrate that SRT improves F1 score by 4.7% and reduces end-to-end latency by 14.7% over prior methods, achieving a balance between reasoning latency and accuracy, and outperforming state-of-the-art baselines.

• The paper introduces Self-Recall Thinking (SRT), which trains language models to copy relevant historical turns into their reasoning trace using supervised fine-tuning and verifiable GRPO rewards.
• SRT outperforms six external-memory and latent-reasoning baselines across SRQA, CoQA, and SimpleQA, achieving average F1 gains of 3.4% and latency reductions of 13.1%.
• The method improves long-dialogue recall without external retrieval, but missing or incorrect recall increases with conversation length and reasoning failures remain common even when evidence is correctly cited.

Overview

Multi-turn dialogue systems built on LLMs face a structural tension: as conversations lengthen, the information required to answer a given query becomes sparse relative to the growing context, while processing the full history incurs attention dilution and rising latency. Existing remedies either attach external memory modules (graph stores, episodic logs, retrieval pipelines), which add system latency and deployment complexity, or learn compressed latent states, which progressively lose fine-grained details. The paper proposes Self-Recall Thinking (SRT), a framework that internalizes recall into the model's own chain-of-thought: during inference, the model emits <HIS> tags that verbatim-copy relevant historical turns into its reasoning trace, then conditions its answer on this explicitly cited evidence. The result is an endogenous, auditable recall mechanism requiring no external retriever or index.

Method

SRT is realized in three stages. Stage 0 (Dependency Construction) curates training data via a dependency-aware filtering pipeline: dialogues of 8–32 turns are retained only if they contain a query–history pair whose combined semantic-similarity and exponentially decayed distance score exceeds 0.6 (decay rate 0.15). Claude 3.7 Sonnet then annotates each turn with a minimal recall set $H_t^*$, a grounded reasoning chain $Z_t$, and an answer $A_t^*$, filtered by string-matching verification for answer–evidence alignment.

Stage 1 (Capability Initialization) performs supervised fine-tuning in two phases: bootstrapping on dependency-pruned histories to teach recall timing and target selection, followed by transfer to full-context reasoning under a standard negative log-likelihood objective on Qwen2.5-7B.

Stage 2 (Reasoning Improvement) applies GRPO with a fully verifiable composite reward combining three terms: a binary format reward enforcing well-formed <HIS> tags; a scaled Jaccard recall reward over $[-1.25, 1.25]$ that symmetrically penalizes missing and over-recall; and a cosine-similarity answer reward over $[-1, 1]$. Because all components are rule-based against ground-truth annotations, no learned reward model is involved. A KL term anchors the policy to the SFT checkpoint to prevent drift.

Formally, self-recall is modeled as a pointer policy $\pi_\theta$ that takes $N$ sequential recall actions $o_{1:N}$ (each copying one utterance) before generating the answer, with the joint probability factorized as a product of per-action probabilities times the answer likelihood conditioned on the final recall set $H_N$.

Main results

Across three QA benchmarks — SRQA (a new 5,000-sample dataset emphasizing long-range dependencies, curated from MG-ShopDial, MultiWOZ, DailyDialog, ReDial, and LoCoMo), CoQA, and SimpleQA — SRT achieves the best accuracy–latency trade-off among six baselines spanning external-memory and latent-reasoning paradigms:

Benchmark	SRT F1	SRT latency (s)	Best baseline F1	Best baseline latency (s)
SRQA	78.4	9.1	75.0 (Coconut)	10.5
CoQA	84.0	8.8	83.1 (Coconut)	10.2
SimpleQA	56.1	8.7	53.6 (RQ-RAG)	14.8

The headline claims are an average F1 gain of 3.4% and average latency reduction of 13.1% across benchmarks, with per-benchmark gains ranging from 1.1% to 4.7% in F1 and 11.1% to 14.7% in latency. Notably, SRT dominates baselines on both axes simultaneously rather than trading one for the other; the paper attributes this to eliminating external-module round trips while keeping reasoning traces short through selective citation. External-memory baselines exhibit latencies of 11.5–14.8 s, whereas SRT stays below 9.1 s throughout.

Ablations confirm that each component contributes: removing RL causes the largest degradation (recall drops from 92.5% to 85.3%, accuracy from 78.9% to 74.1%), removing CoT costs 3.4 points of accuracy, and removing SFT entirely yields the worst variant (81.0% recall, 72.2% accuracy).

Attention analysis supports the mechanism's claimed effect: recall-guided generation concentrates decoder attention mass on the cited turns, effectively pulling early-turn facts out of the diluted middle-context region where non-recall-guided models lose them.

Error analysis and transferability

Error-type distribution shifts with dialogue length: Missing Recall grows from 23.5% at 8 turns to 35.9% at 32 turns, and Wrong Recall from 15.0% to 19.2%, reflecting entity ambiguity in long logs. Notably, Failure Answer — errors made despite correct recall — remains the single largest category at every length (38.8–56.1%), indicating that recall quality is not the sole bottleneck; the model still fails to reason correctly over correctly retrieved evidence in a substantial fraction of cases.

To test generalization beyond fine-tuned models, the authors distill the mechanism into SRT-P, a prompt-level strategy applied to closed-source APIs. On a customer-service workload, SRT-P improves accuracy by 2.7% for DeepSeek-V3, 2.3% for Qwen-Max, and 1.7% for Claude 3.5 Sonnet, with gains concentrated in long-dialogue buckets ($k \geq 24$ turns). This demonstrates that explicit turn-citation prompting transfers to black-box models without any weight access.

Limitations and open questions

Several constraints qualify these results. First, the teacher annotation pipeline depends on Claude 3.7 Sonnet, so the quality ceiling of the recall supervision is bounded by that model's own multi-turn reasoning ability, and the dependency-scoring threshold (0.6) and decay constant (0.15) are empirically set rather than validated systematically. Second, evaluation is limited to a 7B backbone and short-to-medium dialogues (8–32 turns); whether the verbatim-copying behavior remains efficient or degrades when dependencies span hundreds of turns is untested. Third, the error analysis shows that failure-despite-correct-recall dominates bad cases, yet the framework offers no targeted remedy for this reasoning-stage failure mode. Finally, the cosine-similarity answer reward may admit semantically loose matches as correct, and the paper does not report human or exact-match evaluation to verify reward fidelity against hallucinated paraphrases.

Conclusion

SRT reframes dialogue memory management as an endogenous reasoning behavior: the model learns, via SFT and GRPO with verifiable rewards, to cite exactly the historical turns it needs inside its own thinking process. This yields state-of-the-art accuracy–latency trade-offs on three benchmarks, measurable attention reallocation toward cited evidence, and a pluggable prompt-level variant effective on closed-source models. The main open questions concern scaling to much longer horizons, addressing reasoning failures that persist despite correct recall, and tightening the alignment between similarity-based answer rewards and factual correctness.