Ability of conversational agents to identify and leverage valuable information from past interactions

Determine whether Large Language Model-based conversational agents engaged in multi-session collaboration can recognize which information from past user–agent interactions will be valuable for future interactions and leverage that information effectively to improve collaboration outcomes.

Background

The paper observes that most prior memory systems and evaluations emphasize recall of past interaction details, which, while necessary, may not capture the full capability needed to improve long-term collaboration. Specifically, beyond remembering, agents must discern which pieces of information will be helpful later and use them to guide future behavior.

This uncertainty motivates the introduction of MultiSessionCollab, a benchmark that assesses whether agents can learn and apply user interaction preferences across sessions, thereby testing the broader question of recognizing and leveraging valuable information from past interactions.

References

In particular, it remains unclear whether agents can recognize what information is valuable for future interactions and leverage it effectively.

— Learning User Preferences Through Interaction for Long-Term Collaboration  (2601.02702 - Mehri et al., 6 Jan 2026) in Section 1: Introduction

Natural Integration is also a binary judgment; the Reference score and the failure-mode taxonomy (\S\ref{sec:gap_available}) capture partial integration but were not designed or validated as ordinal measures, so a human-validated graded metric remains future work.

— MemUse: Moving Memory Evaluation from Direct QA to Natural Integration in Long-Term Human-AI Conversation  (2608.24189 - Sumida et al., 25 Aug 2026) in Limitations, paragraph “Automated measurement”

A key open challenge remains enabling agents to infer and apply task-relevant implicit preferences from historical trajectories at test time.

— AdaptArena: Evaluating Test-Time Personalization of Web Agents  (2609.36488 - Shin et al., 29 Sep 2026) in Section Related Works, paragraph “Agent Personalization”

Letting Wr grow across sessions brings no meaningful benefit, as artifacts from the current session already capture most of the useful workspace reuse. This may change when many sessions run on the same repository, which we leave for future work.

— AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines  (2610.01108 - Lee et al., 1 Oct 2026) in Appendix B, “Additional Corpus Ablations”

We therefore make no claim that users prefer Reconsider. The results provide preliminary evidence about LLM judgments of generated responses and identify concrete questions for an independent human evaluation.

— Before Bringing It Up: When and How AI Companions Should Use Memor  (2610.09470 - Guo et al., 7 Oct 2026) in Section 2, Introduction; Section 3.4, Evaluator Validity Analysis; Section 5, Limitations and Future Work; Conclusion

Under less specified requests, an agent therefore makes partial progress while failing to recover the user state and procedural knowledge that jointly determine the correct target, action, or execution path. Autonomous performance also depends on whether the task agent invokes memory. We define the retrieval-call rate as the fraction of sessions in which the agent makes at least one memory_search call. The advanced memory systems have an average retrieval-call rate of 57.3\%. Scheduled evaluation supplies retrieval at predefined turns, whereas Autonomous agents leave many sessions without any memory call. They must infer that an underspecified request depends on prior interaction, formulate an effective query, and then connect the retrieved personal context to the applicable procedure. Under the same setting, PraxisMem attains the strongest non-oracle Session SR for Tool, Memory QA, and Artifact tasks at 23.00\%, 22.22\%, and 31.64\%, compared with the next-best 16.67\%, 14.29\%, and 20.51\%. Its advantage is consistent with maintaining both memory content and the organization needed to apply procedures under the current user state, while the remaining gap from Scheduled performance shows that reliable autonomous memory access remains an open challenge.

— Memorizing Who, Knowing How: Benchmarking Personal and Procedural Memory for Persistent Assistants  (2610.20022 - Qin et al., 14 Oct 2026) in Appendix, Section 'Autonomous-Setting Results' (labelled app:autonomous-results)