MultiSessionCollab: Persistent Collaboration

• MultiSessionCollab is a collaboration framework that learns and updates persistent user preferences from conversational signals over consecutive sessions.
• It employs a lightweight memory mechanism to merge session reflections, improving task success by reducing user enforcement and streamlining interactions.
• Reinforcement learning and retrieval techniques are used to adapt dialogue styles, ensuring efficient, long-horizon collaboration across diverse domains.

Searching arXiv for the cited works on MultiSessionCollab and related multi-session collaboration systems. MultiSessionCollab is a multi-session collaboration setting in which an agent interacts with the same user over consecutive sessions, infers hidden interaction preferences from conversational enforcement behavior, stores those preferences in persistent memory, and conditions later responses on that memory to improve collaboration quality over time (Mehri et al., 6 Jan 2026). The label also appears in adjacent work on online preference modeling, mixed-session dialogue, and collaborative systems infrastructure, which suggests a broader research motif centered on persistent cross-session state, memory refinement, and coordination under repeated interaction rather than a single narrow task definition (Hao et al., 21 Mar 2026).

1. Benchmark definition and formal setting

In its canonical benchmark formulation, MultiSessionCollab considers a set of users $U$. For each user $u \in U$, the system runs $S$ consecutive sessions indexed by $s=1,\dots,S$. In session $s$, the user and agent collaborate on a new problem $T_{u,s}$, and only the user sees the ground-truth statement of that problem. The interaction is a turn-based conversation

$C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$

where $u_i$ is the user utterance, $a_i$ is the agent utterance, and $N \le 10$. Each user has an unknown preference set

$u \in U$0

drawn from a fixed taxonomy such as “provide key takeaways,” “step-by-step reasoning,” or “concise style” (Mehri et al., 6 Jan 2026).

The primary objective is not only to solve the task in each session but also to improve collaboration quality as sessions accumulate. The benchmark therefore measures task success $u \in U$1, user effort $u \in U$2, and conversation length $u \in U$3. User effort is defined as the number of times the user must explicitly enforce a violated preference during the conversation, and the central challenge is that preferences are never directly revealed to the agent: they must be inferred from enforcement signals

$u \in U$4

This setting turns long-term collaboration into a preference-inference problem under partial observability (Mehri et al., 6 Jan 2026).

A later online instantiation sharpens the same idea into a larger-horizon simulator benchmark. In that version, an LLM-based assistant is paired with an LLM-based user simulator for $u \in U$5 sessions, with up to $u \in U$6 turns per session, over the domains math-hard, math-500, and bigcodebench. Each simulated user is specified by a persistent “style” persona encoded as structured condition-action rules, with 60 user profiles per domain and an average of 43 preferences and 15 conflict groups per profile. This formulation adds timeout rate and interaction efficiency,

$u \in U$7

to emphasize not just eventual correctness but also the cost of personalization (Hao et al., 21 Mar 2026).

2. Persistent memory and preference representation

The defining architectural mechanism in MultiSessionCollab is a persistent per-user memory $u \in U$8. In the long-term collaborative agents introduced with the benchmark, $u \in U$9 is a free-form text “notes” document, up to a few thousand tokens, describing which preference types apply, the contexts in which they arose, and concrete strategies such as “when explaining code use bullet points.” Initialization is deliberately empty: $S$0 This representation is intentionally lightweight: it does not require a rigid schema, but it is explicit enough to be prepended to later prompts (Mehri et al., 6 Jan 2026).

Memory is updated after each session through a reflection object

$S$1

structured as

$S$2

The new memory is then computed as

$S$3

where, in practice, “Merge” is string concatenation plus light post-processing to avoid duplication. At the beginning of session $S$4, $S$5 is prepended to the system prompt, and a lightweight retriever selects the most relevant memory sentences conditioned on the conversation history so far (Mehri et al., 6 Jan 2026).

The same paper also introduces an abstract view in which memory is treated as a vector $S$6, with session-level updates written as

$S$7

Although the implementation is textual rather than vectorial, this abstraction makes clear that MultiSessionCollab is fundamentally about state estimation and state update across sessions, not merely prompt accumulation (Mehri et al., 6 Jan 2026).

A separate preference-modeling line extends this memory idea into dual user representations. VARS maintains long-term and short-term user vectors in a shared preference space and uses them to bias retrieval scoring over structured preference memory. The reported interpretability results indicate that the long-term vector aligns with stable cross-user preference overlap, whereas the short-term vector captures session-specific adaptation. This suggests a separation between durable user style and transient conversational state within the broader MultiSessionCollab paradigm (Hao et al., 21 Mar 2026).

3. Learning signals, reflection training, and online adaptation

One training strategy for MultiSessionCollab treats the reflection generator $S$8 as a reinforcement-learning policy. For each sampled conversation $S$9, the system knows the enforcement set $s=1,\dots,S$0, and the reflection receives reward

$s=1,\dots,S$1

Here $s=1,\dots,S$2 is scored by an LLM judge according to whether the reflection mentions every preference revealed in the enforcement utterances without hallucinations, and $s=1,\dots,S$3 encourages a clean JSON structure and reasoning trace. The objective is

$s=1,\dots,S$4

optimized with Group Relative Policy Optimization, with an added KL constraint to the pre-fine-tuned LLM. Before RL, the model is warm-started with supervised fine-tuning on a small set of gold reflections (Mehri et al., 6 Jan 2026).

The online preference-modeling variant uses a weaker supervision channel. Rather than consuming the full structured simulator feedback as a dense label, it derives a lightweight scalar reward $s=1,\dots,S$5 from the follow-up user turn $s=1,\dots,S$6. The reward is constructed from sentiment keywords, topic coherence measured by cosine similarity between turn embeddings, and clipping to $s=1,\dots,S$7. It further introduces a retrieval-attribution gate $s=1,\dots,S$8 based on the maximum embedding similarity between the current user turn and retrieved memory cards. The retrieval-specific advantage is then

$s=1,\dots,S$9

where $s$0 is a per-user exponential moving average baseline updated as

$s$1

This makes memory refinement possible without per-user fine-tuning and under frozen backbones (Hao et al., 21 Mar 2026).

A common misconception is that the value of multi-session memory should be read solely from raw task accuracy. Under frozen backbones, the reported main benefit of user-aware retrieval is improved interaction efficiency rather than large gains in raw task accuracy. In that sense, MultiSessionCollab evaluates personalization not only as correctness but also as reduced repetition, fewer corrective turns, and lower user-side overhead (Hao et al., 21 Mar 2026).

4. Evaluation protocols and empirical results

The original benchmark evaluates on five reasoning domains: MATH-500, MATH-Hard, LogiQA, MMLU, and MedQA. For each domain there are 100 simulated users, each with $s$2 random preferences, and 20 problems per user, yielding 10,000 sessions per agent. The ablation settings distinguish direct solve without chat, multi-turn chat without preferences, multi-turn chat with preferences and no oracle information, an oracle agent given preferences at session start, and a memory agent that learns preferences through interaction (Mehri et al., 6 Jan 2026).

On the overall average with Llama-3.3-70B-Instruct, the baseline without memory records $s$3, $s$4, and $s$5, whereas the memory-equipped agent records $s$6, $s$7, and $s$8. The reported interpretation is that memory reduces user enforcement by $s$9 and boosts task success by $T_{u,s}$0. In the same ablation suite, moving from the non-oracle preference setting to the oracle setting regains $T_{u,s}$1–$T_{u,s}$2 task-success points with $T_{u,s}$3, while the learned-memory agent is competitive with oracle, with task success within 1–2% of the oracle agent and $T_{u,s}$4 (Mehri et al., 6 Jan 2026).

The learning dynamics are also explicitly session-dependent. If

$T_{u,s}$5

and similarly for $T_{u,s}$6 and $T_{u,s}$7, then $T_{u,s}$8 grows across sessions $T_{u,s}$9, with large gains early and then a plateau. Both $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$0 and $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$1 decrease over sessions, with the steepest drop in the first five sessions. This indicates that memory is most valuable early, when preference uncertainty is still being resolved (Mehri et al., 6 Jan 2026).

The human user study introduces 19 real participants, including software engineers and students, under single-domain and mixed-domain conditions. Each participant runs four three-session studies: coding without memory, coding with memory, mixed-domain without memory, and mixed-domain with memory. By Session 3, coding with memory has median conversation length 6 versus 10 without memory, and mixed-domain with memory has median length 4 versus 8 without memory. Preference adherence rises from approximately 3 to 4 with memory and stays flat without memory; preference retention rises from approximately 1 to 4 with memory and stays approximately 1 without memory; confidence in future collaboration and overall satisfaction both increase by approximately 1.5 points over sessions with memory and are dropped or flat without memory (Mehri et al., 6 Jan 2026).

In the online benchmark built for VARS, each domain contains $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$2 sessions, with at most 36,000 turns per domain. The full VARS agent is reported to achieve the strongest overall performance, match a strong Reflection baseline in task success, and reduce timeout rate and user effort. This sharpens the empirical picture: effective MultiSessionCollab systems need not dominate on every single metric simultaneously, but they are expected to improve long-horizon efficiency and reduce repeated preference enforcement (Hao et al., 21 Mar 2026).

5. Failure modes, bottlenecks, and common misconceptions

The most direct neighboring benchmark for diagnosing MultiSessionCollab failure modes is MemoryCode, a synthetic multi-session coding benchmark that asks whether LLMs can keep track of simple, evolving coding conventions amid irrelevant chatter and then apply them correctly when generating code. Its dialogue histories span $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$3 sessions, with 54% “short” examples and 46% “long” examples, and instructions can be updated up to eight times per coding instruction. The central result is that models that handle isolated instructions well deteriorate sharply when instructions are distributed across sessions (Rakotonirina et al., 19 Feb 2025).

The degradation is substantial. For GPT-4o, the reported mean accuracies are 0.94 in the Instruction setting, 0.93 in the Session setting, 0.79 on short histories, and 0.30 on long histories; all evaluated models plateau to approximately 0.1–0.2 accuracy as histories approach 100 sessions. The authors’ analysis attributes this not simply to context overload but to a failure to retrieve and integrate information over long instruction chains. In the instruction-chain setup, which removes filler text and retains only the chronological list of instructions and updates, Command R+ shows a near-identical decline to the full-history case, and retrieval-augmented generation produces only marginal improvements for short histories and no benefit on long histories (Rakotonirina et al., 19 Feb 2025).

This point is important for interpreting MultiSessionCollab. A frequent misconception is that persistent collaboration becomes tractable once the model has a large enough context window or a simple retrieval layer. The MemoryCode evidence suggests otherwise: retrieval is not the sole bottleneck, and long-horizon collaboration also requires composition over updates, suppression of deprecated rules, and consistent joint application of multiple constraints. The update-rank analysis reinforces this diagnosis: rank-0 instructions average approximately 0.7 accuracy, whereas rank-5-or-higher instructions fall below 0.3 (Rakotonirina et al., 19 Feb 2025).

A plausible implication is that MultiSessionCollab systems need mechanisms that do more than store user preferences. They must also support revision, overwrite semantics, and selective recall under multi-session drift. This is consistent with the benchmark authors’ proposed directions, including explicit long-term memory modules, augmented reasoning primitives, prospective-memory frameworks, and specialized prompting or fine-tuning for update chains (Mehri et al., 6 Jan 2026).

6. Related architectures and broader uses of the term

The broader literature uses closely related multi-session designs in ways that illuminate the scope of MultiSessionCollab beyond user-preference benchmarks. In mixed-session dialogue, MiSC and the Egocentric Memory Enhanced Mixed-Session Conversation Agent (EMMA) define episodes of six consecutive sessions with four speakers, where one main speaker persists across sessions and three partners rotate. EMMA writes partner-specific memories, links related memories across sessions, retrieves the top-1 memory plus linked neighbors, and prepends them to generation context. Human evaluations report consistency and coherence scores near 4.8–4.9, and EMMA achieves memorability of approximately 4.60/5 while avoiding the partner-confusion failure of single-partner baselines such as MSC 2.7B and ReBot (Jang et al., 2024).

In collaborative editing infrastructure, the term is also used for multi-session routing rather than preference learning. A rich-text editor extension introduces a single-WebSocket architecture in which every message carries a $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$4 header, while both client and server maintain a $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$5 and awareness maps for cursors and presence. The design relies on Yjs, an operation-based CRDT, and reports a $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$6 reduction in WebSocket traffic for both idle and active editing across 1–4 doclets. Here the multi-session problem is demultiplexing, consistency, and per-session awareness rather than longitudinal preference inference (Jatana et al., 2024).

A browser-based collaborative music system offers another system-level analogue. The Multi Web Audio Sequencer maintains room state in a single Node.js process, synchronizes state via WebSocket broadcast, preserves critical ordering through a server-echo protocol, and emphasizes live editing, chat, and presence awareness for remote co-creation. Because all state is held in memory, the authors note that persistence and horizontal scaling would require a backing store or distributed state manager. Their qualitative findings also show that awareness features are essential: without chat and presence, users had no way to coordinate who would edit which track (Favory et al., 2019).

A different line studies collaborative editing sessions as analytically segmented activity bursts. There, a session is a maximal contiguous subsequence of edits under a chosen maximum time gap $C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$7, and sessions are classified as single-authored or co-authored. The conflict analysis shows that potential conflicting cases are few, but they are more likely to become real conflicts. This highlights a distinct but related concern in multi-session collaboration research: not all cross-session structure is about memory; some of it concerns time-position conflict formation, awareness, and intervention design (Nguyen et al., 2020).

The same long-horizon theme also appears in multi-agent work. CoWork-X formalizes peer collaboration across episodes as an Execute$C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$8Diagnose$C_{u,s} = [(u_{1},a_{1}),\dots,(u_{N},a_{N})],$9Update$u_i$0Execute loop, with HTN-executable skill memory for low-latency online behavior and post-episode patch-style consolidation under budget and drift constraints; over 30 episodes, it reports cumulative performance gains together with sub-second coordination and zero in-episode tokens (Lin et al., 4 Feb 2026). In multi-party dialogue, the Interruptible Collaborative Roleplayer framework models collaborator–intervention dynamics as a two-player Modified-Action MDP and uses counterfactual KL regularization to improve common-ground convergence and task utility over multiple turns (Nath et al., 26 Oct 2025).

Taken together, these systems indicate that MultiSessionCollab has become a useful organizing concept for research on persistent collaboration under repeated interaction. Across benchmarks, dialogue agents, editors, music systems, and multi-agent planners, the recurrent technical issues are memory persistence, update handling, retrieval or routing, awareness, and the control of cross-session interference. The benchmark introduced for learning user preferences gives this research direction a particularly explicit evaluation target: agents that improve collaboration quality over time by learning what matters to a specific partner and carrying that knowledge forward coherently (Mehri et al., 6 Jan 2026).