rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation

Abstract: LLMs can improve recommendation quality by reasoning explicitly over user history and candidate items - for example, extracting a user's preferences or explaining why one item fits better than another - rather than mapping history directly to a ranked list. This reasoning, however, is expensive to repeat on every ranking request and, once produced, is typically consumed once and discarded, leaving it neither reusable across future requests nor easy to inspect or correct as user tastes drift. Our insight is that reasoning does not need to be regenerated at every call if it can instead be compressed once into a compact, structured memory that a lightweight model retrieves from. We propose rEDMRec, which distills a teacher LLM's reasoning into four typed, editable experience channels - long-term preference, short-term context, item-perception, and counterfactual hard-negative comparisons - maintained by an LLM memory controller that performs Add/Delete/Modify/Keep operations and refines entries via K-agent debate. A lightweight student LLM then ranks candidates purely by retrieving from this memory, without invoking the teacher again, decoupling online inference cost from reasoning depth. Across ML-1M, Amazon Beauty, and Steam and ten student backbones, rEDMRec improves HR@1 over zero-shot, few-shot, and RAG on every backbone, and over GraphRAG on most backbones, with Impv up to 13.3% vs. the second-best baseline on ML-1M. Channel ablations show that short-term context is the only channel that helps consistently across capacity tiers, whereas long-term, item-perception, and counterfactual contributions are capacity-dependent (and can reverse on the strongest students); debate-based memory optimization lowers bank duplication by 7.4 percentage points while raising downstream HR@1 by up to +0.029 over six optimization epochs.

• rEDMRec distills teacher LLM's reasoning into an editable structured memory, boosting recommendation accuracy and efficiency across three datasets
• That memory operates with four channels:long term preference,short term context, item perception and counterfactual comparisons
• A debate-based optimization process refines the memory content based on reward criteria enhancing performance

rEDMRec addresses a concrete inefficiency in LLM-based recommendation: teacher-level reasoning over user history and candidates is regenerated on every ranking request and then discarded, so its cost is paid repeatedly while the resulting knowledge is neither reusable nor inspectable. The paper proposes an architecture that pays this cost once: a teacher LLM's reasoning is compressed into a typed, editable experience memory, and a frozen student LLM ranks candidates purely by retrieving from that memory (2608.18952). The design is evaluated across three datasets and ten student backbones (3B–20B parameters), with ablations that isolate which memory channels matter, how teacher quality propagates through bank quality to ranking accuracy, and whether debate-based memory maintenance does measurable work.

Architecture: four-channel experience memory

The core object is a persistent non-parametric bank $E = \{E_k\}$ with four independently indexed, retrieved, and edited channels: long-term preference ($\mathrm{lt}$), short-term context ($\mathrm{st}$), item perception ($\mathrm{ip}$), and counterfactual hard-negative comparison ($\mathrm{cf}$). A teacher $\mathrm{LLM}_T$ runs four extraction passes — preference maintenance over history batches, item-perception context, five-step chain-of-thought reasoning, and counterfactual anchor/contrast analysis — whose output fields are routed by a distillation adapter into channel-indexed entries $e=(\tau,\mathbf{v}_e,\mu)$, embedded with a shared sentence-transformer. Three channels live in a vector store; counterfactuals are stored as graph edges (anchor/contrast pairs with rationale embeddings) in a hybrid vector–graph store.

Two design choices deserve emphasis. First, the student is held frozen throughout: it retrieves top-$m$ entries per channel, concatenates them into a ranking prompt, and scores candidates without any weight update or fresh teacher call. This makes accuracy gains attributable to memory content rather than parameter adaptation, and lets the same protocol transfer across teacher–student pairs. Second, entries are not appended blindly: an LLM memory controller inspects a snapshot of the bank against new insights and commits Add/Delete/Modify/Keep operations, adapting the Training-Free GRPO principle of editing an external experience library instead of weights to recommendation-specific signal types. Retrieval is deterministic dense top-$m$ lookup, contrasting with RAG-style re-retrieval of raw records per request and GraphRAG's static, untyped community summaries.

Online optimization: reward-conditioned debate

After each student prediction, a reward vector packs HR@1, HR@10, reciprocal rank, and DCG-style position discount into the critique prompt. $K$ fixed-persona debating agents append critiques to a shared transcript; an arbiter synthesizes revised experience entries, which are committed through the same adapter/controller path used for initial distillation. Notably, the reward is never used as a training loss or to select among agent proposals — it only conditions natural-language critique toward the observed failure mode (missed top-rank, near-miss, deep placement).

Main results

Across ML-1M (49,893 test samples), Amazon Beauty, and Steam, rEDMRec improves HR@1 over zero-shot, few-shot, and RAG for every one of ten students, and over GraphRAG on most backbones — with explicit exceptions. The largest relative gain lands on the smallest student: Qwen2.5 3B achieves Impv $\mathrm{lt}$0 over GraphRAG on ML-1M ($\mathrm{lt}$1), $\mathrm{lt}$2 on Beauty, and $\mathrm{lt}$3 on Steam, consistent with structured memory compensating for limited parametric capacity. On Llama 3.1 8B, however, Impv is negative ($\mathrm{lt}$4 on ML-1M), attributed to weak instruction-following on the concatenated memory prompt rather than to an unhelpful bank; GPT OSS 20B also trails GraphRAG slightly ($\mathrm{lt}$5). Mid-capacity students gain $\mathrm{lt}$6 to $\mathrm{lt}$7, all significant at $\mathrm{lt}$8 but mostly not at the smaller Beauty/Steam splits.

The channel ablation is the most contrarian result in the paper: short-term context is the only channel that helps consistently across capacity tiers, while long-term preference, item-perception, and counterfactual channels show reversed ablation on the strongest student (gpt-5-mini) — removing them improves HR@1 by $\mathrm{lt}$9 to $\mathrm{st}$0. The authors offer three explanations in decreasing order of evidential support: redundancy with raw history already in the prompt, generic low-specificity early entries, and counterfactual edges that occasionally promote plausible-but-wrong candidates. A saturated 120B student appears to ignore the bank altogether. This directly contradicts any "retrieve all four channels always" deployment policy on strong backbones.

Teacher quality and capacity ceiling

Fixing the student and sweeping seven teachers shows that lower bank duplicate rate predicts downstream gain, but only up to a student-capacity ceiling. gpt-5.4-mini has the lowest duplicate rate among strong teachers (12.4%) and the largest gain (+0.060 HR@1 for the strong student); Llama 3.1 8B Instant, the weakest teacher, produces the most duplicated bank (22.8%) and the smallest gain (+0.015). Yet GPT OSS 120B produces the least duplicated bank overall (9.8%) with only middling gains, and drops below the more concise Qwen3 32B bank once the student shrinks to 3B parameters — verbosity beyond what the student can exploit yields no benefit. The causal chain runs teacher quality → bank quality → downstream gain, gated by student capacity.

Debate optimization is functional, not cosmetic

Over six optimization epochs, duplicate rate falls from 18.0% to 10.6% (−7.4 points), mean experience reward rises from 0.52 to 0.78, and Mixtral 8x7B HR@1 rises +0.029 while a no-debate paraphrase control stays flat (+0.007), isolating the critique-and-revise loop as the active mechanism. Gains saturate within two to three epochs. An agent-count sweep places the quality-per-cost knee at $\mathrm{st}$1: going from 1 to 4 agents lifts HR@1 by +0.022, while going from 4 to 10 adds only +0.006 for six extra LLM calls per case. A qualitative study using a deterministic specificity score shows item-perception entries improving most (0.405→0.488), with the short-term channel's slight decrease reflecting compression of verbose narratives into short conditional "session rules" rather than regression. Failure cases include a taste-flip risk where debate overwrites a valid sci-fi profile with noir/crime preferences — evidence that aggressive Modify operations can discard correct prior signal.

Limitations and open questions

The paper is candid about several gaps. Teacher coverage is incomplete: main results fix one teacher, and the teacher-distillation study varies teachers against only two students, leaving the full teacher×student cross-product unmeasured. Explanation faithfulness is untested — the student can emit explanations grounded in retrieved memory, but no human evaluation verifies they are faithful to what they cite. The McNemar tests reconstruct contingency tables approximately from reported rates, a minor statistical caveat. Domain scope is restricted to movies, beauty products, and games with English metadata and stable comparable attributes; the counterfactual schema in particular assumes such structure, and short-video or news domains remain untested. Finally, the operating-point caveat is practical: trends measured across wide ranges of bank size, epochs, and agent counts do not guarantee behavior at any specific intermediate setting.

Conclusion

rEdmRec demonstrates that LLM recommendation reasoning can be amortized: compressed once into a typed, editable memory and served cheaply thereafter by a frozen student. The empirical picture is nuanced rather than uniformly favorable — benefits concentrate on small-to-mid capacity students, GraphRAG remains competitive on some backbones, and three of four memory channels can actively hurt strong models. The open problems the paper leaves are specific: completing the teacher×student coverage matrix, validating explanation faithfulness, and characterizing when debate-driven edits overwrite valid taste signals.