Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents
This lightning talk explores a breakthrough in AI agent adaptation: how frozen-weights language models can learn continuously from deployment feedback without costly retraining. The presentation examines the Spark memory system, which enables agents to distill operational episodes into natural language rules, achieving performance gains of up to 2.6x over static baselines and demonstrating robust cross-model knowledge transfer.Script
Most deployed AI agents are frozen at launch and never adapt, even though every interaction generates valuable feedback. This paper shows how agents can learn continuously from their own operational experience without updating a single model weight.
The researchers tested this on tau-Banking, a brutal real-world simulation with 97 procedural banking tasks where crucial policy knowledge is hidden across 700 documents. Agents had to solve tasks, receive feedback, and bank what they learned for future attempts.
With only binary success or failure signals, agents improved their single-trial success rate 1.6 times over the baseline. When given explicit corrections on failures, performance jumped 2.6 times higher, converting 22 previously unsolvable tasks into reliable wins.
The memory stores transfer across models with startling effectiveness. A weaker model reading lessons written by a stronger model improved its performance by 0.224 points, while the stronger model still gained 0.066 points from the weaker model's hard-won experience.
The Spark system works through a simple loop: agents attempt tasks, distill episodes into when-then rules after completion, and retrieve relevant lessons contextually. All learning happens post-episode with context-limited knowledge, preventing feedback contamination while enabling genuine organizational memory.
This work charts a practical path forward for enterprise AI: continual adaptation without the operational cost and safety risks of constant retraining. Your deployed agents are already generating the feedback they need to improve. Explore how memory-augmented learning can unlock that potential at EmergentMind.com.
