---
title: 'SAMULE: Multi-Level Reflection in LLM Agents'
url: https://www.emergentmind.com/topics/samule
type: topic
---

# SAMULE: Multi-Level Reflection in LLM Agents

SAMULE (Self-Learning Agents enhanced by Multi-level Reflection) is a framework for improving the robustness and adaptivity of large language model (LLM) agents through structured, failure-centric, multi-level reflection and retrospective modeling. It is developed to address the insufficiency of reflection-based methods in complex, failure-dominated environments, where existing approaches often produce generic or non-specific feedback and fail to generalize error correction across tasks. SAMULE systematically constructs, organizes, and exploits reflections from failed trajectories at multiple abstraction levels to synthesize diagnostic feedback and enable continuous self-improvement by LLM agents [2509.20562].

## 1. Motivation and Context

SAMULE is motivated by the observation that reflection-centric LLM agent architectures such as Reflexion, Expel, and reinforcement learning approaches like Retroformer typically suffer under conditions where successful trajectories are rare or the complexity of task interactions yields noisy or shallow posthoc advice. These existing approaches are sensitive to the frequency and quality of successful episodes and often produce feedback that is uninformative or not actionable at the level of concrete failure correction. SAMULE addresses these challenges by constructing a systematic, multi-level framework for synthesizing agent feedback organized by micro-level (single trajectory), meso-level (cross-trajectory within the same task), and macro-level (across multiple tasks and error types) analyses, converting failures—not just successes—into rich learning signals [2509.20562].

## 2. Multi-Level Reflection Synthesis

The core contribution of SAMULE is the Multi-Level Reflection Synthesis pipeline, which decomposes the reflection process into three levels:

1. **Single-Trajectory Learning (Micro-Level):** For each failed agent trajectory, a comparison is made with a reference plan to extract concise, high-level corrective feedback. This explicitly diagnoses the failure with reference to the ideal plan, yielding step-specific recommendations. The reflection prompt is designed to demand diagnosis and actionable revision, beyond generic advice.

2. **Intra-Task Learning (Meso-Level):** By aggregating multiple failed trajectories for the same query, the system induces an error taxonomy and labels each trajectory action with a concrete error type from the taxonomy. The taxonomy is incrementally constructed and iteratively refined through model-guided analysis of accumulated failure cases, with each categorized action step providing structure for subsequent learning.

3. **Inter-Task Learning (Macro-Level):** Across different tasks, SAMULE clusters trajectories sharing the same error types and synthesizes transferable, cross-task reflections describing general principles or correctives for those error classes. Macro-level reflection abstractly summarizes recurring failure patterns independent of specific task content.

These three levels are then merged for each training point into a comprehensive reflection, capturing both fine-grained and abstracted error information. The synthesis process critically relies on both reference outputs (for micro-level correction) and task-derived error taxonomies (for meso- and macro-level abstraction) [2509.20562].

## 3. Retrospective Model Training and Inference

The synthesized multi-level reflections generated offline are not immediately deployable at inference due to their dependence on reference plans and multi-trajectory analysis. To overcome this, SAMULE trains a retrospective language model using supervised fine-tuning on inputs consisting of task background, queries, agent trajectories, and synthesized reflections as targets. At inference, this retrospective model receives the agent's trajectory for a new task and generates a reflection emulating the structure and diagnostic quality of the multilevel synthesis, without access to hidden references. This provides adaptive, trajectory-specific feedback in settings where multiple reference plans or full error taxonomies may be unavailable. Empirically, the use of such a retrospective model outperforms both single-trajectory/reflection heuristics and RL-based retrospectives that are highly sensitive to the quality of synthesized reflection data [2509.20562].

## 4. Foresight-Based Reflection in Interactive Agents

SAMULE extends its reflection paradigm to interactive agent settings via foresight-based reflection triggers. In this mode, after each agent action, the agent predicts the forthcoming user or environment response and compares it against the actual observed response. If the prediction deviates significantly from reality, as determined by a model-based difference detector, the agent invokes the retrospective reflection model to revise its plan. This enables real-time, proactive adaptation to surprises or unforeseen environment/user behavior, rather than waiting to posthoc analyze entire failed trajectories. Foresight-based reflection is critical for tool-using or partially observable tasks where late-stage correction may be infeasible [2509.20562].

## 5. Experimental Validation and Comparative Analysis

SAMULE is evaluated on three benchmarks:
- **TravelPlanner:** A travel-planning task with dense failure space and large solution combinatorics.
- **NATURAL PLAN:** A natural language planning suite interfacing with real-world tool APIs (e.g., Google Flights, Maps).
- **Tau-bench:** Retail and airline domains supporting both non-interactive and interactive sessions.

SAMULE is compared against ReAct, Reflexion, Expel, inter-task error reflection, and a Retroformer variant. Results across all environments demonstrate that SAMULE achieves the highest task success rates, especially on the most challenging (failure-dominated) datasets (e.g., TravelPlanner: 20.00 vs. 12.78 for Retroformer and 5.56 for Reflexion under Claude 3.5 Sonnet-v2). Ablations further reveal that micro-level reference comparison is essential, but overuse of references in meso-level synthesis can hurt diversity of diagnostic signals. Error reduction-on-retry metrics confirm that SAMULE reflections produce more actionable, failure-resolving guidance than prior approaches [2509.20562].

| Benchmark        | Best Baseline (Success %) | SAMULE (Success %) |
|------------------|--------------------------|--------------------|
| TravelPlanner    | 12.78 (Retroformer)      | 20.00              |
| NATURAL PLAN     | 53.79 (Expel)            | 60.31              |
| Tau-bench (NI)   | 82.61 (Reflexion)        | 87.83              |
| Tau-bench (I)    | 69.75 (Reflexion)        | 75.97              |

*NI: Non-interactive, I: Interactive.

## 6. Limitations and Future Directions

SAMULE as described in [2509.20562] names two main limitations. First, the error taxonomy is static and constructed offline, so the system may not adapt to novel error types encountered in previously unseen domains or changing environments; online or incremental taxonomy learning is proposed as future work. Second, the multi-level reflection synthesis is computationally demanding, requiring extensive trajectory analysis and cross-task clustering; while the resulting retrospective model is lightweight at inference, initial synthesis remains costly. This suggests that scalable deployment may require additional data pipeline engineering or incremental update strategies [2509.20562].

## 7. Significance and Implications

SAMULE establishes a principled separation between reflection data quality and the sophistication of the retrospective learning machinery, empirically demonstrating that high-quality, structured multi-level reflections derived from diverse failure modes drive greater agent improvement than advanced RL fine-tuning alone. This framework generalizes the reflection paradigm, turning agent failure into a systematic resource for error taxonomy induction, cross-task abstraction, and continual self-improvement. Its conceptual innovation is the architectural formalization of micro/meso/macro-level reflection and its operationalization in scalable agent training pipelines, yielding more adaptive and robust LLM-based agents than prior reflection-based or success-centric approaches in complex task environments [2509.20562].

Source: https://www.emergentmind.com/topics/samule