---
title: Reflexion Framework in AI & Decision Models
url: https://www.emergentmind.com/topics/reflexion-framework
type: topic
---

# Reflexion Framework in AI & Decision Models

The Reflexion Framework encompasses a diverse set of models and methodologies across computer science, artificial intelligence, and mathematical theory, unified by the principle that agents, programs, or systems improve their performance or adaptability by explicit self-reference, structured reflection, or recursive representations of their own state, reasoning, or design. In computational contexts, Reflexion frameworks range from reinforcement learning agents with self-verbalized feedback, to software transformation systems enabling runtime distribution adaptation, to formal models in game theory that encode hierarchical agent beliefs. This article provides a comprehensive technical survey of Reflexion frameworks as they appear in contemporary research, focusing on language-agent architectures, predicate-based rule memory for LLM agents, multi-agent reflection mechanisms, self-reflective reinforcement in code generation, and foundational decision-theoretic models.

## 1. Historical Roots and Core Definitions

The term “reflexion” (as distinct from “reflection” in programming) first appears in game-theoretic contexts to denote recursive models of agent beliefs and strategies. In the Reflexion Framework of decision-making, each agent maintains (potentially infinite) hierarchies of beliefs about both the environment (nature) and the beliefs/strategies of other agents. A reflexive model, in this sense, explicitly encodes not just first-order (i.e., about the world) but higher-order beliefs (what I think you think I think…) and—critically—may stratify agents by “reflexion rank,” capturing bounded recursion of reasoning layers [1801.07121].

In the context of software and AI, reflexion has evolved to denote not only explicit modeling of self but also mechanisms whereby an agent, code generator, or distributed system inspects, critiques, or restructures its own outputs, behaviors, or architectures. This recursive orientation has become central in language model (LLM) agent research, secure code generation, distributed systems flexibility, and affective computing.

## 2. Reflexion in Large Language Model Agents

### 2.1 Reflexion-Verbal Reinforcement Loop

The Reflexion framework for LLM agents, prominently introduced in "Reflexion: Language Agents with Verbal Reinforcement Learning" [2303.11366], eliminates gradient-based parameter updates in favor of on-the-fly, in-context self-reflection. The core components are:

- **Actor Model ($M_a$):** Generates actions or reasoning steps, conditioned on current state and a buffer of textual self-reflections (episodic memory).
- **Evaluator ($M_e$):** Offers scalar or binary success/failure feedback after each trial.
- **Self-Reflection Model ($M_{sr}$):** Constructs natural language diagnoses and recommendations, based on the current trial’s trajectory and cumulative memory.
- **Reflection Buffer (mem):** Short-term episodic storage of prior reflection outputs, which conditions future Actor prompts.

The agent rollouts are thus:
1. Act in environment, complete a trajectory.
2. Receive success/failure.
3. Generate a self-reflection summarizing failure modes.
4. Add reflection to memory buffer, condition replay of environment on this extended prompt.

This approach causes rapid behavioral adaptation across trials. For instance, in programming, empirical results show that Reflexion-augmented GPT-4 achieves 91% pass@1 accuracy on HumanEval, compared to 80% for unaugmented GPT-4. Similar gains are observed in text-game and multi-hop QA benchmarks [2303.11366].

### 2.2 Multi-Agent Reflexion and Mitigating Degeneration

Single-agent Reflexion can suffer from "degeneration of thought"—the agent replays the same error-laden chains, even when memory signals the mistake. "Multi-Agent Reflexion" (MAR) [2512.20845] addresses this by instantiating multiple LLM-based personas, each bringing different critical perspectives. Each persona independently critiques a failed rollout, engages in one or more rounds of debate, and a judge (also an LLM) synthesizes the critiques into a consensus reflection. This process reliably improves both diversity of corrections and accuracy on Hard QA (HotPotQA) and coding (HumanEval) benchmarks.

## 3. Reflexion as External Rule Memory: Meta-Policy Reflexion

The Meta-Policy Reflexion (MPR) framework [2509.03990] advances the Reflexion paradigm from unstructured textual reflection to structured, externally stored memory in the form of predicate-based rules. After each episode, an LLM parses failure trajectories into first-order logic predicates over state-action pairs (meta-policy memory $\mathcal{M}$), each paired with a confidence score. At inference, rules in $\mathcal{M}$ that match the current state bias or constrain action decoding:

- **Soft Decoding:** Matched predicates are prepended to the LLM prompt, influencing sampling.
- **Hard Admissibility Checks:** Candidate actions violating "forbid" rules are vetoed.

This approach guarantees persistent cross-task memory, i.e., once a common error is detected and codified as a rule, it will be suppressed in future, even on novel tasks. Empirically, MPR converges to 100% task success in AlfWorld within three rounds of self-reflective rule accumulation, substantially outperforming plain Reflexion or ReAct policies [2509.03990].

## 4. Intrinsic Self-Reflexion for Code Generation and Security

### 4.1 ReflexiCoder: RL-Internalized Reflection

ReflexiCoder [2603.05863] brings reflection inside the model weights: code generation is cast as an end-to-end trajectory encompassing initial code, reflection ("bug detected" or "optimization only"), and corrective code, all produced in a single forward pass. A bespoke RL-zero objective shapes the policy to emit well-formed "reflection" tokens, produce validated fixes, and avoid unnecessary cycles. When trained, token efficiency improves by ~40%, and correctness surpasses open and proprietary LLMs of comparable size. Notably, no external critic or test harness is needed post-training: the model has internalized its own self-debugging loop.

### 4.2 Reflexion for Secure Code Generation

Reflexion can be instrumental in code security tasks, as demonstrated in "Secure Code Generation at Scale with Reflexion" [2511.03898]. Here, Reflexion operates as a three-round feedback loop:
- Each attempt, the code is scanned by an Insecure Code Detector (ICD), returning identified CWE IDs.
- The next prompt includes the code, ICD results, and a revised instruction to patch the vulnerabilities.
- Repair, regression, and net gain metrics (see original for formal LaTeX definitions) trace progress across rounds.

Experiments reveal most security improvement is achieved in the first Reflexion iteration (~+6 pp), with diminishing returns thereafter. Reflexion increases secure generation across all tested LLMs; Python sees the highest secure rates, C and C# the lowest.

## 5. Mathematical Reflexion: Decision Theory, Games, and Learning

Within mathematical modeling, as in [1801.07121], the Reflexion Framework formalizes agents that reason about not just the state of nature but also about each other's beliefs and reasoning procedures to arbitrary depth.

- **Awareness structure:** For each agent $i$, this is the infinite hierarchy $\{\theta_{ij_1...j_l}\}$ denoting $i$'s beliefs about nature, about others' beliefs, and so forth.
- **Strategic reflexion:** Agents may have beliefs about others' strategic sophistication—e.g., level-$k$ reasoning (level-0: default, level-1: best respond to level-0, etc.).
- **Reflexive equilibrium:** A solution profile where each agent's strategy is optimal given their entire reflexive belief hierarchy.

Dynamic (learning) versions integrate such models with best-response or reinforcement dynamics stratified by reflexion rank. This captures real observed behaviors in strategic learning, teaching, and multi-agent adaptation in distributed control or economic settings.

## 6. Control and Analysis of Reflexion in LLMs

ReflCtrl [2512.13979] investigates self-reflection as a latent behavior in chain-of-thought LLMs. By segmenting LLM outputs into "inference" and "reflection" steps and extracting reflection directions in the model's latent space, ReflCtrl enables fine-grained control:

- **Stepwise steering:** Adjusts the frequency of reflection by moving activations along the reflection direction; can suppress or enhance self-reflection as desired.
- **Empirical findings:** Suppressing reflection can yield up to 33.6% token savings with negligible accuracy loss on reasoning and coding tasks (GSM8k, MMLU, MATH-500).
- **Interpretability:** The activation in reflection subspace correlates with model uncertainty, suggesting LLMs’ decision to reflect is functionally tied to internal measures of uncertainty [2512.13979].

## 7. Reflexion in Other Computational Paradigms

### 7.1 Reflective Software Distribution: RAFDA

The RAFDA framework [1006.5643] applies reflexion principles to software engineering by automatically transforming ordinary Java programs into distribution-agnostic systems. RAFDA operates by:

- Bytecode transformation to extract interfaces, relocate constructor logic, and replace direct field/method access with interface/ factory calls.
- Generating both local and remote (proxy) implementations for each class. Runtime policy manager selects local/proxy instantiations based on network/ resource metrics.
- Decoupling application logic from distribution, enabling dynamic migration of objects across JVM boundaries via flexible policy.

While no performance or scalability results are given, the resulting system allows runtime alteration of distribution architecture without code or recompilation, a direct outcome of reflective transformation [1006.5643].

### 7.2 Affective Computing: Layered Self-Reflection

Beyond agentic or structural contexts, Reflexion is also instantiated in affective AI systems [2504.20342]. Here, Reflexion consists of staged guidance to users through explicit emotional awareness, reframing, values alignment, and action planning, with LLMs generating metaphoric narratives for psychological distancing and growth.

## References

- Reflexion: Language Agents with Verbal Reinforcement Learning [2303.11366]
- ReflexGrad: Three-Way Synergistic Architecture for Zero-Shot Generalization in LLM Agents [2511.14584]
- Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent [2509.03990]
- MAR: Multi-Agent Reflexion Improves Reasoning Abilities in LLMs [2512.20845]
- Secure Code Generation at Scale with Reflexion [2511.03898]
- ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning [2603.05863]
- Reflexion in mathematical models of decision-making [1801.07121]
- ReflCtrl: Controlling LLM Reflection via Representation Engineering [2512.13979]
- A Reflective Approach to Providing Flexibility in Application Distribution [1006.5643]
- Narrative-Centered Emotional Reflection: Scaffolding Autonomous Emotional Literacy with AI [2504.20342]

---

The Reflexion Framework, across all its instantiations, systematizes recursive self-reference and structured introspection as a means of achieving flexible, sample-efficient, compositional, and robust adaptation in software, agentic, mathematical, and affective systems. Empirical and mathematical evidence indicates that such reflexive mechanisms, if properly controlled or structured, can overcome performance ceilings, eliminate repeated failure loops, and yield provable or observed improvements in adaptability, correctness, and task completion.

Source: https://www.emergentmind.com/topics/reflexion-framework