Reflexion Framework in AI & Decision Models

• The Reflexion Framework is a unified model that applies self-reference and recursive feedback in AI, decision theory, and software engineering to enhance adaptive performance.
• It integrates methods from LLM agents with verbal reinforcement, multi-agent debate, and rule-based meta-policy reflection to bolster accuracy and reduce repeated failures.
• Applications of the framework span secure code generation, dynamic software distribution, and mathematical strategies, achieving measurable improvements in system adaptability and task success.

The Reflexion Framework encompasses a diverse set of models and methodologies across computer science, artificial intelligence, and mathematical theory, unified by the principle that agents, programs, or systems improve their performance or adaptability by explicit self-reference, structured reflection, or recursive representations of their own state, reasoning, or design. In computational contexts, Reflexion frameworks range from reinforcement learning agents with self-verbalized feedback, to software transformation systems enabling runtime distribution adaptation, to formal models in game theory that encode hierarchical agent beliefs. This article provides a comprehensive technical survey of Reflexion frameworks as they appear in contemporary research, focusing on language-agent architectures, predicate-based rule memory for LLM agents, multi-agent reflection mechanisms, self-reflective reinforcement in code generation, and foundational decision-theoretic models.

1. Historical Roots and Core Definitions

The term “reflexion” (as distinct from “reflection” in programming) first appears in game-theoretic contexts to denote recursive models of agent beliefs and strategies. In the Reflexion Framework of decision-making, each agent maintains (potentially infinite) hierarchies of beliefs about both the environment (nature) and the beliefs/strategies of other agents. A reflexive model, in this sense, explicitly encodes not just first-order (i.e., about the world) but higher-order beliefs (what I think you think I think…) and—critically—may stratify agents by “reflexion rank,” capturing bounded recursion of reasoning layers (Novikov et al., 2018).

In the context of software and AI, reflexion has evolved to denote not only explicit modeling of self but also mechanisms whereby an agent, code generator, or distributed system inspects, critiques, or restructures its own outputs, behaviors, or architectures. This recursive orientation has become central in LLM agent research, secure code generation, distributed systems flexibility, and affective computing.

2. Reflexion in LLM Agents

2.1 Reflexion-Verbal Reinforcement Loop

The Reflexion framework for LLM agents, prominently introduced in "Reflexion: Language Agents with Verbal Reinforcement Learning" (Shinn et al., 2023), eliminates gradient-based parameter updates in favor of on-the-fly, in-context self-reflection. The core components are:

• Actor Model ($M_a$): Generates actions or reasoning steps, conditioned on current state and a buffer of textual self-reflections (episodic memory).
• Evaluator ($M_e$): Offers scalar or binary success/failure feedback after each trial.
• Self-Reflection Model ($M_{sr}$): Constructs natural language diagnoses and recommendations, based on the current trial’s trajectory and cumulative memory.
• Reflection Buffer (mem): Short-term episodic storage of prior reflection outputs, which conditions future Actor prompts.

The agent rollouts are thus:

1. Act in environment, complete a trajectory.
2. Receive success/failure.
3. Generate a self-reflection summarizing failure modes.
4. Add reflection to memory buffer, condition replay of environment on this extended prompt.

This approach causes rapid behavioral adaptation across trials. For instance, in programming, empirical results show that Reflexion-augmented GPT-4 achieves 91% pass@1 accuracy on HumanEval, compared to 80% for unaugmented GPT-4. Similar gains are observed in text-game and multi-hop QA benchmarks (Shinn et al., 2023).

2.2 Multi-Agent Reflexion and Mitigating Degeneration

Single-agent Reflexion can suffer from "degeneration of thought"—the agent replays the same error-laden chains, even when memory signals the mistake. "Multi-Agent Reflexion" (MAR) (Ozer et al., 23 Dec 2025) addresses this by instantiating multiple LLM-based personas, each bringing different critical perspectives. Each persona independently critiques a failed rollout, engages in one or more rounds of debate, and a judge (also an LLM) synthesizes the critiques into a consensus reflection. This process reliably improves both diversity of corrections and accuracy on Hard QA (HotPotQA) and coding (HumanEval) benchmarks.

3. Reflexion as External Rule Memory: Meta-Policy Reflexion

The Meta-Policy Reflexion (MPR) framework (Wu et al., 4 Sep 2025) advances the Reflexion paradigm from unstructured textual reflection to structured, externally stored memory in the form of predicate-based rules. After each episode, an LLM parses failure trajectories into first-order logic predicates over state-action pairs (meta-policy memory $\mathcal{M}$), each paired with a confidence score. At inference, rules in $\mathcal{M}$ that match the current state bias or constrain action decoding:

• Soft Decoding: Matched predicates are prepended to the LLM prompt, influencing sampling.
• Hard Admissibility Checks: Candidate actions violating "forbid" rules are vetoed.

This approach guarantees persistent cross-task memory, i.e., once a common error is detected and codified as a rule, it will be suppressed in future, even on novel tasks. Empirically, MPR converges to 100% task success in AlfWorld within three rounds of self-reflective rule accumulation, substantially outperforming plain Reflexion or ReAct policies (Wu et al., 4 Sep 2025).

4. Intrinsic Self-Reflexion for Code Generation and Security

4.1 ReflexiCoder: RL-Internalized Reflection

ReflexiCoder (Jiang et al., 6 Mar 2026) brings reflection inside the model weights: code generation is cast as an end-to-end trajectory encompassing initial code, reflection ("bug detected" or "optimization only"), and corrective code, all produced in a single forward pass. A bespoke RL-zero objective shapes the policy to emit well-formed "reflection" tokens, produce validated fixes, and avoid unnecessary cycles. When trained, token efficiency improves by ~40%, and correctness surpasses open and proprietary LLMs of comparable size. Notably, no external critic or test harness is needed post-training: the model has internalized its own self-debugging loop.

4.2 Reflexion for Secure Code Generation

Reflexion can be instrumental in code security tasks, as demonstrated in "Secure Code Generation at Scale with Reflexion" (Datta et al., 5 Nov 2025). Here, Reflexion operates as a three-round feedback loop:

• Each attempt, the code is scanned by an Insecure Code Detector (ICD), returning identified CWE IDs.
• The next prompt includes the code, ICD results, and a revised instruction to patch the vulnerabilities.
• Repair, regression, and net gain metrics (see original for formal LaTeX definitions) trace progress across rounds.

Experiments reveal most security improvement is achieved in the first Reflexion iteration (~+6 pp), with diminishing returns thereafter. Reflexion increases secure generation across all tested LLMs; Python sees the highest secure rates, C and C# the lowest.

5. Mathematical Reflexion: Decision Theory, Games, and Learning

Within mathematical modeling, as in (Novikov et al., 2018), the Reflexion Framework formalizes agents that reason about not just the state of nature but also about each other's beliefs and reasoning procedures to arbitrary depth.

• Awareness structure: For each agent $i$, this is the infinite hierarchy $\{\theta_{ij_1...j_l}\}$ denoting $i$'s beliefs about nature, about others' beliefs, and so forth.
• Strategic reflexion: Agents may have beliefs about others' strategic sophistication—e.g., level-$k$ reasoning (level-0: default, level-1: best respond to level-0, etc.).
• Reflexive equilibrium: A solution profile where each agent's strategy is optimal given their entire reflexive belief hierarchy.

Dynamic (learning) versions integrate such models with best-response or reinforcement dynamics stratified by reflexion rank. This captures real observed behaviors in strategic learning, teaching, and multi-agent adaptation in distributed control or economic settings.

6. Control and Analysis of Reflexion in LLMs

ReflCtrl (Yan et al., 16 Dec 2025) investigates self-reflection as a latent behavior in chain-of-thought LLMs. By segmenting LLM outputs into "inference" and "reflection" steps and extracting reflection directions in the model's latent space, ReflCtrl enables fine-grained control:

• Stepwise steering: Adjusts the frequency of reflection by moving activations along the reflection direction; can suppress or enhance self-reflection as desired.
• Empirical findings: Suppressing reflection can yield up to 33.6% token savings with negligible accuracy loss on reasoning and coding tasks (GSM8k, MMLU, MATH-500).
• Interpretability: The activation in reflection subspace correlates with model uncertainty, suggesting LLMs’ decision to reflect is functionally tied to internal measures of uncertainty (Yan et al., 16 Dec 2025).

7. Reflexion in Other Computational Paradigms

7.1 Reflective Software Distribution: RAFDA

The RAFDA framework (Portillo et al., 2010) applies reflexion principles to software engineering by automatically transforming ordinary Java programs into distribution-agnostic systems. RAFDA operates by:

• Bytecode transformation to extract interfaces, relocate constructor logic, and replace direct field/method access with interface/ factory calls.
• Generating both local and remote (proxy) implementations for each class. Runtime policy manager selects local/proxy instantiations based on network/ resource metrics.
• Decoupling application logic from distribution, enabling dynamic migration of objects across JVM boundaries via flexible policy.

While no performance or scalability results are given, the resulting system allows runtime alteration of distribution architecture without code or recompilation, a direct outcome of reflective transformation (Portillo et al., 2010).

7.2 Affective Computing: Layered Self-Reflection

Beyond agentic or structural contexts, Reflexion is also instantiated in affective AI systems (Han, 29 Apr 2025). Here, Reflexion consists of staged guidance to users through explicit emotional awareness, reframing, values alignment, and action planning, with LLMs generating metaphoric narratives for psychological distancing and growth.

• Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023)
• ReflexGrad: Three-Way Synergistic Architecture for Zero-Shot Generalization in LLM Agents (Kadu et al., 18 Nov 2025)
• Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent (Wu et al., 4 Sep 2025)
• MAR: Multi-Agent Reflexion Improves Reasoning Abilities in LLMs (Ozer et al., 23 Dec 2025)
• Secure Code Generation at Scale with Reflexion (Datta et al., 5 Nov 2025)
• ReflexiCoder: Teaching LLMs to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning (Jiang et al., 6 Mar 2026)
• Reflexion in mathematical models of decision-making (Novikov et al., 2018)
• ReflCtrl: Controlling LLM Reflection via Representation Engineering (Yan et al., 16 Dec 2025)
• A Reflective Approach to Providing Flexibility in Application Distribution (Portillo et al., 2010)
• Narrative-Centered Emotional Reflection: Scaffolding Autonomous Emotional Literacy with AI (Han, 29 Apr 2025)

---

The Reflexion Framework, across all its instantiations, systematizes recursive self-reference and structured introspection as a means of achieving flexible, sample-efficient, compositional, and robust adaptation in software, agentic, mathematical, and affective systems. Empirical and mathematical evidence indicates that such reflexive mechanisms, if properly controlled or structured, can overcome performance ceilings, eliminate repeated failure loops, and yield provable or observed improvements in adaptability, correctness, and task completion.