---
title: Act-Observe-Rewrite (AOR) Framework
url: https://www.emergentmind.com/topics/act-observe-rewrite-aor
type: topic
---

# Act-Observe-Rewrite (AOR) Framework

Act-Observe-Rewrite (AOR) is a general framework for adaptive agent behavior characterized by a closed-loop process of action execution, outcome observation, and systematic policy revision. The distinctive property of AOR is that policy improvement occurs through explicit, interpretable modifications to the decision logic or control code—rather than through gradient-based updates, black-box learning, or predefined skill selection. Originally formalized in the context of multimodal coding agents for robotic manipulation and active observer models in human vision, AOR provides a rigorous substrate for in-context learning, systematic failure diagnosis, and hypothesis-driven adaptation in both artificial and biological agents [2603.04466], [2108.08145].

## 1. Core Principles and Formalization

AOR consists of three canonical steps, repeated cyclically:

1. **Act**: The agent executes its current policy or strategy in the environment.
2. **Observe**: The agent collects outcome data, including both raw sensory observations and structured diagnostic metrics.
3. **Rewrite**: The agent revises its policy or strategy—often as fully executable code—by reasoning about observed failures and their systematic causes.

Formally, this cycle operates over a history $H_n = \{(\tau_1, I_1), ..., (\tau_n, I_n)\}$, where $\tau_k$ denotes episode outcomes (reward $r\in\{0,1\}$, diagnostic signals $\phi_i$, step count $T$), and $I_k$ are associated key-frame observations. The policy at iteration $n$, $\pi_n$, is typically representable as source code (e.g., a Python class with control logic).

Policy update is performed by an LLM or planning module:
$$
\pi_{n+1} = \text{LLM}(H_n, \text{source}(\pi_n))
$$
This reframes in-context learning as symbolic reasoning and code synthesis, in contrast to parametric gradient descent.

In cognitive modeling, AOR is instantiated as a hypothesize–test–replan loop, where belief updates use Bayesian inference:
$$
P(h|O_{1:t}) = \frac{P(O_t|h,S_t)P(h|O_{1:t-1})}{\sum_{h'}P(O_t|h',S_t)P(h'|O_{1:t-1})}
$$
and action (method) selection is cast as maximizing expected information gain minus cost.

## 2. Algorithmic Structure in Robotic Manipulation

The Act-Observe-Rewrite loop is implemented in two nested frequencies:

- **Fast loop (control frequency, intra-episode):**
   - Executes the current Python policy $\pi_n$ stepwise. At each timestep $t$, state features $f_t$ are computed from RGB-D observations, actions $a_t = \pi_n.\text{get\_action}(f_t)$ are determined, safety clamps applied, and the result commanded to the robot.
   - Key observations and transitions are stored for diagnostic analysis.

- **Slow loop (reasoning frequency, inter-episode):**
   1. Computes episode outcome $\tau_n = \{r, T, \phi_1, ..., \phi_k\}$.
   2. Updates history $H_n$.
   3. Prompts a multimodal coding LLM with $\pi_n$’s source, $H_n$, key-frame images, and targeted questions regarding failure modes and root causes.
   4. Receives a revised controller class $\pi_{n+1}$; verifies its safety and correctness.
   5. Iterates until successful execution or call budget exhaustion [2603.04466].

A typical generated controller is a finite-state machine with explicit PID-style proportional control, safety clamping, and parameterizations (e.g., gains, thresholds) that are subject to direct revision.

## 3. AOR in Human Active Vision and Cognitive Programs

Solbach & Tsotsos empirically demonstrated the AOR cycle in human 3D visual problem solving [2108.08145]. Participants in a same–different block structure task cycled between:

- **Act:** Hypothesizing and deploying visual/comparative strategies such as "divide-and-conquer" or "alternating fixation," each mapped to Cognitive Program (CP) methods.
- **Observe:** Acquiring new visual input via eye fixations (mean ≈ 92.4 per trial) and attentional feature extraction, updating confidence in hypotheses.
- **Rewrite:** Switching strategies or parameterizations if confidence thresholds are unmet, dynamically composing new methods with Bayesian action selection and replanning criteria.

Cognitive Programs embody the methods and scripts (≈50 elemental methods mined from behavioral data) supporting this cycle, with the STAR architecture providing working memory, executive control, and the data structures necessary for adaptive strategy deployment.

## 4. Empirical Validation and Benchmark Tasks

In robotic manipulation, AOR was validated on three robosuite tasks:

| Task           | Scene & Challenge                 | LLM Calls | Final Success Rate |
|----------------|----------------------------------|-----------|--------------------|
| Lift           | Single cube, depth/camera bias    | 3         | 100%               |
| PickPlaceCan   | Can & bin, color/vision failure   | 2         | 100%               |
| Stack          | Two cubes, vision conventions     | 20        | 91%                |

- **Lift**: Success reached after correcting a systematic depth bias and adding stationary grasp logic.
- **PickPlaceCan**: Vision masking and component filtering revised to identify objects robustly.
- **Stack**: Multiple controller revisions addressed camera y-flip, matrix conventions, grasp retries, and placement collisions, achieving high but not perfect performance due to residual local optima [2603.04466].

In the human active observer domain, subjects achieved 93.8% accuracy (σ=3.9%), with zero detectable learning effect over 18 consecutive trials, suggesting reliance on dynamic AOR-driven strategy composition rather than cumulative parametric learning [2108.08145].

## 5. Interpretability and Systematic Failure Diagnosis

A distinctive property of AOR is its reliance on interpretable policy representations. Policy as code enables:

- Direct localization of systematic errors—“line 27: y_p is negated but robosuite uses OpenGL conventions”—and causal patching.
- Expressive changes such as insertion of new behavioral phases, logic modifications, or algorithmic modules in a single LLM call.
- Diagnosis and rapid correction of both vision and control failures, in contrast to opaque neural policies that support only incremental weight updates.

Opaque or black-box policies lack the ability to expose internal causal mechanisms to the agent’s reasoning process, reducing failure diagnosis to coarse reward or performance metrics [2603.04466].

## 6. Comparative Analyses, Limitations, and Prospects

AOR efficacy depends critically on the underlying reasoning and code synthesis capabilities of the LLM. Empirically, Codex GPT-5.3 did not solve any tasks in the same budget on the benchmark suite, establishing a lower bound on required LLM competence [2603.04466]. Safety and stability ablations demonstrated that the compile sandbox and action clamping are necessary for robust policy iteration.

In human cognitive modeling, while the AOR framework successfully models dynamic strategy adaptation, limitations include the restriction to same–different block tasks and the absence of fully quantitative Selective Tuning model integration. No learning or transfer across tasks was detected [2108.08145].

*This suggests* that AOR provides a general and interpretable template for in-context, non-parametric policy improvement in both artificial and biological agents. The framework’s reliance on explicit reasoning about policy representations marks a qualitative shift from reinforcement-based or parametric learning, with direct implications for robotic autonomy, human–machine interaction, and models of executive control.

## 7. Extensions and Future Directions

Proposed extensions include:

- Application of AOR mechanisms to broader classes of visuomotor and spatial-relation tasks in robotics and neurocognitive modeling.
- Closing the perception–action loop by deploying Cognitive Programs in simulation or on physical robots.
- Automatic learning of diagnostic metrics, cost models, and method libraries from demonstration data to optimize AOR policy search.
- Systematic investigation into the types of failure amenable to single-shot code-level intervention and the scaling properties of AOR as task complexity increases [2603.04466], [2108.08145].

Source: https://www.emergentmind.com/topics/act-observe-rewrite-aor