ENPIRE: Autonomous Robot Policy Self-Improvement

• ENPIRE is a modular framework that automates real-world robot policy improvement by integrating reset, execute, verify, and refine loops.
• It leverages agentic code evolution, closed-loop verification, and parallel rollouts to achieve near-perfect success rates in complex manipulation tasks.
• The architecture employs decentralized Git-based coordination and sensor-driven verification to scale policy evolution with minimal human oversight.

ENPIRE (Agentic Robot Policy Self-Improvement in the Real World) is a modular harness framework designed to enable coding agents to autonomously improve physical robot policies through a closed-loop process. The architecture abstracts the real-world robot learning cycle—reset, execute, verify, and refine—by providing structured interfaces and automation for scene management, policy evolution, parallel rollouts, and log-driven algorithmic improvement. The system achieves high levels of dexterity and sample efficiency while minimizing direct human intervention, representing a practical and scalable methodology for advancing autonomous robotics in real environments (Xiao et al., 18 Jun 2026).

1. Modular Architecture and Closed-Loop Policy Evolution

ENPIRE decomposes the problem of real-world robot learning into four core, reusable modules that together instantiate a repeatable feedback loop:

• Environment (EN): Facilitates automatic reset and verification routines. Scene reset is executed via a programmatic series of perception and manipulation tool calls (e.g., SAM3 segmentation, cuRobo motion planning, AnyGrasp pose sampling, torque-verified grasp), restoring the robot workspace to a predetermined start state. Automated verification employs an agent-synthesized binary reward function $v(\tau)\in\{0,1\}$ derived from limited demonstration data, operating over live sensor streams to assign credit and terminate episodes.
• Policy Improvement (PI): Maintains and iteratively refines a minimal policy training codebase, supporting behavior cloning (BC), offline/online reinforcement learning (RL), heuristic action selection, and hybrid strategies. Coding agents autonomously edit code, launch experiments, introspect on results, and commit improvements, with only higher-performing variants propagated.
• Rollout (R): Exposes fixed Gym-style APIs to facilitate policy execution. Rollouts take the form of sequential sensor readings and action commands, integrated with the EN module for reward/verification assignment. All data (trajectories, visual streams, rewards) are systematically logged for subsequent analysis.
• Evolution (E): Implements a decentralized, multi-agent protocol backed by Git workflows. Agents push branches, cherry-pick or merge ideas, and iterate on hypotheses. Periodic log ingestion allows identification of failure modes, prompting literature consultation, training infrastructure updates, and algorithmic refinements.

This closed-loop procedure, summarized as reset–execute–verify–refine, systematically transforms physical policy improvement into a controllable, automatable optimization process.

2. Mathematical Formulation and Verification

The ENPIRE optimization objective seeks a policy $\pi^*$ that maximizes success rate:

$$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$

Empirical estimation over $N$ episodes,

$$\hat \rho(\pi) = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\{ v(\tau_i) = 1 \}, \quad \tau_i \sim \text{rollout}(\pi)$$

For RL-based optimization, this binary verification yields a sparse reward structure:

$$J(\pi) = \mathbb{E} \left[ \sum_{t=0}^{T-1} \gamma^t r_t \right], \quad r_t = \begin{cases} 1, & t = T-1 \text{ and } v(\tau) = 1 \ 0, & \text{otherwise} \end{cases}$$

ENPIRE also introduces a "retries" metric—allowing up to 8 attempts with resets—to estimate episode-level success:

$$1 - \prod_{k=1}^8 \bigl( 1 - p(\text{success} \mid \pi)\bigr)$$

Reported results focus on single-trial success under this reset-and-retry semantics, such that recovery behaviors within $\pi$ are credited.

3. Operational Loops and Policy Self-Improvement Algorithms

ENPIRE’s core interaction is realized through a sequence of modular loops that automate the policy improvement cycle:

Automated Reset & Verification Pipeline

$$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$2

Policy Refinement (Single Agent)

$$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$3

Parallel Rollouts & Data Collection

$$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$4

Evolution (Log-Driven Hypothesis Selection)

$$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$5

These routines instantiate distributed, asynchronous self-improvement with direct links from log analysis to codebase mutation.

4. Hardware Implementation and Task-Specific Reward Design

ENPIRE experiments utilize eight decentralized bimanual I2RT YAM manipulators (6-DoF arms, 1-DoF torque-limited grippers), with observation and action rates of 100 Hz (control) and 30 Hz (policy inference, EN APIs). Visual input is aggregated from a top-down RealSense D405, two wrist-mounted D405s, and, for GPU insertion, a side-mounted D435i. Observations $s_t$ include RGBD images, joint angles $q_t$, velocities $\pi^*$0, and gripper torque data; actions $\pi^*$1 specify $\pi^*$2 (joint commands) or end-effector targets, with direct gripper torque or binary open/close commands.

Reward and verification logic—auto-discovered and implemented by agents—exemplifies modular abstraction:

• Pin insertion: Combines visual alignment (distance pin tip to hole center), insertion depth (proprioceptive), and torque spike detection.
• Zip-tie cutting: Employs dual-view geometric tests on masks ("strap passes through head"), via multi-camera segmentation and mask overlap.
• Push-T shape: Considers both spatial proximity to goal and orientation within $\pi^*$3.
• GPU insertion: Merges bounding-box visual alignment with force/torque verification on insertion.

Illustrative pseudocode for zip-tie reward: $$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$6

5. Empirical Results and Scaling Analysis

ENPIRE demonstrates robust policy self-improvement in dexterous manipulation benchmarks:

• Heuristic learning (Push-T): Simulation yields $\pi^*$4 success in 2 hours for Codex/GPT-5.5; in the physical world, only one of three agents successively combines heuristics and lightweight BC for task completion.
• Gradient-based optimization (Pin insertion): All agents hill-climb from $\pi^*$5 to $\pi^*$6–$\pi^*$7 success. Codex achieves $\pi^*$8 in 40 minutes on eight robots, versus 90 minutes on a single robot.
• Scaling with robot fleets: Time-to-target success is nearly inversely proportional to the number of robots ($\pi^*$9 scaling up to $$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$0), with robot utilization (MRU) declining from 0.85 to 0.4, GPU utilization rising from 0.5 to 0.9, and near-linear maintenance of model token usage (MTU) until eight-agent concurrency.
• Ablation studies: Native direct image access outperforms both function-call only and visionless variants in time-to-success; Codex/GPT-5.5 surpasses Anthropic Opus 4.7 and Moonshot K2.6 in speed and final task success; reward/verification code meets real-time precision/recall requirements under a 150 ms compute budget.

6. Scalability, Limitations, and System Considerations

ENPIRE is engineered for minimal human overhead; environment construction requires several minutes of demonstration, after which all policy learning and reward or reset functions are autonomously tuned. Decentralized Git-based coordination obviates the need for a central server, enabling robot fleet parallelism. Efficiency is tracked by MRU (robot busy time) and MTU (tokens per minute), reflecting hardware and inference usage.

Documented system limitations include:

• Robot underutilization during log analysis or large-model inference phases, with declining MRU as fleet size grows.
• Superlinear token expenditure with increasing fleet size—token consumption per wall-clock speedup accelerates beyond linear scaling.
• Bottlenecks stemming from perception API reliability (e.g., SAM3 segmentation failures), necessitating active prompt-tuning or resolution adjustments by agents.
• Persistent real-world physics non-determinism, which demands an interplay of heuristic and gradient-based training—purely heuristic methods have observed failure cases.

In summary, ENPIRE’s compositional APIs—covering environment reset/verification, rollout, policy refinement, and distributed algorithmic evolution—enable contemporary coding agents to autonomously and efficiently achieve near-perfect ($$\pi^* = \arg\max_{\pi\in\Pi} \rho(\pi) \quad \text{where} \quad \rho(\pi) = \mathbb{E}_{\tau\sim p_\pi} \bigl[ \mathbf{1}\{ v(\tau) = 1 \} \bigr]$$1) performance on complex manipulation tasks in real-world settings while substantially reducing human supervision requirements (Xiao et al., 18 Jun 2026).