EgoPush: Egocentric RL for Object Rearrangement

• EgoPush is a framework that employs multi-stage reinforcement learning with object-centric latent representations to address long-horizon non-prehensile object rearrangement.
• It uses a two-phase teacher–student architecture where a privileged teacher guides a visual student via relational distillation and action cloning under severe partial observability.
• Its staged reward assignment and constrained teacher approach enable robust sim-to-real transfer and outperform classical, end-to-end baselines in sequential manipulation tasks.

EgoPush is a reinforcement learning framework designed for long-horizon egocentric perception-driven multi-object non-prehensile rearrangement by mobile robots operating under severe partial observability. Unlike prior approaches that rely on global coordinate estimation or top-down state information, EgoPush achieves robust sequential object manipulation using only a single forward-facing RGB-D camera and object masks derived from egocentric observations. The key contributions of EgoPush include an object-centric latent space encoding of semantic groups, a constrained privileged teacher for reinforcement learning, a staged student policy distilled with relational objectives, and problem decomposition for reward assignment over extended tasks. Performance is demonstrated to exceed previous classical and end-to-end RL baselines, including successful zero-shot sim-to-real transfer on a real robotic platform (An et al., 20 Feb 2026).

1. System Overview and Problem Setting

The EgoPush setup consists of a differential-drive mobile robot, equipped with a single forward–facing RGB-D sensor, with no access to reliable external localization (such as odometry or SLAM). The robot manipulates several objects in cluttered tabletop environments, where objects must be sequentially pushed into specified patterns (such as cross or line configurations) relative to a designated anchor. Each manipulation stage requires that an "active" object be reached, aligned within tolerance $\epsilon_{\text{align}}$ to the anchor, and oriented within $\epsilon_{\Phi}$ of the target yaw.

The primary challenges addressed by EgoPush include:

• Partial Observability: Frequent occlusions and limited field of view make global scene understanding infeasible.
• No Global State: The absence of reliable global robot pose or map prohibits standard navigation and manipulation pipelines.
• Long-Horizon Credit Assignment: Multiple sequential reach-and-place actions with sparse reward signals.
• Teacher–Student Gap: Privileged teachers with omniscient access are prone to generating unrecoverable behaviors for real egocentric observation constraints.

2. Object-Centric Latent Representations

EgoPush abstracts the manipulated scene into three semantic groups: active object ("act"), anchor object ("anc"), and obstacles ("obs"). During the teacher phase, virtual reference keypoints ("ref") reflecting the goal configuration are included.

• Teacher Phase: Receives, for every group $$k \in \{\text{act}, \text{anc}, \text{obs}, \text{ref}\}$$, a set of sparse 3D keypoints $P_t^k$ filtered by a virtual camera frustum representing the robot's instantaneous field of view. Each set is padded or truncated to a uniform size.
• PointNet Embeddings: The point sets are processed by a PointNet to produce group latents $$Z_t^k = \mathrm{PointNet}_\theta(P_t^k) \in \mathbb{R}^d$$, with the full scene latent formed by concatenation across groups.
• Student Phase: Semantic grouping occurs on masked depth images. For group $k$, a composite depth map $\tilde{d}_t^k$ is produced using instance masks $M_t^{(i)}$ from RGB-based segmentation. The student's latent embedding is generated via a CNN.

EgoPush introduces a relational distillation mechanism, where cosine similarity matrices $$S_{i, j} = \frac{Z^i \cdot Z^j}{\|Z^i\|_2 \|Z^j\|_2}$$ for $i, j \in \{\text{act}, \text{anc}, \text{obs}\}$ are used to encode relative spatial structure. The student is directly supervised to match these relational measures, bridging visual and keypoint-derived latent spaces.

3. Two-Phase Reinforcement Learning and Policy Transfer

EgoPush employs a teacher–student policy learning architecture:

• Phase 1: Privileged RL Teacher
  • Keypoints outside the virtual egocentric FOV are replaced by null vectors.
  • Reference keypoints $\epsilon_{\Phi}$1 are revealed only when the anchor object remains within a gated central region of the image, enforcing viewpoint-dependent grounding.
  • The teacher is trained via Proximal Policy Optimization (PPO), exploiting extensive domain randomization in simulation.
• Phase 2: Visual Student Policy and Distillation The student policy receives only RGB-D images, segmented to produce groupwise depth masks. Policy heads are trained by cloning the teacher's actions (action cloning loss $\epsilon_{\Phi}$2), with an additional relational distillation loss $\epsilon_{\Phi}$3 enforcing correspondence of inter-group similarities between teacher and student latents. Training is performed with a DAgger-style online supervised update, with further domain randomization to promote transferability.

Stage	Input Modality	Latent Encoding	Policy Head
Teacher	3D keypoints	PointNet embeddings	MLP
Student	Egocentric RGB-D	CNN on masked depths	MLP

4. Active Perception via Constrained Teacher

Key to the transferability of policies is the "constrained teacher" approach:

• The teacher's access to reference keypoints is limited to when the anchor is centrally visible, discouraging policies that push objects out of view.
• Off-screen and occluded keypoints are hidden by FOV and gating restrictions.
• Policies learned under these constraints exhibit active perception behaviors—in particular, deliberately reorienting the robot to reacquire anchor visibility prior to pushing.

Ablation studies show that unconstrained teachers (with global access to all keypoints) yield irrecoverable student policies with 0% success rate, while FOV-only constraints yield brittle behaviors (21% SR). The adoption of both FOV and central gating results in robust student performance.

5. Stage-Decomposed Reward Assignment for Long-Horizon Tasks

EgoPush formalizes the long-horizon rearrangement challenge as a sequence of "stages," each corresponding to an object reach-and-place subtask. Each stage is budgeted by a timer $\epsilon_{\Phi}$4 and is associated with time-decayed completion rewards:

• Reach Completion Reward:

$\epsilon_{\Phi}$5

• Place Completion Reward:

$\epsilon_{\Phi}$6

• Shaping and Penalties: Progressive distance reduction, smoothness costs, and target-proximal slowdown rewards.

Ablations demonstrate that:

• Exclusive terminal rewards yield 16% SR.
• Adding per-stage rewards without decay improves SR to 87.5%.
• Introducing a global decay timer raises SR to 97.9%.
• The stage timer (as implemented) achieves 98.6% SR in half the steps for two-object arrangements.

6. Evaluation: Simulation and Real-World Performance

Experiments are conducted in Isaac Lab at large scale, with 8,192 environments for teacher PPO and 512 for distillation. Environments contain rigid objects (cube, cylinder, prism; base/diameter 15 cm, height 15 cm) and a robot (body: 14×14×14.3 cm; pusher: 7.5 cm; RGB-D sensor: 69° FOV, 240×180 px).

Performance in the simplified two-object cross task:

• EgoPush achieves 100% reach and 100% success rate (SR), with an average trajectory length of 4.66 steps, whereas all tested end-to-end RL baselines achieve under 1% SR.
• Classical SIM baseline (with odometry) achieves 19% SR.

Shape generalization on the student policy shows robust performance:

• Cylinder: 99.4% reach, 67.5% SR.
• Prism: 100% reach, 54.3% SR.

Real-world zero-shot sim-to-real transfer is demonstrated on TurtleBot3 Burger with RealSense D435i and Jetson Nano, with 10 episodes of a five-box cross task yielding 80% SR under a loose metric. Depth data is processed with Navier–Stokes inpainting and simulated noise injection; colored box segmentation is performed with HSV thresholding.

Baseline	Reach (%)	SR (%)	Notes
EgoPush	100	100	Two-object cross (sim)
SIM	30	19	Odometry required
E2E-RGB(-D)	<1	<1	End-to-end RL

7. Significance and Design Implications

EgoPush’s approach—combining object-centric latent representations, a constrained privileged RL teacher, relational student distillation, and staged time-decayed rewards—demonstrates that robust, long-horizon, non-prehensile manipulation is achievable in highly partial-observable, egocentric settings without reliance on global state estimation. The relational distillation explicitly bridges the representation gap between privileged and visual modalities, and the teacher constraints induce behaviors that are visually recoverable by the student. Results indicate that each design element is integral for performance, with ablations validating their necessity for both simulation and real-world tasks. The methodology suggests further potential for generalization to other tasks involving occlusions, partial views, and lengthy sequential subgoals in robotics (An et al., 20 Feb 2026).