EgoPush Framework: Egocentric Rearrangement

• EgoPush is a framework for end-to-end, perception-driven multi-object non-prehensile rearrangement that uses object-centric relational reasoning instead of explicit global state estimation.
• It employs a two-stage process where a privileged teacher is trained via PPO and its policy is distilled into a camera-based visual student, enabling effective sim-to-real transfer.
• The framework leverages hierarchical reward shaping, constrained active perception, and latent representation learning to achieve state-of-the-art success in simulated and real robotic tasks.

EgoPush is a framework for end-to-end learning of egocentric, perception-driven multi-object non-prehensile rearrangement in mobile robotics. The architecture eliminates explicit global state estimation in favor of object-centric relational reasoning and distills policies from a privileged reinforcement learning teacher to a camera-based visual student. The pipeline emphasizes latent representation, constrained active perception, hierarchical (stage-wise) reward shaping, and a supervised distillation regime to bridge the gap between fully observed synthetic training and partially observed real robot deployment. EgoPush achieves state-of-the-art success rates in simulated and real-world multi-object rearrangement tasks, demonstrating robust zero-shot sim-to-real generalization (An et al., 20 Feb 2026).

1. Object-Centric Latent Space Representation

EgoPush structures perception and decision making around a semantic partitioning of scene objects. At each time step $t$, objects are grouped as follows:

• Active object (to be rearranged in the current subtask)
• Anchor object (defines the goal reference frame)
• Obstacles (remaining objects in the scene)
• Reference target keypoints (privileged for the teacher policy)

For the privileged teacher, each group $$k \in \{\mathrm{act}, \mathrm{anc}, \mathrm{obs}, \mathrm{ref}\}$$ is represented by $P_t^k \subset \mathbb{R}^{3 \times K}$, the set of visible 3D keypoints, with non-visible (out-of-FOV) keypoints masked by a constant $\epsilon$. These observations are encoded into latent vectors via a shared PointNet module: $$Z_t^k = \mathrm{PointNet}_\theta(P_t^k) \in \mathbb{R}^d.$$

For the visual student, RGB images are segmented (e.g., instance masks $M_t^{(i)}$), followed by aggregation of depth into three semantic layers $\tilde d_t^k$. Each is encoded via a CNN: $$\widehat{Z}_t^k = \mathrm{CNN}_\phi(\tilde d_t^k) \in \mathbb{R}^d,\quad k \in \{\mathrm{act}, \mathrm{anc}, \mathrm{obs}\}.$$

The relational structure is preserved via the cosine similarity matrix $S_{i, j}$ among groups: $$S_{i, j} = \frac{Z_t^i \cdot Z_t^j}{\|Z_t^i\|_2 \|Z_t^j\|_2}, \quad i, j \in \mathcal{K}.$$

2. Problem Formulation and Reward Signal Design

The framework operationalizes multi-object rearrangement as an episodic RL problem with long-horizon credit assignment. The observation and action spaces differ for teacher and student:

• Teacher: Four groups of masked keypoints plus previous action.
• Student: RGB-based segmented depth layers for three groups plus previous action.

The robotic action space is $a_t = [v_t, \omega_t] \in \mathbb{R}^2$ representing differential-drive velocities.

The reward function $r_t$ combines:

• Stage-aligned completion rewards:
  • Reach stage:

  $$r_{\mathrm{reach}, t} = \eta_t\,\mathbf{1}[\|p_{\mathrm{rbt}, t} - p_{\mathrm{act}, t}\| < \varepsilon_{\mathrm{reach}}]$$ - Place stage:

  $$r_{\mathrm{place}, t} = \eta_t\,\mathbf{1}\bigl[\|p_{\mathrm{act}, t} - p_{\mathrm{ref}, t}\| < \varepsilon_{\mathrm{align}} \wedge |\Delta \psi_t| < \varepsilon_\Phi \wedge \|v_{\mathrm{act}, t}\| < \varepsilon_v\bigr]$$

  with $\eta_t = (T_s - \tau_t)/(T_s + \epsilon_0)$ a stage-local temporal decay factor.

• Progress shaping:

$$r_{\mathrm{dist}, t} = \begin{cases} w_{\mathrm{rbt}} (d_{\mathrm{rbt}, t-1} - d_{\mathrm{rbt}, t}), & g_t = 0 \ w_{\mathrm{ref}} (d_{\mathrm{ref}, t-1} - d_{\mathrm{ref}, t}), & g_t = 1 \end{cases}$$

gating by substage $g_t$ (0: reach, 1: place).

• Smoothness penalties and slowdown near target.

The teacher's policy is trained to maximize $J(\theta) = \mathbb{E}[\sum_t \gamma^t r_t]$ using PPO, under extensive domain randomization.

3. Privileged Teacher and Policy Distillation

The privileged teacher leverages access to sparse, near noise-free 3D scene keypoints but is explicitly constrained to reduce the supervision gap:

• Field-of-view masking: Keypoints out of egocentric FOV are suppressed.

• Center-gated anchor: Reference target points are only visible when the anchor is centered, forcing the teacher to maintain visual lock on subgoals.

• Policy network: Shared PointNet encoders for all groups, with concatenated latents and previous action passed through an MLP.

Policy distillation employs a DAgger-style online setup:

• At each roll-out, the student observes $o_t^{\mathrm{stu}}$, produces $\widehat{a}_t^{\mathrm{stu}}$, and queries the teacher for optimal $a_t^{\mathrm{tea}}$ plus teacher latents $Z_t^k$.

• The loss combines:

  • Action cloning:

  $$\mathcal{L}_{\mathrm{act}} = \|\widehat{a}_t - a_t^{\mathrm{tea}}\|_2^2$$ - Relational distillation:

  $$\mathcal{L}_{\mathrm{rel}} = \|S_t^{\mathrm{tea}} - \widehat{S}_t^{\mathrm{stu}}\|_F^2$$

• Total loss: $$\mathcal{L} = \mathcal{L}_{\mathrm{act}} + \lambda \mathcal{L}_{\mathrm{rel}}$$ (for tunable $\lambda$).

• Student policy head is warm-started from teacher head; CNN encoder for perceptual mapping is learned from scratch.

• Camera pose randomization is applied to support sim-to-real transfer.

4. Active Perception via Constrained Observations

By restricting teacher observations to respect egocentric visibility and anchor centering, the framework induces explicit active perception behaviors:

• The agent must re-center visual attention on the anchor before proceeding with the push action, mirroring the partial observability constraints of onboard cameras.

• This ensures recovery of teacher-induced behavior by a perception-constrained camera-based student.

• Empirical results demonstrate the significance: students distilled from teachers without FOV or anchorcentering constraints exhibit catastrophic failure (<1% success), whereas constrained setups achieve >70% task success rates (An et al., 20 Feb 2026).

5. Long-Horizon Decomposition and Credit Assignment

EgoPush addresses the challenges of delayed reward and temporal abstraction by decomposing multi-object rearrangement into a fixed sequence of $N$ stages (corresponding to $N$ objects per episode), each consisting of a reach-and-place subtask:

• At each stage, the local timer $\tau_t$ is reset.

• Stage-specific, temporally decayed completion factors provide direct incentive for efficiency.

• Phase-gated progress shaping rewards further yield dense learning signals even in long episodes.

• Ablation studies confirm that stage-aligned reward decay enables faster convergence and higher overall task success, outperforming both sparse terminal rewards and episode-level shaping.

6. Empirical Results and Simulation-to-Real Transfer

Simulated evaluations use NVIDIA Isaac Lab over 8,192 parallel environments (teacher PPO) and 512 (student distillation):

• Task objects include cubes, cylinders, and prisms (15 cm scale); the robot is equipped with a 69° FOV RGB-D camera and a 7.5 cm pusher.

• On a canonical 4-box cross-shape rearrangement:

  • Teacher PPO achieves 99% success.
  • Distilled student reaches 70.7% success (≈100% reach sub-task completion), with ablations showing success dropping to near zero without visual constraints.
  • End-to-end RL methods from RGBD/masks/RNNs achieve ≪1% success.
  • Odometry-based Spatial Intention Maps baseline achieves ≈19% success, limited by drift and contact errors.

Generalization to unseen object geometries is validated (cylinder: 67.5%, prism: 54.3%). Zero-shot deployment on TurtleBot3 in the real world yields 80% success over 10 five-object episodes.

7. Algorithmic Pipeline

The EgoPush training regime is structured as:

Algorithm EgoPush

Phase 1: Constrained-Teacher RL
  Initialize teacher parameters θ
  repeat
    for each of M parallel envs do
      simulate roll-out under θ with masked keypoints
      collect (oₜ, aₜ, rₜ)
    end
    update θ via PPO using collected trajectories
  until teacher converges

Phase 2: Online Student Distillation
  Initialize student encoder φ, policy head ψ ← teacher head
  for iter = 1…N do
    for each of M' parallel envs do
      rollout student: observe oₜ^stu, take aₜ^stu = π_{φ,ψ}(oₜ^stu)
      query teacher at same state: get aₜ^tea, latents Zₜ^tea
      compute student latents \hat Zₜ^stu
      compute L_act = ||aₜ^stu − aₜ^tea||²
      compute S^tea, S^stu and L_rel = ||S^tea − S^stu||_F²
      L = L_act + λ L_rel
      update (φ,ψ) via [SGD](https://www.emergentmind.com/topics/differentially-private-stochastic-gradient-descent-sgd) step on L
    end
  end
Deployment
  Run student policy π_{φ,ψ} on real robot with RGB→mask→depth preprocessing.

8. Significance, Limitations, and Implications

EgoPush demonstrates that egocentric, object-centric, relation-driven policy amalgamation is tractable for long-horizon, multi-object non-prehensile rearrangement without explicit global localization, with privileged-to-student distillation and constrained supervision identified as critical enablers of real-world transfer. The success of stage-local temporal shaping and relational latent structure emphasizes the value of temporal and semantic abstraction in reinforcement learning for manipulation. All claims and results above are reported in (An et al., 20 Feb 2026).