---
title: 'EgoPush Framework: Egocentric Rearrangement'
url: https://www.emergentmind.com/topics/egopush-framework
type: topic
---

# EgoPush Framework: Egocentric Rearrangement

EgoPush is a framework for end-to-end learning of egocentric, perception-driven multi-object non-prehensile rearrangement in mobile robotics. The architecture eliminates explicit global state estimation in favor of object-centric relational reasoning and distills policies from a privileged reinforcement learning teacher to a camera-based visual student. The pipeline emphasizes latent representation, constrained active perception, hierarchical (stage-wise) reward shaping, and a supervised distillation regime to bridge the gap between fully observed synthetic training and partially observed real robot deployment. EgoPush achieves state-of-the-art success rates in simulated and real-world multi-object rearrangement tasks, demonstrating robust zero-shot sim-to-real generalization [2602.18071].

## 1. Object-Centric Latent Space Representation

EgoPush structures perception and decision making around a semantic partitioning of scene objects. At each time step $t$, objects are grouped as follows:

- **Active object** (to be rearranged in the current subtask)
- **Anchor object** (defines the goal reference frame)
- **Obstacles** (remaining objects in the scene)
- **Reference target keypoints** (privileged for the teacher policy)

For the privileged teacher, each group $k \in \{\mathrm{act}, \mathrm{anc}, \mathrm{obs}, \mathrm{ref}\}$ is represented by $P_t^k \subset \mathbb{R}^{3 \times K}$, the set of visible 3D keypoints, with non-visible (out-of-FOV) keypoints masked by a constant $\epsilon$. These observations are encoded into latent vectors via a shared PointNet module:
\[
Z_t^k = \mathrm{PointNet}_\theta(P_t^k) \in \mathbb{R}^d.
\]

For the visual student, RGB images are segmented (e.g., instance masks $M_t^{(i)}$), followed by aggregation of depth into three semantic layers $\tilde d_t^k$. Each is encoded via a CNN:
\[
\widehat{Z}_t^k = \mathrm{CNN}_\phi(\tilde d_t^k) \in \mathbb{R}^d,\quad k \in \{\mathrm{act}, \mathrm{anc}, \mathrm{obs}\}.
\]

The relational structure is preserved via the cosine similarity matrix $S_{i, j}$ among groups:
\[
S_{i, j} = \frac{Z_t^i \cdot Z_t^j}{\|Z_t^i\|_2 \|Z_t^j\|_2}, \quad i, j \in \mathcal{K}.
\]

## 2. Problem Formulation and Reward Signal Design

The framework operationalizes multi-object rearrangement as an episodic RL problem with long-horizon credit assignment. The observation and action spaces differ for teacher and student:

- **Teacher:** Four groups of masked keypoints plus previous action.
- **Student:** RGB-based segmented depth layers for three groups plus previous action.

The robotic action space is $a_t = [v_t, \omega_t] \in \mathbb{R}^2$ representing differential-drive velocities.

The reward function $r_t$ combines:

- **Stage-aligned completion rewards:**  
  - *Reach stage*:  
    \[
    r_{\mathrm{reach}, t} = \eta_t\,\mathbf{1}[\|p_{\mathrm{rbt}, t} - p_{\mathrm{act}, t}\| < \varepsilon_{\mathrm{reach}}]
    \]
  - *Place stage*:  
    \[
    r_{\mathrm{place}, t} = \eta_t\,\mathbf{1}\bigl[\|p_{\mathrm{act}, t} - p_{\mathrm{ref}, t}\| < \varepsilon_{\mathrm{align}} \wedge |\Delta \psi_t| < \varepsilon_\Phi \wedge \|v_{\mathrm{act}, t}\| < \varepsilon_v\bigr]
    \]
    with $\eta_t = (T_s - \tau_t)/(T_s + \epsilon_0)$ a stage-local temporal decay factor.
- **Progress shaping:**  
  \[
  r_{\mathrm{dist}, t} = 
    \begin{cases}
    w_{\mathrm{rbt}} (d_{\mathrm{rbt}, t-1} - d_{\mathrm{rbt}, t}), & g_t = 0 \\
    w_{\mathrm{ref}} (d_{\mathrm{ref}, t-1} - d_{\mathrm{ref}, t}), & g_t = 1
    \end{cases}
  \]
  gating by substage $g_t$ (0: reach, 1: place).
- **Smoothness penalties** and **slowdown near target**.

The teacher's policy is trained to maximize $J(\theta) = \mathbb{E}[\sum_t \gamma^t r_t]$ using PPO, under extensive domain randomization.

## 3. Privileged Teacher and Policy Distillation

The privileged teacher leverages access to sparse, near noise-free 3D scene keypoints but is explicitly constrained to reduce the supervision gap:

- **Field-of-view masking**: Keypoints out of egocentric FOV are suppressed.
- **Center-gated anchor**: Reference target points are only visible when the anchor is centered, forcing the teacher to maintain visual lock on subgoals.
- Policy network: Shared PointNet encoders for all groups, with concatenated latents and previous action passed through an MLP.

Policy distillation employs a DAgger-style online setup:

- At each roll-out, the student observes $o_t^{\mathrm{stu}}$, produces $\widehat{a}_t^{\mathrm{stu}}$, and queries the teacher for optimal $a_t^{\mathrm{tea}}$ plus teacher latents $Z_t^k$.
- The loss combines:
  - **Action cloning:**  
    \[
    \mathcal{L}_{\mathrm{act}} = \|\widehat{a}_t - a_t^{\mathrm{tea}}\|_2^2
    \]
  - **Relational distillation:**  
    \[
    \mathcal{L}_{\mathrm{rel}} = \|S_t^{\mathrm{tea}} - \widehat{S}_t^{\mathrm{stu}}\|_F^2
    \]
- Total loss: $\mathcal{L} = \mathcal{L}_{\mathrm{act}} + \lambda \mathcal{L}_{\mathrm{rel}}$ (for tunable $\lambda$).
- Student policy head is warm-started from teacher head; CNN encoder for perceptual mapping is learned from scratch.
- Camera pose randomization is applied to support sim-to-real transfer.

## 4. Active Perception via Constrained Observations

By restricting teacher observations to respect egocentric visibility and anchor centering, the framework induces explicit active perception behaviors:

- The agent must re-center visual attention on the anchor before proceeding with the push action, mirroring the partial observability constraints of onboard cameras.
- This ensures recovery of teacher-induced behavior by a perception-constrained camera-based student.
- Empirical results demonstrate the significance: students distilled from teachers without FOV or anchorcentering constraints exhibit catastrophic failure (<1% success), whereas constrained setups achieve >70% task success rates [2602.18071].

## 5. Long-Horizon Decomposition and Credit Assignment

EgoPush addresses the challenges of delayed reward and temporal abstraction by decomposing multi-object rearrangement into a fixed sequence of $N$ stages (corresponding to $N$ objects per episode), each consisting of a reach-and-place subtask:

- At each stage, the local timer $\tau_t$ is reset.
- Stage-specific, temporally decayed completion factors provide direct incentive for efficiency.
- Phase-gated progress shaping rewards further yield dense learning signals even in long episodes.
- Ablation studies confirm that stage-aligned reward decay enables faster convergence and higher overall task success, outperforming both sparse terminal rewards and episode-level shaping.

## 6. Empirical Results and Simulation-to-Real Transfer

Simulated evaluations use NVIDIA Isaac Lab over 8,192 parallel environments (teacher PPO) and 512 (student distillation):

- Task objects include cubes, cylinders, and prisms (15 cm scale); the robot is equipped with a 69° FOV RGB-D camera and a 7.5 cm pusher.
- On a canonical 4-box cross-shape rearrangement:
  - Teacher PPO achieves 99% success.
  - Distilled student reaches 70.7% success (≈100% reach sub-task completion), with ablations showing success dropping to near zero without visual constraints.
  - End-to-end RL methods from RGBD/masks/RNNs achieve ≪1% success.
  - Odometry-based Spatial Intention Maps baseline achieves ≈19% success, limited by drift and contact errors.

Generalization to unseen object geometries is validated (cylinder: 67.5%, prism: 54.3%). Zero-shot deployment on TurtleBot3 in the real world yields 80% success over 10 five-object episodes.

## 7. Algorithmic Pipeline

The EgoPush training regime is structured as:

``` 
Algorithm EgoPush

Phase 1: Constrained-Teacher RL
  Initialize teacher parameters θ
  repeat
    for each of M parallel envs do
      simulate roll-out under θ with masked keypoints
      collect (oₜ, aₜ, rₜ)
    end
    update θ via PPO using collected trajectories
  until teacher converges

Phase 2: Online Student Distillation
  Initialize student encoder φ, policy head ψ ← teacher head
  for iter = 1…N do
    for each of M' parallel envs do
      rollout student: observe oₜ^stu, take aₜ^stu = π_{φ,ψ}(oₜ^stu)
      query teacher at same state: get aₜ^tea, latents Zₜ^tea
      compute student latents \hat Zₜ^stu
      compute L_act = ||aₜ^stu − aₜ^tea||²
      compute S^tea, S^stu and L_rel = ||S^tea − S^stu||_F²
      L = L_act + λ L_rel
      update (φ,ψ) via SGD step on L
    end
  end
Deployment
  Run student policy π_{φ,ψ} on real robot with RGB→mask→depth preprocessing.
```

## 8. Significance, Limitations, and Implications

EgoPush demonstrates that egocentric, object-centric, relation-driven policy amalgamation is tractable for long-horizon, multi-object non-prehensile rearrangement without explicit global localization, with privileged-to-student distillation and constrained supervision identified as critical enablers of real-world transfer. The success of stage-local temporal shaping and relational latent structure emphasizes the value of temporal and semantic abstraction in reinforcement learning for manipulation. All claims and results above are reported in [2602.18071].

Source: https://www.emergentmind.com/topics/egopush-framework