Self-Evolving Diffusion Policy (SeedPolicy)

• The paper's main contribution is integrating SEGA into Diffusion Policy to overcome weak temporal modeling in long-horizon tasks.
• SeedPolicy compresses long observation histories into a fixed-size latent state, enabling efficient, history-aware action generation using RGB images and joint poses.
• Empirical results reveal up to 169% relative improvement in challenging settings, validating the method’s effectiveness in enhancing visuomotor control.

Searching arXiv for SeedPolicy and closely related diffusion-policy papers. Searching arXiv for “SeedPolicy horizon scaling self-evolving diffusion policy”. SeedPolicy is a diffusion-policy variant for long-horizon robot manipulation that augments Diffusion Policy with a recurrent temporal module, SEGA (Self-Evolving Gated Attention), to compress long observation histories into a fixed-size latent state and to enrich current observations before diffusion-based action generation. In the formulation reported by the original work, the policy maps observations—primarily RGB images and 14-DoF joint poses—to future action sequences, and its central claim is that the main bottleneck in vanilla Diffusion Policy is not diffusion itself but inadequate temporal modeling as observation horizons grow (Gui et al., 5 Mar 2026).

1. Definition, scope, and nomenclature

SeedPolicy is explicitly defined as “Self-Evolving Diffusion Policy”, and it is built by integrating SEGA (Self-Evolving Gated Attention) into Diffusion Policy. The core architectural change is the insertion of a temporal module between observation encoding and diffusion-based action generation, so that the policy conditions on a time-evolving latent state rather than only on a short stack of observations (Gui et al., 5 Mar 2026).

This name should be distinguished from other diffusion-policy methods that use superficially similar terminology. In particular, ET-SEED expands SEED as “Efficient Trajectory-Level SE(3) Equivariant Diffusion Policy”, not “Self-Evolving Diffusion Policy,” and that work does not use the phrase “Self-Evolving” (Tie et al., 2024). A separate line of work, Self-Guided Action Diffusion, is likewise not SeedPolicy; it is an inference-time self-guidance mechanism for action-chunk diffusion policies rather than a temporal module for horizon scaling (Malhotra et al., 17 Aug 2025).

SeedPolicy is situated within the broader diffusion-policy literature initiated by action-diffusion visuomotor behavioral cloning, in which a policy represents a conditional distribution over future action sequences and generates actions by iterative denoising rather than one-shot regression (Chi et al., 2023). What differentiates SeedPolicy within that family is its emphasis on horizon scaling: the ability to benefit from longer observation histories instead of degrading as more temporal context is provided (Gui et al., 5 Mar 2026).

2. Problem setting and diagnosed bottleneck

The method is developed for robot imitation learning in visuomotor manipulation settings where success depends on historical context rather than the current observation alone. The reported observation modalities are primarily RGB images and 14-DoF joint poses, and the policy predicts future action sequences of 14-DoF actions (Gui et al., 5 Mar 2026).

The motivating failure mode is specific. In the baseline Diffusion Policy implementation described by the paper, past observations are handled largely as a stack of image frames. The authors characterize this as a weak temporal model: increasing the observation horizon increases the amount of input history, but does not inherently provide selective temporal reasoning. As the frame stack grows, the policy must process more redundant or irrelevant information while still recovering a small number of historically important cues (Gui et al., 5 Mar 2026).

The paper identifies several long-horizon manipulation difficulties that follow from this bottleneck. These include state aliasing, where visually similar observations correspond to different task phases; execution stagnation, where the robot stalls or loops because it cannot infer progress from history; and spatial ambiguity, where short-memory monocular RGB policies struggle to infer depth or geometry from motion over time (Gui et al., 5 Mar 2026). This diagnosis is consistent with the earlier diffusion-policy observation that modeling future action chunks is beneficial for multimodal visuomotor control, but does not by itself solve long-range temporal reasoning (Chi et al., 2023).

In this setting, horizon scaling means that performance should improve, or at least not collapse, as the observation history grows. SeedPolicy is explicitly proposed to make long-horizon context useful by compressing it into a bounded latent state, rather than attending over an ever-growing raw history (Gui et al., 5 Mar 2026).

3. SEGA: Self-Evolving Gated Attention

SEGA is the central mechanism of SeedPolicy. It is described as a dual-stream attention-based recurrent mechanism operating on a current observation feature sequence $O_t \in \mathbb{R}^{N_o \times D}$ and a previous latent state $S_{t-1} \in \mathbb{R}^{N_s \times D}$, where $N_o$ is the number of observation tokens, $N_s$ the number of latent-state tokens, and $D$ the feature dimension (Gui et al., 5 Mar 2026).

The module has two parallel streams. The State Update stream updates the historical latent state using the current observation, while the State Retrieval stream uses the previous latent state to enrich the current observation before action generation. This separation is architecturally important: one path evolves memory, and the other queries memory for immediate control (Gui et al., 5 Mar 2026).

The update stream first applies self-attention to both state and observation: $S'_{t-1} = S_{t-1} + \mathrm{MSA}(S_{t-1}),$

$O'_t = O_t + \mathrm{MSA}(O_t).$

It then performs cross-attention from state to observation: $$\mathrm{Inter}\cdot S_t,\; \mathcal{A} = \mathrm{CA}(S'_{t-1}, O'_t, O'_t),$$ where $\mathcal{A}$ denotes the raw cross-attention maps collected across layers and heads (Gui et al., 5 Mar 2026).

The distinctive component is the Self-Evolving Gate (SEG). Instead of using a generic learned gate, SEG derives relevance directly from the cross-attention maps: $$R = \frac{1}{L \cdot H \cdot N_o} \sum_{l=1}^{L}\sum_{h=1}^{H}\sum_{j=1}^{N_o} A^{(l,h)}_{:,j},$$

$S_{t-1} \in \mathbb{R}^{N_s \times D}$0

with $S_{t-1} \in \mathbb{R}^{N_s \times D}$1. The final memory update is

$S_{t-1} \in \mathbb{R}^{N_s \times D}$2

This design means the state is updated only when current observations are relevant to what the latent state is querying; otherwise, prior memory is preserved (Gui et al., 5 Mar 2026).

In parallel, the retrieval stream enriches the observation via

$S_{t-1} \in \mathbb{R}^{N_s \times D}$3

so the current observation acts as query and the previous state acts as key/value (Gui et al., 5 Mar 2026).

A compact summary of the reported per-step pipeline is given below.

Component	Definition	Function
Observation encoding	$S_{t-1} \in \mathbb{R}^{N_s \times D}$4	Encodes RGB image and joint pose
State update	$S_{t-1} \in \mathbb{R}^{N_s \times D}$5	Evolves fixed-size latent memory
State retrieval	$S_{t-1} \in \mathbb{R}^{N_s \times D}$6	Enriches observation with history
Action generation	$S_{t-1} \in \mathbb{R}^{N_s \times D}$7	Predicts future action chunk

The paper’s interpretation is that this recurrent latent state acts as a compact memory of task progress, geometry cues, and prior decisions. Because $S_{t-1} \in \mathbb{R}^{N_s \times D}$8 has fixed size, the entire history is recursively compressed into bounded memory rather than stored as an ever-growing token sequence (Gui et al., 5 Mar 2026).

4. Integration with Diffusion Policy and horizon scaling mechanism

SeedPolicy preserves the diffusion-policy action expert, but changes the conditioning pathway. Instead of conditioning only on a short fixed observation stack, the diffusion model conditions on $S_{t-1} \in \mathbb{R}^{N_s \times D}$9, the observation features enhanced by historical state retrieval (Gui et al., 5 Mar 2026).

The diffusion backend reported in the appendix uses a DDPM scheduler, 100 training timesteps, 100 inference timesteps, Squared Cosine Cap v2 beta schedule, beta range $N_o$0, and epsilon prediction (Gui et al., 5 Mar 2026). The paper does not print a full closed-form diffusion objective in the main text. It states that the implied objective is standard DDPM-style noise prediction, but that formula should be treated as implied rather than explicitly stated (Gui et al., 5 Mar 2026).

The implementation section reports the following concrete settings: observation history $N_o$1, image size $N_o$2 RGB, proprioception as 14-DoF joint pose, latent state sequence length $N_o$3, and feature dimension $N_o$4 (Gui et al., 5 Mar 2026). A notable point is that even though the explicit raw observation window is only three steps, the recurrent latent state evolves across time, so the effective temporal context extends beyond the raw input stack (Gui et al., 5 Mar 2026).

The paper compares this design conceptually with two baselines. Vanilla Diffusion Policy conditions on a fixed observation stack and therefore lacks explicit selective recurrence. A straightforward temporal-attention alternative scales quadratically with the history length $N_o$5, with self-attention cost on the order of $N_o$6. By contrast, the recurrent state mechanism in SeedPolicy avoids attention over the entire history. The paper does not provide an explicit complexity formula, but it explicitly claims that the recurrent state avoids the quadratic cost of long-window attention. A plausible implication is that per-step cost is bounded by interactions between $N_o$7 and $N_o$8, rather than by the elapsed episode length (Gui et al., 5 Mar 2026).

The same section also reports architectural compatibility with both Transformer and CNN backbones. This is central to the paper’s argument that SEGA is not tied to one visual encoder family. Reported parameter counts on the 50-task RoboTwin benchmark are: DP-Transformer 20.61M, DP-CNN 96.80M, SeedPolicy-Transformer 33.36M, SeedPolicy-CNN 147.26M, ACT 80M, and RDT 1.2B (Gui et al., 5 Mar 2026). This suggests that the method adds moderate overhead relative to Diffusion Policy while remaining substantially smaller than large VLA models.

5. Empirical performance and ablations

The main simulation benchmark is RoboTwin 2.0, evaluated on 50 manipulation tasks with the ALOHA-AgileX robot in simulation. The protocol uses 50 expert demonstrations per task, 600 epochs of training, and 100 rollouts per task, averaged over 3 independent trials, with evaluation in both Easy / demo_clean and Hard / demo_randomized settings (Gui et al., 5 Mar 2026).

The headline result is that, averaged across CNN and Transformer backbones, SeedPolicy achieves 36.8% relative improvement over Diffusion Policy in clean settings and 169% relative improvement in randomized challenging settings (Gui et al., 5 Mar 2026). The benchmark averages reported by the paper are:

Method	Easy	Hard
RDT	34.50%	13.72%
ACT	29.74%	1.74%
DP-Transformer	33.10%	1.44%
DP-CNN	28.04%	0.64%
SeedPolicy-Transformer	40.08%	4.28%
SeedPolicy-CNN	42.76%	1.54%

These averages unpack into the reported backbone-wise gains: Transformer Easy +7.0 absolute, +21.10% relative; CNN Easy +14.72 absolute, +52.50% relative; Transformer Hard +3.0 absolute, +197.22% relative; CNN Hard +0.9 absolute, +140.63% relative (Gui et al., 5 Mar 2026).

The task-level summary further states that SeedPolicy is better or equal on 45/50 tasks with the Transformer backbone and on 44/50 tasks with the CNN backbone (Gui et al., 5 Mar 2026). The paper also groups tasks by average episode length and reports that the performance gap widens with task length. For the Transformer backbone, improvements are +2.9% on short tasks, +6.4% on medium tasks, and +16.0% on long tasks. For the CNN backbone, the corresponding gains are +13.6%, +12.9%, and +21.9% (Gui et al., 5 Mar 2026). This is presented as direct evidence for the horizon-scaling claim.

Ablation studies isolate the role of temporal modeling. On representative tasks, the progression DP $N_o$9 DP + Temporal Attention $N_s$0 DP + State $N_s$1 DP + Gating (CA) $N_s$2 DP + Gating (FFN) shows that explicit temporal attention helps over stacked-frame DP, a recurrent fixed-size state helps more, and cross-attention-based gating gives the largest gains, especially on longer tasks. For example, on Stack Bowls Two (long), the sequence reported is 33, 48, 65, 73, 70 (Gui et al., 5 Mar 2026).

The appendix also reports architecture-level ablations. For latent state length, $N_s$3 is described as insufficient capacity, $N_s$4 as best overall, and $N_s$5 as a plateau or slight drop. For SEGA depth, depth 2 < 4 < 6 in performance, while depth 8 degrades, with the best tested depth reported as 6 (Gui et al., 5 Mar 2026).

Real-world evaluation is conducted on the Dexmal DOS-W1 robot with a fixed front-view Intel RealSense D435 RGB camera, using three tasks: Looping_Place-Retrieval, Sequential_Picking, and Bottle_Handover. Training uses 50 demonstrations per task, 600 epochs, and 20 evaluation trials per task. The text explicitly reports one large jump: Bottle Handover improves from 15% to 56% (Gui et al., 5 Mar 2026). The figure indicates gains on all three tasks, although the other exact percentages are not numerically printed in the provided text.

6. Adjacent methods, limitations, and interpretive significance

SeedPolicy belongs to a broader class of diffusion-based robot policies, but its contribution is narrower and more specific than several neighboring methods. Relative to the original Diffusion Policy, which represents a policy as a conditional denoising diffusion process over action sequences, SeedPolicy addresses a particular weakness: the inability of plain stacked-frame conditioning to use long observation histories effectively (Chi et al., 2023). Relative to DPPO, which provides an on-policy RL fine-tuning framework for pretrained diffusion policies, SeedPolicy remains an imitation-learning method centered on temporal compression rather than online policy-gradient improvement (Ren et al., 2024). Relative to Self-Guided Action Diffusion, which performs inference-time trajectory-coherence guidance using a prior action chunk, SeedPolicy changes the temporal representation inside the policy rather than the decoding procedure at test time (Malhotra et al., 17 Aug 2025). Relative to ET-SEED, it solves a different problem entirely: temporal modeling for long-horizon manipulation rather than trajectory-level $N_s$6-equivariant generation (Tie et al., 2024).

The paper states a main limitation clearly: although SeedPolicy improves substantially in randomized hard settings relative to vanilla Diffusion Policy, it still lags large foundation models such as RDT in broader generalization. This is most evident in the Hard setting, where RDT reaches 13.72%, compared with 4.28% for SeedPolicy-Transformer and 1.54% for SeedPolicy-CNN (Gui et al., 5 Mar 2026).

Other stated or visible limitations are methodological rather than purely empirical. The paper does not provide a full formal derivation of the diffusion objective inside SeedPolicy, relying instead on standard Diffusion Policy/DDPM practice; it does not report detailed runtime or memory profiling beyond parameter counts and qualitative “moderate overhead” claims; and it does not specify recurrent training mechanics such as truncated BPTT, state reset rules, or hidden-state detachment procedures (Gui et al., 5 Mar 2026).

A broader interpretation is that SeedPolicy isolates temporal modeling as a primary failure mode of long-horizon imitation learning with diffusion policies. This suggests that some previously observed degradation with longer contexts is not a limitation of diffusion action generation per se, but of the conditioning pathway used to summarize history. A plausible implication is that future work may combine SeedPolicy-style temporal compression with diffusion-policy post-training or inference-time guidance, but such an overview is not part of the reported method (Ren et al., 2024).

In the current literature, SeedPolicy is therefore best understood as a recurrently enhanced diffusion policy that addresses the temporal bottleneck of vanilla Diffusion Policy through fixed-size latent memory, cross-attention-based gating, and history-aware observation retrieval. Its empirical significance lies less in altering the diffusion action expert itself than in demonstrating that long-horizon manipulation performance can improve materially when temporal context is compressed selectively rather than stacked naively (Gui et al., 5 Mar 2026).