DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion (2510.20766v1)

Abstract: Diffusion Transformer models can generate images with remarkable fidelity and detail, yet training them at ultra-high resolutions remains extremely costly due to the self-attention mechanism's quadratic scaling with the number of image tokens. In this paper, we introduce Dynamic Position Extrapolation (DyPE), a novel, training-free method that enables pre-trained diffusion transformers to synthesize images at resolutions far beyond their training data, with no additional sampling cost. DyPE takes advantage of the spectral progression inherent to the diffusion process, where low-frequency structures converge early, while high-frequencies take more steps to resolve. Specifically, DyPE dynamically adjusts the model's positional encoding at each diffusion step, matching their frequency spectrum with the current stage of the generative process. This approach allows us to generate images at resolutions that exceed the training resolution dramatically, e.g., 16 million pixels using FLUX. On multiple benchmarks, DyPE consistently improves performance and achieves state-of-the-art fidelity in ultra-high-resolution image generation, with gains becoming even more pronounced at higher resolutions. Project page is available at https://noamissachar.github.io/DyPE/.

• The paper introduces DyPE, a training-free method that dynamically adjusts positional encodings to align with the spectral progression of the diffusion process.
• It employs a novel scaling scheduler to balance early global structure stabilization with later recovery of high-frequency details.
• Experimental results on benchmarks like Aesthetic-4K and ImageNet show DyPE outperforms static baselines, achieving superior fidelity and texture preservation.

DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion

Introduction

The paper introduces Dynamic Position Extrapolation (DyPE), a training-free method for enabling pre-trained diffusion transformers to synthesize ultra-high-resolution images, far exceeding their original training context, without retraining or additional inference overhead. DyPE leverages the spectral progression inherent to the diffusion process, dynamically adjusting positional encodings (PE) at each denoising step to match the evolving frequency spectrum of the generated sample. This approach is motivated by the observation that low-frequency image structures converge early in the reverse diffusion process, while high-frequency details are resolved later. By synchronizing the PE with this progression, DyPE achieves state-of-the-art fidelity in ultra-high-resolution image generation, with improvements that become more pronounced as the target resolution increases.

Spectral Dynamics of Diffusion and Position Extrapolation

Diffusion models generate images by progressively denoising a sample from pure Gaussian noise to a clean image, parameterized by a time variable $t$. In Fourier space, the process can be described as:

$\hat{x}_t = (1-t) \hat{x} + t \hat{\epsilon}$

where $\hat{x}$ is the Fourier transform of the target image and $\hat{\epsilon}$ is noise. The power spectrum density (PSD) of natural images follows a power-law decay, $1/f^\omega$, with $\omega \approx 2$. Empirical analysis reveals that low-frequency components of the image converge rapidly during denoising, while high-frequency components evolve more gradually throughout the process.

Figure 1: Frequency Behavior Across Scaling Strategies. DyPE dynamically interpolates between RoPE and NTK-aware by blending their effective periods as a function of the diffusion timestep $t$.

This spectral progression suggests that static position extrapolation schemes—such as Position Interpolation (PI), NTK-aware, and YaRN—are suboptimal for ultra-high-resolution synthesis, as they do not account for the time-dependent evolution of frequency bands. Existing methods either compress high frequencies (leading to blurriness) or overemphasize low frequencies (resulting in periodic artifacts), especially when extrapolating far beyond the training context.

DyPE: Dynamic Position Extrapolation

DyPE introduces explicit time dependence into position extrapolation strategies. The key innovation is a scheduler $\kappa(t) = \lambda_s \cdot t^{\lambda_t}$, where $\lambda_s$ and $\lambda_t$ are hyperparameters controlling the magnitude and progression of scaling. Early in the denoising process ($t \approx 1$), DyPE applies maximal scaling to stabilize global structure; as $t$ decreases, scaling attenuates, allowing the PE to represent finer high-frequency details.

DyPE generalizes three main extrapolation schemes:

• Dy-PI: Uniform position scaling modulated by $\kappa(t)$.
• Dy-NTK: Frequency-dependent scaling, with the NTK-aware exponent multiplied by $\kappa(t)$.
• Dy-YaRN: Dynamic ramp thresholds in YaRN, shifting the frequency bands assigned to each scaling regime as denoising progresses.

  Figure 2: Wavelengths of the RoPE embeddings under different strategies. DyPE interpolates NTK-aware with Extrapolation, dynamically expanding high-frequency coverage as denoising advances.

This dynamic allocation allows the denoiser to operate under conditions closer to its training regime, particularly in later steps, while maximizing the representation of high-frequency modes necessary for ultra-high-resolution synthesis.

Experimental Results

DyPE is evaluated on top of FLUX and FiTv2 diffusion transformers, with extensive quantitative and qualitative analysis on DrawBench, Aesthetic-4K, and ImageNet benchmarks. Metrics include CLIPScore, ImageReward, Aesthetic-Score, FID, sFID, Inception Score, Precision, and Recall.

• Ultra-High-Resolution Text-to-Image: DyPE consistently outperforms static baselines (NTK-aware, YaRN, PI) across all metrics, with the advantage increasing at higher resolutions ($4096^2$ and above). Human evaluation shows DyPE is preferred in 87–90% of pairwise comparisons for text alignment, structure, and detail.

  Figure 3: Zoom-in comparison at $4096^2$ of Dy-YaRN vs. YaRN. DyPE preserves fine details and texture fidelity at extreme resolutions.

  

  Figure 4: Qualitative results at $4096^2$ resolution using two representative prompts from Aesthetic-4K. DyPE variants yield sharper, more coherent outputs than static baselines.

• Resolution Scaling Analysis: DyPE maintains stable performance up to $6144^2$ resolution, whereas FLUX and YaRN degrade sharply beyond $3072^2$ and $4096^2$, respectively.
• Class-Conditional Generation (ImageNet): DyPE variants (Dy-PI, Dy-NTK, Dy-YaRN) improve FID, sFID, IS, Precision, and Recall over their static counterparts, with Dy-YaRN achieving the best overall results.
• Panoramic and Extreme Aspect Ratios: DyPE demonstrates robust spatial consistency and aspect ratio preservation in panoramic generation ($4096 \times 1365$), outperforming YaRN and FLUX.

  Figure 5: Qualitative comparison of panoramic generation at $4096 \times 1365$ resolution. DyPE maintains coherent spatial structure across extreme aspect ratios.

• Comparison with Lumina-Next: DyPE achieves superior CLIPScore, ImageReward, Aesthetic-Score, and FID compared to Time-Aware Scaled RoPE (Lumina-Next), which suppresses high frequencies and yields blurrier outputs.

  Figure 6: Qualitative comparison between DyPE and Time-Aware Scaled RoPE (Lumina-Next) on the Aesthetic-4K benchmark.

Implementation and Ablation

DyPE is implemented as a test-time modification to the positional encoding layer, requiring no retraining or additional sampling steps. The scheduler parameters ($\lambda_s$, $\lambda_t$) are tuned via ablation studies, with exponential progression ($\lambda_s=2$, $\lambda_t=2$) yielding optimal trade-offs between structural fidelity and detail. All experiments are conducted on a single L40S GPU, demonstrating the method's computational efficiency.

Implications and Future Directions

DyPE establishes a principled framework for dynamic position extrapolation in diffusion transformers, enabling ultra-high-resolution synthesis without retraining. The method's generality allows for straightforward integration with existing architectures and extrapolation schemes. Theoretical implications include a deeper understanding of the interplay between spectral progression and positional encoding in generative models.

Practically, DyPE unlocks new capabilities for high-fidelity image generation in domains requiring extreme resolutions (e.g., medical imaging, satellite imagery, digital art). The approach is extensible to video generation, where time-aware positional extrapolation can enhance both spatial and temporal coherence.

Future work may explore joint training with time-dependent PE, further scaling to $8$K and beyond, and adaptation to multimodal and spatiotemporal generative tasks.

Conclusion

DyPE provides a robust, training-free solution for ultra-high-resolution image synthesis in diffusion transformers, grounded in a Fourier-space analysis of the denoising process. By dynamically modulating positional encoding to match the evolving frequency spectrum, DyPE achieves superior generalization and fidelity compared to static extrapolation methods. The approach is computationally efficient, broadly applicable, and sets a new standard for high-resolution generative modeling.