PixNerd-XXL/16 Pixel Diffusion Model

• The paper introduces a novel, single-stage diffusion framework that replaces VAEs with dynamically-parameterized neural fields for patch-wise image synthesis.
• It employs DCT-based coordinate encodings within each 16x16 patch, boosting per-patch detail reconstruction and computational efficiency.
• The model achieves competitive benchmarks (FID: 2.15/2.84, GenEval: 0.73, DPG: 80.9) on ImageNet and text-to-image tasks while supporting arbitrary resolution outputs.

PixNerd-XXL/16 is a large-scale pixel-space diffusion model introduced in the PixNerd framework for efficient, single-stage, end-to-end image synthesis without recourse to variational autoencoders or complex cascade pipelines. By leveraging dynamically-parameterized neural fields over large image patches, PixNerd-XXL/16 provides a high-fidelity, computationally efficient approach for both class-conditioned and text-to-image generation, demonstrated by strong benchmarking on ImageNet and leading text-to-image evaluation suites (Wang et al., 31 Jul 2025).

1. Framework Overview

PixNerd-XXL/16 operates directly in raw pixel space—eschewing the two-stage latent paradigm common to prior diffusion models—by using a patch size of $16\times16$, resulting in a manageable token count despite acting on full-resolution images. The core innovation is the "patch-wise neural field": for each patch, rather than applying a fixed linear projection (as in traditional diffusion transformers), a set of neural field parameters is dynamically generated and used to decode pixelwise outputs via an implicit MLP-based function. Training and sampling are conducted in a single stage across both class-to-image and text-to-image tasks.

Key features:

• Patch-wise neural field replaces linear heads for fine-grained detail reconstruction
• Coordinate encoding within each patch leveraging DCT-based or trigonometric signals
• No VAE; fully pixel-space, single-scale architecture
• Low memory footprint and low inference latency relative to comparable pixel diffusion models
• Applicability to arbitrary resolution image generation via coordinate-based neural field decoding

2. Technical Innovations

PixNerd-XXL/16 adapts the transformer-based diffusion machinery for the computational demands of pixel space:

• Neural Field Decoding: For each patch $n$, the transformer computes a feature $X^n$. The model predicts the weights of a neural field MLP for the patch—$W_1^n$, $W_2^n$—by linear transformations:

$W_1^n, W_2^n = \mathrm{Linear}(\mathrm{SiLU}(X^n))$

These weights are normalized (row-wise) and then used to define an MLP which, conditioned on pixel coordinate encodings and the current noisy pixel value, predicts the denoising 'velocity' for every pixel in the patch.

• Coordinate Encoding: Within each patch, each pixel's 2D location $(i, j)$ is encoded as $PE(i,j)$ using DCT-based positional encodings, which are empirically preferred over sinusoidal alternatives. For each pixel,

$$V^n(i, j) = \mathrm{MLP}(\mathrm{Concat}(PE(i,j), x^n(i, j)) \mid \{\tilde{W}_1^n, \tilde{W}_2^n\})$$

$v^n(i,j) = \mathrm{Linear}(V^n(i, j))$

where $v^n(i,j)$ is the predicted denoising velocity required for the diffusion model.

• Diffusion Modeling: The forward and reverse diffusion processes are standard:

$$x_t = \alpha_t x_{\mathrm{real}} + \sigma_t \epsilon$$

The neural field directly predicts velocities for sampling or training using solvers such as Adams-2nd order (Adams linear multistep method) for improved quality and computational performance.

• Architecture Enhancements: Transformer blocks employ SwiGLU activation, RMSNorm, RoPE positional embeddings, and careful normalization on neural field parameter predictions. This supports gradient stability and improves sample quality over long-range dependencies.

3. Performance Metrics

PixNerd-XXL/16 demonstrates competitive quantitative results in both class-to-image and text-to-image settings:

• ImageNet FID: Achieves 2.15 FID on ImageNet $256\times256$ and 2.84 FID on $512\times512$ generation using a single-stage, non-cascaded framework.
• Text-to-Image Benchmarks:
  • GenEval overall score: 0.73
  • DPG overall score: 80.9

In the GenEval suite, the 0.73 overall score represents robust semantic and structural alignment, with high fidelity in object similarity, color consistency, and spatial arrangement. DPG's 80.9 average score further substantiates the model's ability to synthesize globally consistent and finely-detailed images directly from textual prompts.

4. Comparison with Prior Approaches

The PixNerd-XXL/16 model offers substantial differences relative to both latent diffusion and preceding pixel-space diffusion techniques:

• Elimination of VAE: Avoids the two-stage VAE pretraining and decoding, thereby removing cumulative errors and decoding artifacts typical of latent approaches.
• Single-Stage Training/Inference: Contrasts with cascade pipelines often required to stabilize or enhance pixel diffusion outputs at high resolution.
• Patch-Wise Neural Field vs. Linear Decoding: Provides a more expressive patch decoder than a single global linear head, improving per-patch granularity, especially for large patch sizes.
• Resource Efficiency: Outperforms models like ADM-G and PixelFlow in terms of runtime and memory usage, achieving up to eightfold improvements in computational metrics.
• Fidelity and Flexibility: Maintains strong FID and benchmark scores without increasing pipeline complexity, and is capable of generating images at arbitrary resolutions thanks to the coordinate-based neural field.

The following table organizes comparison points already present in the data:

Model	Decoding Head	Cascade?	FID (256x256)	FID (512x512)	GenEval	DPG
ADM-G/PixelFlow	Linear	Yes	Not specified	Not specified	Lower	Lower
Latent Diffusion	Linear (latent)	No	Not specified	Not specified	Lower	Lower
PixNerd-XXL/16	Patch-wise Neural Field	No	2.15	2.84	0.73	80.9

5. Methodological Details and Mathematical Formulation

The technical workings of PixNerd-XXL/16 are characterized by a combination of explicit mathematical models and architectural techniques:

• Forward Diffusion Process:

$$x_t = \alpha_t \cdot x_{\mathrm{real}} + \sigma_t \cdot \epsilon$$

• Dynamically-Parameterized Neural Field Per Patch:

$W_1^n, W_2^n = \mathrm{Linear}(\mathrm{SiLU}(X^n))$

• Velocity Prediction for Each Pixel:

$$V^n(i, j) = \mathrm{MLP}(\mathrm{Concat}(PE(i, j), x^n(i, j)); \{\tilde{W}_1^n, \tilde{W}_2^n\})$$

$v^n(i, j) = \mathrm{Linear}(V^n(i, j))$

• DCT Coordinate Encoding:

$$\text{DCT-PE}(i, j) = \{ \cos(k_1 \cdot i) \cos(k_2 \cdot j) \mid k_1, k_2 \in \{1, \ldots, K\} \}$$

This structure enables the model to efficiently interpolate and represent arbitrary resolutions, a property inherited from neural field representations.

6. Applications and Extensions

PixNerd-XXL/16 is targeted at high-fidelity generative modeling in several modalities:

• Text-to-Image Generation: Demonstrates high semantic and structural alignment on GenEval and DPG, parsing varied textual prompts into visually-coherent outputs.
• Class-Conditional Synthesis: Achieves state-of-the-art FID at standard and high ($512\times512$) resolutions.
• Arbitrary Resolution Output: Through continuous coordinate-based neural field decoding, supports synthesis at image resolutions not encountered during training.
• Potential for Video and Multi-Modal Generation: The architecture could be naturally extended to spatiotemporal domains or multi-modal inputs by increasing token dimensions or conditioning mechanisms (Editor's term).

A plausible implication is the model's suitability for tasks in vision-language research requiring high information preservation, efficient scaling, and rapid sampling.

7. Significance and Research Impact

PixNerd-XXL/16 advances the field by demonstrating that high-quality image synthesis, class-conditional or text-conditional, can be achieved in pixel space without trade-offs in computational efficiency or output fidelity, typically associated with large-token, pixel-space models. Its employment of patch-wise neural field decoders provides a promising architectural alternative for future scalable, resolution-agnostic generative frameworks. The empirical results on benchmark datasets confirm the feasibility of direct pixel-space diffusion as a rival to or replacement for latent diffusion in numerous tasks, and the model's explicit mathematical descriptions further facilitate adaptation and expansion by subsequent research (Wang et al., 31 Jul 2025).