PerceptionDLM: Parallel Region Diffusion Model

• The paper demonstrates that PerceptionDLM achieves parallel caption generation of image regions using non-autoregressive diffusion, significantly reducing inference time.
• It integrates structured attention masking, region prompt embeddings, and RoI-aligned features to accurately condition on multiple localized regions.
• Using the ParaDLC-Bench, the research shows up to 3.44× throughput improvements with competitive caption quality in multi-region perception tasks.

Searching arXiv for PerceptionDLM and closely related diffusion MLLM work to ground the article. PerceptionDLM is a multimodal diffusion LLM for efficient parallel region perception in images. In the formulation introduced in "PerceptionDLM: Parallel Region Perception with Multimodal Diffusion LLMs" (Sun et al., 17 Jun 2026), the model addresses a limitation of most existing multimodal LLMs: autoregressive decoding processes regions sequentially and therefore scales poorly for tasks requiring descriptions of multiple localized image regions. PerceptionDLM instead exploits the non-autoregressive denoising dynamics of discrete diffusion LLMs to generate multiple region captions simultaneously, while conditioning on an image and a set of region masks. The same paper also introduces ParaDLC-Bench, a benchmark designed to evaluate both caption quality and inference efficiency under multi-region perception (Sun et al., 17 Jun 2026).

1. Definition and problem setting

PerceptionDLM is defined around the task of jointly captioning multiple masked image regions in a single decoding process. The central problem is not generic image captioning, but localized visual description under a multi-region setting in which an image $X_v$ and region masks $\{R_i\}_{i=1}^N$ are provided together (Sun et al., 17 Jun 2026). In conventional autoregressive multimodal systems, each region typically requires an additional full decode, so latency grows with the number of queried regions. PerceptionDLM reformulates this setting as a single conditional diffusion-language generation problem over all region-caption tokens (Sun et al., 17 Jun 2026).

The model is built upon PerceptionDLM-Base, described as a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs (Sun et al., 17 Jun 2026). Its contribution is therefore architectural rather than merely benchmark-oriented: it combines visual encoding, region-conditioned prompting, and structured attention masking within a discrete diffusion LLM so that multiple regional caption streams can be denoised in parallel.

A plausible implication is that PerceptionDLM is best understood as a perception-optimized multimodal DLM rather than as a general-purpose multimodal reasoning model. The paper explicitly positions it against existing approaches for perception tasks that require captioning multiple regions, and its evaluation is correspondingly centered on region caption quality and decoding efficiency rather than chain-of-thought competence or broad instruction following (Sun et al., 17 Jun 2026).

2. Core architecture

PerceptionDLM extends a discrete-diffusion LLM to process an image together with multiple region masks in one shot (Sun et al., 17 Jun 2026). The architecture has three stated components: an image encoder, a diffusion language-model backbone, and region-mask inputs.

The image encoder uses a pretrained SigLIP-2 vision tower $\Phi_v$ to map the raw image $X_v$, or its $512\times512$ tiles, to dense visual features

$$Z_v=\Phi_v(X_v), \quad Z_v\in\mathbb{R}^{T_v\times d_v}.$$

A two-layer GELU MLP connector $\Phi_c$ then projects these features into the diffusion LLM embedding space,

$H_v=\Phi_c(Z_v)\in\mathbb{R}^{T_v\times d}.$

The language backbone is built on LLaDA-8B, which models text token sequences $y=(y_1,\dots,y_L)$ as a discrete denoising Markov chain $\{z_t\}_{t=0}^T$ and denoises $\{R_i\}_{i=1}^N$0 back to $\{R_i\}_{i=1}^N$1 during generation (Sun et al., 17 Jun 2026).

Region-mask conditioning is handled by extracting RoI-aligned features from $\{R_i\}_{i=1}^N$2 for each mask $\{R_i\}_{i=1}^N$3, projecting them into placeholder token slots, and adding a learnable region prompt embedding $\{R_i\}_{i=1}^N$4 to the corresponding RoI tokens (Sun et al., 17 Jun 2026). These prompt embeddings are broadcast-added so that the model can identify which slot corresponds to which region. The full sequence is then concatenated as

$\{R_i\}_{i=1}^N$5

and fed into the Transformer-based denoiser (Sun et al., 17 Jun 2026).

The resulting conditional distribution over all region-caption tokens is written as

$\{R_i\}_{i=1}^N$6

This formulation makes the multi-region setting intrinsic to the probabilistic model rather than an external loop over regions (Sun et al., 17 Jun 2026).

3. Diffusion-language formulation

PerceptionDLM adopts a standard discrete diffusion language-model formulation, specialized to multimodal conditional generation (Sun et al., 17 Jun 2026). The forward corruption process is defined as

$\{R_i\}_{i=1}^N$7

where $\{R_i\}_{i=1}^N$8 is uniform over masked token configurations and each transition independently replaces a portion of tokens with the $\{R_i\}_{i=1}^N$9 symbol according to a schedule $\Phi_v$0 (Sun et al., 17 Jun 2026).

The reverse process conditions on the image and all region masks:

$\Phi_v$1

Training minimizes a variational bound that reduces, for discrete tokens, to a weighted denoising objective. If $\Phi_v$2 indexes the masked positions at step $\Phi_v$3, the loss is

$\Phi_v$4

with an equivalent log-likelihood form

$\Phi_v$5

Here, the denoiser predicts the unmasked token distribution at each position (Sun et al., 17 Jun 2026).

This makes PerceptionDLM part of the broader class of diffusion-based LLMs in which all token positions can be updated in parallel during each denoising step. A plausible implication is that the model’s efficiency gain is fundamentally tied to this token-parallel reverse process rather than only to prompt engineering or vision-side amortization.

4. Parallel region perception mechanism

The defining mechanism of PerceptionDLM is simultaneous caption generation for multiple regions (Sun et al., 17 Jun 2026). The paper attributes this to three coupled design choices: efficient prompting, structured attention masking, and parallel decoding.

Efficient prompting introduces $\Phi_v$6 learnable region prompt vectors $\Phi_v$7, each added to the RoI tokens of its corresponding region (Sun et al., 17 Jun 2026). These prompts disambiguate region streams inside the joint denoising process. Ablation results indicate that without region prompting, grounding collapses, with average accuracy dropping to approximately $\Phi_v$8 (Sun et al., 17 Jun 2026).

Structured attention masking enforces region-wise independence while preserving shared global context. If $\Phi_v$9 denotes the total sequence length, the model builds a binary attention mask

$X_v$0

For tokens in region $X_v$1’s caption, attention is allowed only to global visual tokens, shared instruction tokens, RoI tokens of region $X_v$2, and caption tokens of region $X_v$3; all cross-region attention is masked (Sun et al., 17 Jun 2026). Self-attention becomes

$X_v$4

The paper describes this as a block-diagonal-plus-global pattern that enforces independence across region streams while sharing a single global visual context (Sun et al., 17 Jun 2026). Without structured attention, cross-region interference reduces average accuracy by about $X_v$5.

Parallel decoding follows directly from the diffusion backbone: all token positions are denoised at once, so all region sequences are generated simultaneously. At each diffusion step,

$X_v$6

for every position $X_v$7, subject to the sparse attention mask (Sun et al., 17 Jun 2026). Without RoI-aligned feature replay, accuracy drops by about $X_v$8, which indicates that spatially localized visual features remain necessary even under globally shared visual conditioning.

5. Evaluation and ParaDLC-Bench

To evaluate both quality and speed under multi-region decoding, the authors construct the Parallel Detailed Localized Captioning Benchmark, or ParaDLC-Bench (Sun et al., 17 Jun 2026). It is derived by scaling DLC-Bench to include multiple region masks per image and is intended to enable joint evaluation of caption quality and inference efficiency.

Dataset construction begins from Objects365 V2 and DaTaSeg images with instance masks (Sun et al., 17 Jun 2026). The benchmark selects challenging combinations of $X_v$9 to $512\times512$0 spatially adjacent or semantically similar masks and generates positive and negative attribute questions using GPT-5.2 plus human curation. The resulting benchmark contains 100 images and 2345 verified multiple-choice questions, including positive checks for correct attributes and negative checks for hallucinated or cross-region details (Sun et al., 17 Jun 2026).

The benchmark uses three caption-quality metrics and two efficiency metrics.

Category	Metric	Definition
Quality	Pos %	Fraction of correctly mentioned required attributes
Quality	Neg %	Fraction of avoided hallucinations
Quality	Average accuracy	$512\times512$1
Efficiency	TPF	Tokens per forward pass
Efficiency	Inference time	Total inference time in seconds on ParaDLC-Bench

The benchmark design is significant because it treats efficiency as a first-class evaluation target. A plausible implication is that ParaDLC-Bench is meant to capture a regime in which localized captioning is not only a semantic prediction problem but also a systems problem involving throughput under region multiplicity.

6. Empirical results, trade-offs, and relation to adjacent work

On 8B-parameter models evaluated on ParaDLC-Bench, the autoregressive region model GAR takes 479 seconds, has $512\times512$2, and achieves average accuracy of approximately $512\times512$3, whereas PerceptionDLM takes 276 seconds, has $512\times512$4, and achieves average accuracy of $512\times512$5 (Sun et al., 17 Jun 2026). The paper therefore characterizes the model as maintaining competitive performance in region captioning while providing substantial speed improvements. It reports that, despite a small drop in accuracy versus highly specialized autoregressive region models, PerceptionDLM runs about $512\times512$6 faster overall and yields up to $512\times512$7 throughput improvements as region count grows (Sun et al., 17 Jun 2026).

On the single-region DLC-Bench, PerceptionDLM still outperforms other diffusion VLMs, with average accuracy of $512\times512$8 versus approximately $512\times512$9 to $$Z_v=\Phi_v(X_v), \quad Z_v\in\mathbb{R}^{T_v\times d_v}.$$0, although the TPF advantage is smaller (Sun et al., 17 Jun 2026). This suggests that the model’s main advantage is not restricted to the multi-region setting, but becomes most pronounced there.

The paper also identifies specific limitations. Multi-step diffusion sampling, for example 32 steps, caps raw inference speed; extremely dense or highly similar masks still incur some entanglement; and diffusion’s arbitrary updating order limits complex reasoning (Sun et al., 17 Jun 2026). Proposed future directions include pseudo-trajectory distillation to reduce the number of steps, adding region-contrastive losses or finer-granularity masks, and integrating RL fine-tuning to narrow the reasoning gap to autoregressive LLMs (Sun et al., 17 Jun 2026).

These limitations align with a broader tension documented in multimodal systems between perception efficiency and higher-order reasoning. "Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off" reports an "Efficiency-Stability Paradox" in which reasoning injection raises abstract-task performance but degrades low-level sensory processing, including OCR and ASR robustness (Zhang et al., 27 Feb 2026). Although that work is not about PerceptionDLM in the narrow architectural sense, it provides adjacent evidence that perception-oriented multimodal design and reasoning-oriented multimodal design can impose competing optimization pressures.

A different but complementary line of work is "Boosting Latent Diffusion with Perceptual Objectives" (Berrada et al., 2024), which addresses a decoder-diffusion disconnect in latent image generation by adding a latent perceptual loss defined on decoder features. That work concerns latent diffusion image synthesis rather than diffusion language modeling, but it is relevant insofar as it underscores that diffusion architectures in multimodal settings often require explicit mechanisms to preserve perceptually salient structure (Berrada et al., 2024).

7. Position within multimodal perception research

PerceptionDLM occupies a distinct position within multimodal modeling because it uses diffusion not primarily for image generation, but for non-autoregressive conditional language generation grounded in visual regions (Sun et al., 17 Jun 2026). Its contribution is therefore orthogonal to work on runtime monitoring of perception systems, such as diagnostic-graph-based fault detection in autonomous stacks (Antonante et al., 2022), and also distinct from multimodal commonsense controllers that intervene under perception deficits in driving scenarios (Hu et al., 10 Mar 2025). Those works treat perception outputs as objects to be monitored, corrected, or compensated for; PerceptionDLM instead redesigns the decoding substrate for a localized perception task.

The model is also distinct from the audio metric DPLM, "A Deep Perceptual Spatial-Audio Localization Metric" (Manocha et al., 2021), which uses hidden activations of a DOA-estimation network as a perceptual embedding for binaural localization assessment. The shared theme is perceptual alignment through learned internal representations, but the operational objective differs fundamentally: DPLM is a differentiable evaluation metric for localization similarity, whereas PerceptionDLM is a generative perception model for parallel region captioning.

Within the diffusion-LLM literature, the salient claim of PerceptionDLM is that discrete diffusion models can realize parallel region caption and perception by exploiting their intrinsic non-autoregressive denoising dynamics (Sun et al., 17 Jun 2026). This suggests a broader design principle: when multimodal tasks require simultaneous predictions over multiple localized subproblems, diffusion LLMs may offer an architectural advantage over autoregressive decoders, provided that conditioning structure and attention constraints are designed to prevent cross-instance interference.