PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

Abstract: Multimodal LLMs (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion LLM optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion LLMs for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion LLMs. Code, models, and datasets are released.

• The paper introduces a novel parallel captioning mechanism using diffusion language models that drastically reduces the latency of processing multiple regions.
• It integrates region prompting and structured attention masking with a SigLIP-2 encoder and LLaDA-8B backbone, significantly boosting accuracy on region-level tasks.
• Benchmarked on ParaDLC-Bench, PerceptionDLM achieves up to a 3.44× throughput speedup and nearly doubles captioning accuracy compared to AR-based models.

PerceptionDLM: Parallel Region Perception with Multimodal Diffusion LLMs

Introduction and Motivation

PerceptionDLM addresses the scalability bottleneck in dense, fine-grained region-level vision-language perception by leveraging diffusion LLMs (DLMs) for parallel multi-region caption generation. Traditional Multimodal LLMs (MLLMs) predominantly rely on autoregressive (AR) decoding, which introduces linear latency growth when sequentially processing multiple image regions—restricting their practical applicability for real-time, high-throughput visual understanding scenarios. PerceptionDLM exploits the inherent parallelism of DLMs, introducing novel architectural and training advances to facilitate region-conditioned, concurrent textual generation at both the sequence and token level.

Figure 1: Overview and efficiency analysis of PerceptionDLM, comparing its parallel captioning and throughput scaling to AR models.

The core system supports input images with multiple binary region masks. It generates one caption per mask in a single denoising step, yielding sublinear inference cost growth as the number of regions increases. This design sets it apart from AR-based models (e.g., DAM, GAR), which exhibit linearly increasing latency due to sequential decoding.

Architectural Innovations

The base model, PerceptionDLM-Base, integrates a pretrained SigLIP-2 vision encoder with an LLaDA-8B diffusion language backbone via visual instruction tuning. Architecturally, PerceptionDLM introduces several enhancements to enable parallel region-conditioned captioning:

• Region Prompting: Each region mask receives a unique, learnable embedding. These region prompts are broadcast and fused with RoI visual tokens, providing explicit region identity cues for the model.
• RoI-aligned Feature Replay: For each region, features are extracted from the vision encoder and projected as RoI-specific visual tokens that serve as input placeholders for subsequent text generation.
• Structured Attention Masking: The transformer applies a highly regularized attention mask so that, within the denoising process, caption tokens for a region attend only to (i) the shared global context, (ii) region-specific visual/textual tokens, and (iii) the corresponding textual prompt. Cross-region attention is explicitly masked, preventing attribute entanglement and interference between concurrent region captions.

  Figure 2: The parallel region perception architecture, with region prompting, RoI-aligned feature replay, and block-wise structured attention.

This combination yields a block-separable parallel generation mechanism, where captions are jointly decoded for all regions but spatial localization and attribute assignment remain systemically disentangled.

Benchmarking: ParaDLC-Bench

Evaluating parallel region perception requires benchmarks that jointly assess caption quality and efficiency under multi-mask scenarios. The paper introduces ParaDLC-Bench, an extension of DLC-Bench for dense multi-region evaluation. Each image is annotated with multiple instance masks, producing scenarios where region identification, attribute localization, and inter-region disentanglement become critical.

Evaluation proceeds as follows:

• The model generates captions for all annotated regions in parallel.
• An LLM judge (GPT-5.2) receives each (mask, caption) pair, answering positive (attribute) and negative (hallucination/mislocalization) questions for each region.
• Positive questions reward inclusion of true attributes and penalize errors; negative questions penalize cross-region hallucinations and attribute leakage.

  Figure 3: Detailed evaluation pipeline of ParaDLC-Bench, from simultaneous generation to LLM-based regionwise scoring.

Data statistics reveal a challenging benchmark: mask counts (per image) vary from 2 up to 8, with a high density of small, fine-grained regions, and 2,345 manually curated evaluation questions covering both intra- and inter-region interference.

Figure 4: Distribution of mask quantities and sizes in ParaDLC-Bench, highlighting the challenge of fine-grained multi-target localization.

Experimental Results

Extensive experiments demonstrate that PerceptionDLM-Base sets a new SOTA in open-source diffusion-based multimodal LLMs, outperforming LLaDA-V on 15/16 diverse V&L benchmarks (VQA, document, chart, fine-grained perception). On ParaDLC-Bench, PerceptionDLM achieves 62.4% average region captioning accuracy—nearly double that of the prior best diffusion VLM baselines (LLaDA-V: 35.2%; SDAR-VL: 31.3%).

Importantly, PerceptionDLM achieves strong results with substantial computational advantages:

• Parallel throughput scaling: With 4+ region masks per image, PerceptionDLM achieves a throughput speedup of up to 3.44× compared to sequential AR pipelines, with per-image latency almost invariant to region count.
• Competitive quality: On fine-grained captioning, PerceptionDLM approaches the accuracy of AR region-specific models (e.g., GAR-8B), yet completes the full evaluation set in less than 60% of the time required by the fastest AR competitor.

Ablation analyses confirm the necessity of all core architectural modules; removing region prompting or structured attention leads to catastrophic performance degradation (≤2% accuracy), indicating that region-conditioned grounding and interference mitigation are essential for robust parallel region perception.

Limitations and Future Research

While PerceptionDLM unlocks parallel efficiency for region perception, limitations remain. The multi-step denoising process of diffusion models still imposes a latency floor. Hallucinations and attribute entanglement can occur in cases of extreme region density or semantic ambiguity. Reasoning-intensive multimodal tasks (e.g., mathematical diagram understanding) reveal an upper bound in DLMs' reasoning capacity, as parallel decoding may dilute causal traceability; AR-derived models currently outperform DLMs on such benchmarks.

Theoretical developments in pseudo-trajectory distillation or step reduction, explicit region-contrastive losses, and hybrid training with reinforcement learning (as in recent LLM developments) are promising directions to address the residual efficiency and reasoning gaps.

Implications and Outlook

PerceptionDLM concretely demonstrates that diffusion-based multimodal models can outperform autoregressive counterparts in scenarios requiring scalable, fine-grained, dense region-level vision-language understanding. The architectural principles introduced—structured parallelism, disentangled region prompting, and mask-separable attention—are extensible to future work on panoptic scene parsing, spatially distributed visual reasoning, or densely annotated video perception. ParaDLC-Bench establishes a foundation for future comparative research in parallel multimodal localization.

The transition from sequential AR pipelines to parallel DLMs represents a paradigm shift in scalable multimodal perception, poised to influence both system design for efficiency-critical applications (e.g., robotics, video understanding) and the research agenda for next-generation VLM architectures.

Conclusion

PerceptionDLM proposes and validates a scalable parallel generation mechanism for region-level perception. By bridging local visual grounding and global sequence modeling in a diffusion paradigm, it attains a new efficiency-accuracy Pareto frontier in region-conditioned captioning. ParaDLC-Bench provides a challenging, reference-free evaluation framework for future research in parallel multimodal understanding. Continued architectural and algorithmic innovation in this direction will drive further advances in the practical deployment of high-throughput AI perception systems.