Occlusion-Aware Diffusion Models (ODM)

• Occlusion-Aware Diffusion Models (ODM) are diffusion frameworks that explicitly incorporate occlusion as a modeling variable to infer hidden structures and maintain visual consistency.
• They leverage techniques such as occluded input conditioning, amodal completion, and reverse-process masking to achieve robust 3D reconstruction, virtual try-on, and object recognition.
• ODM architectures employ both implicit strategies like teacher–student self-supervision and explicit modules including visibility maps to optimize performance across diverse applications.

Occlusion-Aware Diffusion Model (ODM) denotes a class of diffusion-based, latent-diffusion, or rectified-flow systems that treat occlusion as an explicit modeling variable rather than as incidental corruption. In papers, that explicitness appears in several forms: conditioning directly on occluded inputs and learning amodal completion, predicting visibility maps for layered composition, clamping observed regions during reverse diffusion, encoding occlusion order through cumulative masks or Z-order, or using diffusion priors as occlusion-robust feature extractors. "DeOcc-1-to-3" is a canonical recent formulation in single-image 3D reconstruction, converting a standard multi-view diffusion backbone into an ODM by training on occluded inputs with multi-view pseudo-ground-truth from corresponding unoccluded images (Qu et al., 26 Jun 2025). Parallel formulations have appeared in virtual try-on, layout-grounded image generation, visual active tracking, amodal segmentation, biometric inpainting, pedestrian intention prediction, and object recognition under partial occlusion (Yu et al., 20 Jan 2026, Li et al., 20 May 2026, Sun et al., 23 Apr 2026, Ao et al., 2024, Liu et al., 2 Nov 2025, Mallick et al., 8 Apr 2025).

1. Terminology and conceptual scope

The acronym ODM is not a uniform formal label across the literature. Some papers explicitly name an “Occlusion-Aware Diffusion Model,” while others state that the acronym is not used but that the method nevertheless instantiates the same principles. The shared idea is that diffusion is not merely used to denoise or generate, but to reason about hidden structure, visibility, or layer ordering in a way that remains useful for a downstream objective such as 3D reconstruction, compositing, planning, or recognition (Qu et al., 26 Jun 2025, Li et al., 20 May 2026, Zhu et al., 2024).

This commonality is clearest in the distinction between visible-only generative control and occlusion-aware generation. In "DeOcc-1-to-3," the core claim is that standard single-image-to-3D pipelines based on multi-view diffusion implicitly assume fully visible objects and therefore hallucinate inconsistent geometry under partial occlusion, breaking downstream 3D reconstruction (Qu et al., 26 Jun 2025). In "OcclusionFormer," the corresponding failure mode is ambiguity in overlap regions when layouts specify only 2D boxes without explicit Z-order, yielding texture entanglement and physically inconsistent layering (Li et al., 20 May 2026). In "Sequential Amodal Segmentation via Cumulative Occlusion Learning," the analogous problem is the inability to predict occluded regions and occlusion order for uncertain object categories unless cumulative foreground structure is explicitly modeled during diffusion (Ao et al., 2024).

A concise operational definition therefore emerges from the papers themselves: an ODM takes incomplete or ambiguously layered observations, infers hidden or suppressed content, and constrains the generated output so that it remains consistent with a visibility model relevant to the task. Depending on the application, that visibility model may be a binary mask, a visibility-refined latent, a cumulative occlusion mask, a target bounding box, an ordered occluder set, or an energy term tied to interpenetration and depth ordering (Yu et al., 20 Jan 2026, Sun et al., 23 Apr 2026, Han et al., 22 Mar 2025).

2. "DeOcc-1-to-3" as a reference ODM for single-image 3D reconstruction

"DeOcc-1-to-3" frames ODM most explicitly in the setting of single-image-to-multi-view generation for 3D reconstruction. The method keeps the Zero123++ architecture unchanged and fully fine-tunes it as an end-to-end occlusion-aware multi-view generator. The backbone is a U-Net latent diffusion model that outputs a $3 \times 2$ tiled image containing six canonical views. The model retains the two image-based conditionings inherited from Zero123++: local conditioning via scaled reference attention to the input image, and global conditioning via CLIP image embeddings. No text prompts are used, and no occlusion-specific modules or mask encoders are added (Qu et al., 26 Jun 2025).

The key mechanism is self-supervised teacher–student training. Occlusions are created on 2D images using SAM masks and compositing, producing a clean image $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$ and an occluded image $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$. For each unoccluded $I_{\text{full}}$, a frozen teacher generates six-view pseudo-ground-truth $G(I_{\text{full}})$ in the same canonical $3 \times 2$ layout. The student is then trained to map $I_{\text{occ}}$ to that target under the standard denoising objective

$$L_{\text{denoise}} = \mathbb{E}_{x_0,\epsilon,t}\left[\left\|\epsilon - \epsilon_\theta(x_t, t \mid I_{\text{occ}})\right\|^2\right],$$

with $x_0 = G(I_{\text{full}})$. Completion is therefore learned implicitly, because the conditioning hides object parts while the target depicts the complete object (Qu et al., 26 Jun 2025).

The paper explicitly rejects several mechanisms often assumed necessary for occlusion-aware generation. It does not introduce a masked completion loss, a perceptual loss restricted to occluded pixels, a cross-view consistency loss, geometric warping losses, epipolar or cycle consistency, or 3D feature sharing. Consistency arises instead from the teacher-generated six-view target, the Zero123++ backbone’s inherent multi-view coherence, and training on a single tiled target that couples all six views through a shared U-Net and shared conditioning (Qu et al., 26 Jun 2025).

The camera set is fixed and identical to Zero123++, with elevations $\{30^\circ, -20^\circ\}$ and azimuths $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$0. Training uses AdamW with learning rate $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$1, batch size $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$2, and $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$3k steps; the full U-Net is fine-tuned, the CLIP encoder and reference attention are unchanged, and EMA with $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$4 is used for inference. The self-supervised 2D occlusion dataset contains $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$5K samples, and after filtering the final 3D de-occlusion training set contains approximately $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$6K high-quality pairs (Qu et al., 26 Jun 2025).

Its evaluation is tied directly to 3D reconstruction. Six generated views are passed to InstantMesh, and quality is measured on Occ-LVIS with Chamfer Distance, F-Score, and Volume IoU. On Occ-LVIS, DeOcc-1-to-3 reports CLIP $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$7, FID $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$8, KID $$I_{\text{full}} = I_{\text{raw}} \odot M_{\text{full}}$$9, CD $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$0, F-Score $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$1, and V-IoU $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$2, outperforming both 3DRecon and the two-stage Pix2Gestalt-based pipeline reported in the same comparison (Qu et al., 26 Jun 2025).

3. Architectural patterns that make diffusion occlusion-aware

Across the literature, ODMs fall into several recurring architectural patterns. One pattern is implicit occlusion awareness through training distribution and supervision, exemplified by "DeOcc-1-to-3," where the architecture is unchanged and occlusion awareness emerges from pairing occluded inputs with amodal multi-view teacher targets (Qu et al., 26 Jun 2025). A second pattern is explicit visibility estimation, exemplified by "GO-MLVTON," which predicts an occlusion attention map

$$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$3

and applies it to the inner-garment latent as

$$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$4

The refined latent replaces the raw inner-garment latent in the Stable Diffusion-based morphing and fitting module, and a dedicated supervision term

$$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$5

aligns the refined inner latent with the visible inner garment region from ground truth (Yu et al., 20 Jan 2026).

A third pattern is reverse-process masking, in which observed content is preserved analytically while diffusion reconstructs only the missing content. The pedestrian-intention ODM is explicit on this point: for observed tokens it uses the exact posterior $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$6, for occluded tokens it uses the learned reverse model, and the two are blended at each reverse step as

$$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$7

This is paired with an occlusion-aware diffusion transformer that uses AdaLN-style conditioning, temporal and spatial offsets, and an occlusion masking block to reconstruct masked latent tokens while preserving observed tokens (Liu et al., 2 Nov 2025). Closely related mask-conditioned behavior appears in accessory-aware ear inpainting, where known pixels are clamped at every denoising step and only the accessory mask region is synthesized (Arun et al., 27 Jan 2026).

A fourth pattern is compositional visibility rendering. "OcclusionFormer" explicitly models Z-order priority by predicting a time-dependent per-instance density $$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$8, converting it to opacity

$$I_{\text{occ}} = I_{\text{mix}} \odot M_{\text{occ}}$$9

computing transmittance

$I_{\text{full}}$0

and compositing instance features via volume rendering. In this formulation, occlusion is neither an afterthought nor a purely local mask operation; it is a latent-space rendering rule tied to an explicit occlusion graph (Li et al., 20 May 2026).

A fifth pattern is guidance by geometry or interaction energy. In the two-hand reconstruction framework, diffusion is guided by interpenetration and occlusion-aware energies computed from mesh distances, normals, and depth ordering. The reverse dynamics use an energy-guided noise estimate

$I_{\text{full}}$1

where $I_{\text{full}}$2 contains penetration, contact, and occlusion-order terms. This makes occlusion awareness an explicit property of the sampling trajectory rather than only of the training data (Han et al., 22 Mar 2025).

Finally, some ODMs treat diffusion less as a sampler than as a structured prior. "DPMesh" uses a pre-trained Stable Diffusion U-Net with ControlNet conditioning as a single-step image backbone for occluded human mesh recovery, extracting features and cross-attention maps rather than performing iterative diffusion sampling. This demonstrates that, within the literature, “diffusion model” in ODM can refer either to a full generative reverse process or to a diffusion-pretrained representation used for occlusion reasoning (Zhu et al., 2024).

4. Representative task instantiations

The same ODM principle has been instantiated across distinct problem classes. The mechanisms differ, but each system uses diffusion together with an explicit treatment of hidden content, layer ordering, or visibility-constrained recovery.

System	Task	Representative reported results
"DeOcc-1-to-3" (Qu et al., 26 Jun 2025)	Single-image-to-3D under occlusion	Occ-LVIS: CLIP 0.7892, FID 29.0836, KID 0.0035; CD 0.0086, F-Score 0.4835, V-IoU 0.3445
"GO-MLVTON" (Yu et al., 20 Jan 2026)	Multi-layer virtual try-on	FID 22.82, KID 0.35, SSIM 0.858, LPIPS 0.108, LACD 0.623
"OA-VAT" (Sun et al., 23 Apr 2026)	Occlusion-aware trajectory planning	0.93 average SR on UnrealCV, 90.8% average CAR, 81.6% TSR, 35 FPS
"OcclusionFormer" (Li et al., 20 May 2026)	Layout-grounded image generation	OverLayBench–Complex: mIoU 0.6037, O-mIoU 0.3468, Occ. 0.7797, Dep. 0.1602
"ODM for Pedestrian Intention Prediction" (Liu et al., 2 Nov 2025)	Intention prediction under incomplete observations	Bounding-box ADE remains ≈15 pixels; under EO5 on PIE, improves over TrEP by up to ≈5% Acc, 4% AUC, 5% F1

Additional task families broaden the concept further. "O$I_{\text{full}}$3-Recon" combines a pre-trained 2D diffusion inpainting model with per-instance neural implicit optimization to complete surfaces for hidden object parts in scene-level RGB-D videos; it uses human-guided mask generation, projected masks, a cascaded SDF, and CLIP-based semantic consistency to convert 2D plausible completion into 3D-consistent geometry (Hu et al., 2023). "GOATex" makes mesh texturing occlusion-aware through hit levels computed by multi-view ray casting, then progressively reveals and textures deeper visibility layers with Stable Diffusion 1.5 and depth ControlNet, finally merging layers by soft UV-space blending (Kim et al., 28 Nov 2025). "DiffOOM" couples a de-occlusion branch and a movement branch so that an object can be completed and relocated simultaneously within a scene, using color fill, latent hold, progressively refined attention masks, latent optimization, and local text-conditioned guidance (Duan et al., 2 Apr 2025).

Recognition-oriented ODMs adopt yet another formulation. In accessory-aware ear recognition, diffusion inpainting is a preprocessing stage that reconstructs masked ear anatomy from automatically derived accessory masks before ViT-based verification, with the largest gains reported on occlusion-heavy settings such as EarVN1.0 (Arun et al., 27 Jan 2026). In D-Feat Occlusions, a frozen Stable Diffusion v2.1 model is used either to inpaint missing regions or to provide diffusion features fused with classifier features, improving robustness of Swin-Base and ConvNeXt-Base on ImageNet under simulated and real occlusions (Mallick et al., 8 Apr 2025).

5. Supervision, objectives, and evaluation protocols

ODM training is not tied to a single supervision regime. "DeOcc-1-to-3" uses self-supervised teacher–student denoising, with pseudo-ground-truth views from unoccluded images and no explicit completion or warping loss (Qu et al., 26 Jun 2025). "GO-MLVTON" adds a dedicated visibility loss $I_{\text{full}}$4 to a latent diffusion objective $I_{\text{full}}$5, yielding an overall loss $I_{\text{full}}$6 with $I_{\text{full}}$7 (Yu et al., 20 Jan 2026). "OcclusionFormer" combines rectified flow with a queried alignment loss, optimizing

$I_{\text{full}}$8

with $I_{\text{full}}$9, thereby tying instance-level mask precision directly to the generative objective (Li et al., 20 May 2026). "Sequential Amodal Segmentation via Cumulative Occlusion Learning" instead keeps the RGB image and cumulative mask fixed and diffuses only the amodal mask channel, using the standard DDPM noise-prediction loss on the current mask (Ao et al., 2024).

Evaluation protocols are correspondingly task-specific. "DeOcc-1-to-3" introduces Occ-LVIS, based on Objaverse-LVIS assets with controlled occlusions, canonical and random views, and 3D ground truth. Its occlusion stratification is given by visible fraction: L1: $G(I_{\text{full}})$0–$G(I_{\text{full}})$1 ($G(I_{\text{full}})$2), L2: $G(I_{\text{full}})$3–$G(I_{\text{full}})$4 ($G(I_{\text{full}})$5), L3: $G(I_{\text{full}})$6–$G(I_{\text{full}})$7 ($G(I_{\text{full}})$8), L4: $G(I_{\text{full}})$9–$3 \times 2$0 ($3 \times 2$1), and L5: $3 \times 2$2 ($3 \times 2$3). The protocol uses FID, KID, and CLIP for 2D quality, and CD, F-Score, and V-IoU for 3D reconstruction (Qu et al., 26 Jun 2025). "GO-MLVTON" introduces the MLG dataset and the Layered Appearance Coherence Difference (LACD), which emphasizes boundary and transition regions between garment layers (Yu et al., 20 Jan 2026). "OcclusionFormer" relies on OverLayBench and SA-Z Eval, with mIoU and O-mIoU for spatial accuracy, Occ. for occlusion-order F1, and Dep. for depth-order WHDR (Li et al., 20 May 2026). OA-VAT evaluates planning with Success Rate, Correct Action Rate, and Tracking Success Rate, and ties planner performance to a dedicated Planning-20k dataset of occluded recovery trajectories (Sun et al., 23 Apr 2026).

This heterogeneity is important. ODM is not defined by a universal benchmark but by a recurring modeling objective: the generated output must remain valid under missing visibility. In practice, that validity is assessed as view consistency and 3D shape quality in reconstruction, inter-layer coherence in try-on, physically correct front–back arrangement in layout generation, reliable recovery behavior in tracking, or restored discriminability in recognition (Qu et al., 26 Jun 2025, Yu et al., 20 Jan 2026, Li et al., 20 May 2026, Arun et al., 27 Jan 2026).

6. Misconceptions, limitations, and open directions

A frequent misconception is that occlusion awareness necessarily requires new architectural components. "DeOcc-1-to-3" is explicit that no architectural change is required: the backbone, reference attention, CLIP encoder, sampling, and canonical output structure are retained, and occlusion awareness is learned from the data pairing and supervision strategy alone (Qu et al., 26 Jun 2025). "GOATex" similarly leaves the pretrained diffusion backbone unchanged and instead wraps it in geometry-driven visibility control, staged rendering, and UV-space blending (Kim et al., 28 Nov 2025). Conversely, other papers show that explicit visibility modules can be beneficial, as in GO-MLVTON’s Garment Occlusion Learning module or OcclusionFormer’s density-and-transmittance rendering (Yu et al., 20 Jan 2026, Li et al., 20 May 2026). The literature therefore does not support a single design rule.

A second misconception is that occlusion awareness is equivalent to explicit mask loss. Several methods contradict this. "DeOcc-1-to-3" learns completion without any $3 \times 2$4 or masked perceptual loss (Qu et al., 26 Jun 2025). The cumulative-occlusion amodal segmentation model relies on sequential conditioning with cumulative masks rather than an explicit occlusion-order loss (Ao et al., 2024). DPMesh uses diffusion-pretrained features and cross-attention rather than a reverse diffusion completion loss, yet is still presented as leveraging diffusion priors for occlusion-robust recovery (Zhu et al., 2024). This suggests that ODM is better understood as a modeling stance than as a fixed loss recipe.

The papers also identify recurrent limitations. "DeOcc-1-to-3" reports no ablation studies and no explicit per-level Occ-LVIS metrics, even though it qualitatively shows robustness across occlusion severities (Qu et al., 26 Jun 2025). Ear inpainting, O$3 \times 2$5-Recon, and DiffOOM are all sensitive to mask quality; missed detections, over-coverage, or inaccurate attention-derived masks can induce artifacts, drift, or identity changes (Arun et al., 27 Jan 2026, Hu et al., 2023, Duan et al., 2 Apr 2025). "OcclusionFormer" is sensitive to incorrect Z-order annotations, since wrong occluder sets can mis-layer objects (Li et al., 20 May 2026). OA-VAT learns from static-occlusion A* demonstrations and may be challenged by fast-moving obstacles (Sun et al., 23 Apr 2026). DPMesh notes that extreme occlusion or severe overlap can still produce ambiguous reconstructions (Zhu et al., 2024).

Recurring future directions are similarly task-specific but conceptually aligned. The papers propose stronger temporal modeling for long occlusions, semantic-aware refinement of geometric visibility layers, more explicit multi-view or cross-level consistency, learned visibility priors, and tighter integration of occlusion reasoning with downstream control or recognition objectives (Sun et al., 23 Apr 2026, Kim et al., 28 Nov 2025, Zhu et al., 2024). A plausible implication is that the next phase of ODM research will be less about whether diffusion can hallucinate plausible hidden content and more about how that hallucination is constrained by geometry, semantics, and task-level correctness.

In this sense, ODM is best understood not as a single model family with a fixed architecture, but as a research program: diffusion is combined with explicit visibility reasoning so that generated or reconstructed content remains useful when the observable image is incomplete. "DeOcc-1-to-3" is a particularly clear instantiation because it shows that, for single-image 3D reconstruction, strong occlusion awareness can emerge from full fine-tuning of an existing multi-view diffusion backbone under self-supervised occluded–unoccluded pairing, without extra modules, masked losses, or geometric warping terms (Qu et al., 26 Jun 2025).