Shadow Injection Module in Generative Models

• Shadow Injection Modules are parametrized mechanisms that integrate shadow signals into generative models, enabling precise control over shadow synthesis, removal, and adaptation.
• They employ techniques like dual-stream conditioning, cross-attention, and per-pixel modulation to ensure spatial accuracy and semantic consistency across vision and language tasks.
• Empirical results demonstrate improved image quality and robust language model adaptation, validating the efficacy of these modules in multi-modal generative systems.

A Shadow Injection Module is a parametrized mechanism for integrating shadow-relevant signals into a generative model—in particular, within diffusion or transformer architectures—to control the synthesis, removal, or adaptation of shadows in images or latent representations. Its design can encompass diverse injection modalities, ranging from explicit feature modulation and cross-attention, to residual state refinement in neural LLMs. This article details the conceptual foundation, mathematical formalism, integration strategies, representative implementations, and empirical outcomes across vision and language domains.

1. Shadow Injection Modules in Image Synthesis

Shadow Injection Modules are primarily utilized in conditional generative models for shadow synthesis (or removal) in composited imagery. They modulate the generative process by injecting geometric, semantic, and scene-level information to steer the formation of visually plausible and physically consistent shadows.

Dual-Stream Conditioning in Diffusion Pipelines

A prominent instantiation is found in CoShadow, which addresses multi-object shadow generation by extending a Stable Diffusion U-Net backbone with parallel conditioning streams (Ahmed et al., 3 Mar 2026). The module comprises:

• Image Pathway: Extraction of dense, multi-scale features from the shadow-free composite image and object masks. Features are injected via geometry-aware affine modulation (GAAM), where spatially adaptive scale and bias maps are learned and applied to the U-Net activations at various encoder/decoder depths.
• Text Pathway: A frozen CLIP text encoder parses prompts with embedded, quantized shadow-box positional tokens. These learned tokens, representing per-object shadow bounding boxes, are injected via cross-attention at the middle and late U-Net layers. An attention-alignment loss grounds each token’s attention to the corresponding shadow region.
• The joint conditioning affords the model (i) spatially precise shadow placement and (ii) semantic disambiguation among multiple inserted foregrounds.

Other conditional diffusion models, including VSDiffusion, leverage visibility priors and cross-modal feature gating as part of shadow injection. In this case, shadow-gated cross-attention fuses learned illumination and depth priors into the denoising U-Net at multiple scales, and a learned soft-prior map reweights pixel-wise losses to refine shadow structure (Li et al., 9 Mar 2026).

Per-pixel Mask and Intensity Modulation

Shadow Injection Modules may instead focus on intensity control and spatial selectivity of the denoising process. For example, in a ControlNet-based pipeline (Liu et al., 2024), the module injects conditioning via:

• A control encoder for spatial grounding of shadow regions,
• An intensity encoder producing global scale and bias vectors based on background shadow statistics,
• A shadow mask predictor generating soft spatial masks,
• Channel-wise noise modulation applied only within predicted shadow regions, modulating the output of the diffusion denoiser for intensity match with surrounding shadows.

These approaches establish a robust mechanism for location- and property-aware shadow formation during synthesis.

2. Shadow Injection in Document and Object-centered Tasks

For document shadow removal, DocShaDiffusion pioneers a shadow soft-mask generation module to spatially inject noise and mask-aware diffusion steps in latent space. The process includes:

• Soft Shadow Mask: Derived via image preprocessing (grayscale conversion, dilation, median filtering, adaptive thresholding) to yield a continuous mask indicating shadow probability per pixel.
• Mask-aware Diffusion: During both forward (noise injection) and reverse (denoising) processes, the mask selectively amplifies or suppresses stochasticity only in shadowed regions, preserving unshadowed content (Liu et al., 2 Jul 2025).

Object-centered frameworks such as MetaShadow employ a reference-guided shadow injection module: features representing reference shadow shape, direction, and intensity are extracted by a frozen analyzer and injected into the synthesizer via cross-attention in U-Net blocks. This enables sample-consistent, reference-controllable shadow generation, tightly localized by explicit object masks (Wang et al., 2024).

3. Parameter-Efficient Fine-Tuning via Shadow Injection in LLMs

Shadow Injection Modules extend beyond vision, providing depth-shared, centralized adaptation of LLMs. ShadowPEFT introduces a shadow network that:

• Maintains a single shadow state across all decoder layers,
• Injects layer-wise corrections to the frozen backbone via low-rank projections of the discrepancy between the base and shadow states,
• Updates the shadow state through gated residual mechanisms at each depth,
• Enables independent pretraining, scalable parameter sharing, and deployment in a detachable mode for edge inference (Li et al., 21 Apr 2026).

The module is the locus for all trainable adaptation, sharply contrasting with the distributed, local perturbations in LoRA or DoRA. Empirically, this global, recurrent adaptation yields improved in-distribution performance and robustness to out-of-domain transfer, with minimal overhead.

4. Mechanistic Formalization

A general formalization of Shadow Injection Module behavior in image diffusion takes the form

$$\tilde{U}^l = \mathrm{Modulate}(U^l; F^l, T^{\text{shd}})$$

where $U^l$ is the U-Net activation, $F^l$ are local image features (e.g., GAAM), and $T^{\text{shd}}$ encodes injected symbolic or box-wise tokens (text), or priors (illumination, depth). The modulation can be:

• Affine: $$\mathrm{GN}(U^l) \odot (1 + \Delta^l(F^l)) + B^l(F^l)$$
• Cross-attention: feature/embedding-based, using learned tokens and ground-truth alignment via KL or Dice loss
• Mask-guided: concatenating masks or modulating residuals/noise only within spatial regions defined by masks

For transformer-based LLMs, the operation involves projecting discrepancies between states through low-rank adapters and merging via residual addition, followed by gated shadow state updates.

5. Integration Strategies and Optimization

Integration of Shadow Injection Modules occurs at preselected semantic or spatial bottlenecks—encoder/decoder block boundaries, or multi-scale injection anchors. Modules operate under end-to-end differentiable loss structures that may include:

• Denoising or reconstruction objectives (L2/losses for DDPM, cross-entropy for LLMs)
• Region- or mask-weighted terms to emphasize shadowed content
• Alignment losses to enforce spatial correspondence (KL divergence for attention, Dice for mask predictions)
• Auxiliary objectives for background preservation, local realism, or perceptual consistency

Optimization requisites (e.g., AdamW, Lion, EMA) and explicit training schedules (learning rate decay, staged curriculum) are tuned per-task.

6. Empirical Outcomes and Evaluation

State-of-the-art shadow injection strategies have established notable performance improvements:

• Multi-object, physically plausible shadow compositing (CoShadow: 29.16-30.73 PSNR, 93.49 SSIM (Wang et al., 2024))
• Accurate shadow intensity alignment with background and superior spatial coherence (Liu et al., 2024)
• Robustness in shadow removal for document enhancement via spatially masked diffusion (Liu et al., 2 Jul 2025)
• Outperforming LoRA/DoRA in LLM adaptation, scaling, and edge deployment scenarios (Li et al., 21 Apr 2026)

Extensive ablation confirms the necessity of per-object conditioning, alignment losses, and centralized adaptation mechanisms for optimal outcomes.

7. Representative Design Variants

Model/Domain	Injection Mechanism	Key Form of Conditioning
CoShadow	GAAM + cross-attention w/ shadow box tokens	Per-object geometry/spatial region
VSDiffusion	SGCA (shadow-gated cross-attention) + soft prior	Visibility, lighting, depth priors
SGDiffusion	Affine modulation via intensity encoder	Global + local intensity consistency
MetaShadow	Cross-attention to reference feature code	Full-featured shadow knowledge
DocShaDiffusion	Masked noise and latent gating	Soft shadow region
ShadowPEFT	Low-rank layer-space injection, shadow state	Layer-shared rep. for all inputs
SSN	Ambient occlusion + light-injected U-Net	Mask, user-edit, environment map

8. Limitations and Future Directions

The explicit reliance on precomputed or pretrained features (e.g., CLIP, shadow analyzers, visibility estimators) constrains generalization to domains beyond the training distribution. Modular shadow injection facilitates fine-grained control but may require task-specific adaptation for extensions to reflections, inter-object shadow dynamics, or complex domain transfers. Continual advances in scalable cross-modal representation learning and region-centric conditioning are anticipated to yield further gains in both vision and LLM adaptation (Ahmed et al., 3 Mar 2026, Li et al., 9 Mar 2026, Wang et al., 2024, Li et al., 21 Apr 2026).