Diffusion-Aware Feature Fusion (DAFFus)

• The paper demonstrates that DAFFus improves classification by adaptively fusing multi-scale features from U-Net’s upsampling blocks.
• DAFFus is defined to resolve layer selection ambiguity by selectively integrating semantic and task-relevant features from a diffusion model.
• Empirical results show DAFFus outperforms other fusion methods, boosting accuracy by up to 8.3% on benchmark AMC datasets.

Diffusion-Aware Feature Fusion (DAFFus) denotes a feature-fusion formulation in which representations produced by a diffusion model are aggregated to form a more discriminative downstream embedding. In the strict sense, the term refers to the module introduced in "ModFus-DM: Explore the Representation in Modulated Signal Diffusion Generated Models," where DAFFus operates as a feature-fusion head over a frozen modulated signal diffusion generation model (MSDGM) for automatic modulation classification (AMC) (Tan et al., 3 Aug 2025). In a broader interpretive sense, closely related papers use different names while pursuing the same design logic: diffusion is treated not only as a generative mechanism, but also as a structured source of multi-scale, multi-stage, or condition-aware features whose fusion improves classification, segmentation, multimodal understanding, or generation. This suggests that DAFFus is both a concrete module and a useful conceptual lens for a wider research pattern, although the broader terminology is not standardized.

1. Definition and original problem formulation

DAFFus was proposed to address a specific representational problem in AMC: although a diffusion model can learn modulation semantics through progressive denoising, no single U-Net block consistently provides the best feature for classification across datasets, signal-to-noise ratios, and signal characteristics (Tan et al., 3 Aug 2025). The paper identifies two failure modes. First, there is layer selection ambiguity: shallow or downsampling layers emphasize local, low-level patterns, whereas deeper or upsampling layers contain more semantic, reconstructed, and task-relevant content. Second, single-layer features are incomplete: a single block may omit either fine-grained local structure or global semantic information.

Within ModFus-DM, DAFFus is therefore defined as a module that adaptively aggregates multi-scale diffusion features extracted from multiple U-Net blocks of a trained diffusion backbone. The backbone itself, MSDGM, is trained in a self-supervised manner and then frozen. DAFFus is trained afterward, using limited labeled data, as a discriminative head over those frozen diffusion features. Architecturally, the method is explicitly two-stage: Stage 1 trains MSDGM on unlabeled signals, and Stage 2 freezes MSDGM and trains DAFFus plus the classifier on labeled signals per type per SNR, with experiments using $N = 2, 5, 10, 20$ (Tan et al., 3 Aug 2025).

The original formulation is thus narrower than generic multimodal fusion. DAFFus does not fuse arbitrary backbone outputs, and it is not described as a handcrafted ensemble heuristic. It is a structured aggregation mechanism designed around the internal hierarchy of a diffusion U-Net and the semantics of the denoising process itself.

2. Architectural formulation within ModFus-DM

In ModFus-DM, the diffusion model uses a U-Net as the reverse-process noise predictor, and DAFFus collects intermediate features from multiple U-Net blocks according to

$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$

where $s_t$ is the noisy signal at diffusion step $t$, ${\cal U}_{\theta^*}^{b_i}$ is the trained MSDGM feature output at block $b_i$, ${\cal A}{\cal P}$ pools the final dimension to 1, and $L$ is the number of fusion layers (Tan et al., 3 Aug 2025). The reported configuration uses feature dimension $d = 128$ and number of fusion layers $L = 8$.

The decisive design choice is block selection. Rather than using all blocks uniformly, DAFFus fuses features from the upsampling blocks $${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$0–$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$1. The fusion step is channel-wise concatenation,

$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$2

followed by a learnable projection,

$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$3

where $${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$4 and $${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$5 are trainable parameters, $${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$6 is an activation function, and $${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$7 is the final discriminative feature (Tan et al., 3 Aug 2025). Classification then uses a softmax layer,

$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$8

with cross-entropy-style loss

$${\cal F}_s = \left\{ f_s^{b_i} = {\cal A}{\cal P}\left( {\cal U}_{\theta^*}^{b_i}(s_t) \right) \mid i \in 1,\cdots,L \right\},$$9

The diffusion backbone on which DAFFus depends is trained with total diffusion steps $s_t$0, for 2000 epochs, using AdamW and learning rate 0.0002. DAFFus itself is trained for 50 epochs with Adam, and its learning rate decays from 0.01 to 0 via cosine annealing (Tan et al., 3 Aug 2025). At inference, no special DAFFus-specific test-time adaptation is described.

3. Why the method is “diffusion-aware”

The defining property of DAFFus is not mere multi-layer aggregation, but the fact that the aggregation rule is derived from the structure of a diffusion denoising hierarchy. The underlying MSDGM is trained with the standard forward and reverse processes,

$s_t$1

and noise-prediction loss

$s_t$2

(Tan et al., 3 Aug 2025). DAFFus is “aware” of this process because it assumes that different blocks correspond to different semantic and structural levels induced by progressive denoising.

The original paper contrasts three strategies: Fusion down, which uses downsampling blocks $s_t$3–$s_t$4; Fusion all, which uses $s_t$5–$s_t$6; and DAFFus, which focuses on $s_t$7–$s_t$8 (Tan et al., 3 Aug 2025). The rationale is explicit. Downsampling features are weaker because information is lost. Fusing all blocks is also suboptimal because low-quality downsampling features dilute the representation. Upsampling blocks, by contrast, combine deeper semantic content with skip-connected detail and are therefore more discriminative for AMC.

This design differs from generic feature fusion in three stated ways. It does not fuse arbitrary backbone features, it does not rely on handcrafted layer selection, and it does not treat all layers equally (Tan et al., 3 Aug 2025). A plausible implication is that DAFFus should be understood as a representation-selection mechanism over a denoising trajectory, rather than as a generic concatenation block.

4. Empirical behavior and ablation evidence

The abstract reports that ModFus-DM achieves over 88.27% accuracy in 24-type recognition tasks at SNR $s_t$9dB with only 10 labeled signals per type, while also outperforming existing methods in limited-label settings, distribution shifts, variable-length signal recognition, and channel fading scenarios (Tan et al., 3 Aug 2025). The DAFFus-specific ablations make clear that these gains are tied to multi-block diffusion feature fusion rather than to the diffusion backbone alone.

Using $t$0 as input, DAFFus outperforms the second-best single block $t$1 by 2.4% on RML2016.10A, 8.3% on RML2016.10B, 3.9% on RML2018.01A, and 2.3% on RML2022 (Tan et al., 3 Aug 2025). Against alternative fusion schemes, again with $t$2 as input, DAFFus improves over Fusion down by 13.36% and over Fusion all by 1.12%. The t-SNE analysis shows the clearest clustering for DAFFus, supporting the claim that the fused feature is more separable.

The same ablation section also refines the interpretation of block quality. The paper notes that $t$3 and $t$4 often produce the best single-block results, while $t$5 is less discriminative because it is too biased toward reconstruction fidelity rather than classification usefulness (Tan et al., 3 Aug 2025). This detail is important because it shows that “deeper” is not automatically “better”: the most useful representation lies in a middle-to-late regime of the reverse U-Net, and DAFFus benefits from combining several such regimes instead of selecting only the last block.

5. Related formulations across domains

Although the exact term DAFFus is specific to ModFus-DM, several adjacent papers instantiate closely related ideas under different names or as explicit analogues.

Method	Domain	Relation to DAFFus
ModFus-DM (Tan et al., 3 Aug 2025)	AMC	Canonical DAFFus module
DIFF (Ji et al., 2024)	Domain generalization segmentation	Conceptually similar, different name
Dif-Fusion (Yue et al., 2023)	Infrared-visible image fusion	Uses diffusion features for fusion
Text-DiFuse (Zhang et al., 2024)	Multi-modal image fusion	Embeds fusion inside diffusion steps
MLFFM-SegDiff (Gu et al., 25 Jun 2026)	Skin lesion segmentation	Diffusion-conditioned multi-level fusion
Align4Gen (Lee et al., 11 Sep 2025)	Video generation	Fuses external features with diffusion internals
Thermographic breast cancer framework (Salem et al., 8 Sep 2025)	Medical classification	DPM augmentation plus fused deep and nonlinear descriptors

The strongest “same-family” relation is the breast thermography framework in "Breast Cancer Detection in Thermographic Images via Diffusion-Based Augmentation and Nonlinear Feature Fusion." There, a DPM generates 1000 synthetic grayscale ROI patches, evenly split between benign and malignant classes, and these diffusion-augmented samples are represented through a fusion of 2048-dimensional ResNet-50 features and 4-dimensional handcrafted nonlinear descriptors, producing a 2052-dimensional fused vector for an XGBoost classifier (Salem et al., 8 Sep 2025). The full model reaches 98.0% accuracy, 98.1% sensitivity, and 97.9% specificity, with the improvement over baselines reported as statistically significant with $t$6. The paper explicitly frames this as diffusion enriching the feature space that later fusion exploits, which is why it was described as DAFFus-like.

A second important clarification concerns terminology. "Diffusion Features to Bridge Domain Gap for Semantic Segmentation" names its method DIFF (DIffusion Feature Fusion), not DAFFus (Ji et al., 2024). DIFF uses Stable Diffusion v1-5, extracts multi-step U-Net decoder latent variables and cross-attention maps, concatenates them across timesteps and layers, and fuses them with a simple residual bottleneck. The paper is therefore conceptually aligned with DAFFus but does not claim the same identity.

Other neighboring methods expand the same design principle into different operational settings. "Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractor" studies concatenation and cross-attention fusion between CLIP and question-conditioned diffusion features (Agarwal et al., 9 Jul 2025). "Improving Video Diffusion Transformer Training by Multi-Feature Fusion and Alignment from Self-Supervised Vision Encoders" fuses DINOv2 and SAM2.1 Hiera features and aligns them to intermediate video diffusion transformer features (Lee et al., 11 Sep 2025). "Ensembling Diffusion Models via Adaptive Feature Aggregation" performs block-wise, spatially varying, timestep- and prompt-conditioned fusion across multiple U-Net denoisers (Wang et al., 2024). "Text-DiFuse" embeds feature-level multi-modal fusion directly into each reverse diffusion step through a Fusion Control Module (Zhang et al., 2024), while "MLFFM-SegDiff" uses attention-enhanced, scale-aligned, adaptive cross-level fusion inside a conditional diffusion segmentor (Gu et al., 25 Jun 2026). A plausible implication is that DAFFus now denotes a broader architectural motif: diffusion-aware fusion can occur after pretraining, during denoiser training, or inside iterative reverse sampling.

6. Misconceptions, limitations, and research significance

A common misconception is to treat DAFFus as a generic label for any use of diffusion features. The literature does not support that equivalence. DAFFus is an explicit module name in ModFus-DM (Tan et al., 3 Aug 2025); DIFF is a separate method name in domain generalization segmentation (Ji et al., 2024); MLFFM, Align4Gen, AFA, Dif-Fusion, and Text-DiFuse use their own terminology and optimization logic. The family resemblance is architectural rather than nominal.

A second misconception is that diffusion-aware fusion always means adding diffusion outputs after generation. Several papers instead fuse intermediate features or fuse within the reverse process. AFA aggregates block-wise U-Net features using spatial attention maps conditioned on prompts, timesteps, and latent state, while keeping all base denoisers frozen (Wang et al., 2024). Text-DiFuse repeats modality fusion at every reverse step rather than only once at input or output (Zhang et al., 2024). Align4Gen integrates feature alignment into the diffusion or flow training objective and removes the alignment MLP at inference (Lee et al., 11 Sep 2025). These variants indicate that the timing and location of fusion are themselves major design axes.

The limitations reported in adjacent work are also instructive. Some DAFFus-like methods incur substantial computational cost because they preserve iterative diffusion dynamics or run multiple denoisers per step. AFA notes higher per-step inference cost than parameter merging because all base models are executed at each step (Wang et al., 2024), and Text-DiFuse explicitly notes the burden of $t$7-step sampling (Zhang et al., 2024). In multimodal understanding, question-conditioned diffusion features can introduce a leakage phenomenon, where the LLM recovers information from the original diffusion prompt rather than the image; with guidance scale $t$8, the mismatched setting still achieves a CIDEr score of 12.97, and caption dropout is proposed as mitigation (Agarwal et al., 9 Jul 2025). These observations indicate that diffusion-aware fusion improves representational richness, but it can also amplify computational overhead or unintended conditioning channels.

Taken together, the literature establishes DAFFus as a distinctive response to a recurring problem: useful information in diffusion models is distributed across timesteps, layers, modalities, and conditioning pathways rather than localized in a single final representation. The original AMC formulation provides the clearest canonical definition, while subsequent work in medical imaging, segmentation, multimodal understanding, video generation, and diffusion-model ensembling suggests that the same principle generalizes widely whenever the denoising hierarchy itself is treated as a structured feature space rather than merely as a sampler.