Differential Feature Feedback Module (DFFM)

• The paper presents DFFM as a feedback mechanism that extracts differential guidance for iterative multispectral feature fusion.
• DFFM uses lightweight normalization and a two-layer MLP to project differences between refined visible and thermal features for enhanced detection.
• Empirical results show that adding DFFM improves mAP scores, underscoring its effectiveness in suppressing background interference and boosting salient details.

Searching arXiv for the specified paper to ground the article and citation. The Differential Feature Feedback Module (DFFM) is a module in IRDFusion, an “Iterative Relation-Map Difference guided Feature Fusion” framework for multispectral object detection that is designed to compute inter-modal differential features, use them as guidance signals, and feed them back into iterative feature refinement (Shen et al., 11 Sep 2025). Within the architecture, DFFM operates in conjunction with the Mutual Feature Refinement Module (MFRM): MFRM produces refined visible and thermal features, and DFFM extracts their pixel-wise or element-wise difference to highlight complementary information, suppress common-mode background interference, and support progressively cleaner and more discriminative cross-modal fusion (Shen et al., 11 Sep 2025).

1. Position within IRDFusion

IRDFusion is built around two sub-modules: the Mutual Feature Refinement Module (MFRM) and the Differential Feature Feedback Module (DFFM) (Shen et al., 11 Sep 2025). MFRM aligns and fuses the two modalities by means of intra-modal self-attention plus a small cross-modal gating, while DFFM sits “below” the MFRM in the diagram and receives the refined outputs $F_v'^{(k)}$ and $F_t'^{(k)}$ from the $k$-th iteration.

The stated role of DFFM is to compute the difference between the two refined modalities, use that difference to highlight strictly complementary information, and then feed it back into the next MFRM iteration. In this feedback loop, the module is intended to suppress the common-mode background noise and amplify the truly discriminative features. The resulting mechanism is described as an Iterative Relation-Map Differential Guided Feature Fusion process, in which salient relational signals are progressively amplified through iterative feedback while feature noise is suppressed (Shen et al., 11 Sep 2025).

This positioning makes DFFM the explicit feedback component of the IRDFusion block. A plausible implication is that the module is not merely an auxiliary differencing operator, but the mechanism through which iterative refinement becomes adaptive across successive fusion stages.

2. Core operational principle

The operational premise of DFFM is differential guidance. Given refined visible and thermal features from MFRM, the module computes an inter-modal difference $F_{\mathrm{dif}}$, applies normalization and a lightweight projection, and injects the resulting correction into the branch features for the next iteration (Shen et al., 11 Sep 2025). In the paper’s description, iterating this loop yields progressively cleaner, more discriminative, and better-aligned cross-modal features.

This behavior is explicitly framed as being inspired by feedback differential amplifiers. In that analogy, the differential component between modalities is treated as informative, whereas the shared component is treated as common-mode noise. The module therefore emphasizes information that is complementary across visible and thermal streams rather than information that is merely redundant. The paper summarizes this function as adaptively enhancing salient structures by fusing object-aware complementary cross-modal features while suppressing shared background interference (Shen et al., 11 Sep 2025).

Taken in context, DFFM is meant to address a failure mode in multispectral feature fusion in which extraneous background or noise is retained during fusion. This suggests that DFFM should be understood not as a generic residual enhancement layer, but as a modality-contrastive feedback mechanism specialized for cross-modal suppression and amplification.

3. Mathematical formulation

Let $F_v'^{(k)}$ and $F_t'^{(k)}$ denote the outputs of the $k$-th MFRM block for the visible and thermal modalities, respectively. The DFFM update for the visible branch is given as

$\begin{aligned} F^{(k)}_{\mathrm{dif\_v}} &= F_t'^{(k)} - \beta\,F_v'^{(k)},\ F_v^{(k+1)} &= \mu\,F_v^{(k)} + \alpha\;\mathrm{MLP}\bigl(\mathrm{LN}(F^{(k)}_{\mathrm{dif\_v})\bigr)\,. \end{aligned}$

Here, $\beta$ is a learnable scalar or vector that weights the contribution of $F_v'$ in the difference; $F_t'^{(k)}$0 is Layer Normalization; $F_t'^{(k)}$1 is a small two-layer feed-forward network; and $F_t'^{(k)}$2 and $F_t'^{(k)}$3 are learnable scalars that control, respectively, how strongly the differential correction is injected and how much of the previous feature $F_t'^{(k)}$4 is retained (Shen et al., 11 Sep 2025).

The paper notes that the thermal branch is kept fixed in the simplest form, since an explicit update for $F_t'^{(k)}$5 is omitted in Eq. (8), although one can symmetrically define $F_t'^{(k)}$6 if desired. In Section 3.4, the feedback is also rearranged into a single “relation-map difference” update:

$F_t'^{(k)}$7

The two forms are described as equivalent up to re-grouping of the learnable parameters (Shen et al., 11 Sep 2025).

These equations define DFFM as a residual-style feedback update in which differential features are normalized, projected, and then reintroduced into the visible branch through a learned balance between memory and correction. A plausible implication is that the module combines iterative state retention with targeted cross-modal correction rather than performing fusion in a single pass.

4. Architectural composition and dataflow

DFFM is deliberately lightweight. All feature maps have channel dimension $F_t'^{(k)}$8, for example 256 or 512, matching the Transformer. The module itself contains one LayerNorm layer of dimension $F_t'^{(k)}$9, one two-layer MLP with the same input and output feature dimension $k$0 and a nonlinear activation (ReLU) in between, and three per-branch scalars $k$1, each initialized and learned end-to-end (Shen et al., 11 Sep 2025).

No spatial convolutions or additional residual blocks are added. The residual-style update

$k$2

is described as serving both as a highway for the previous iteration’s features and as the injection point for differential guidance. This means that DFFM adds a feedback path without substantially increasing architectural complexity.

In the connectivity diagram described for Fig. 2, the two outputs of MFRM, $k$3 and $k$4, are fed into a difference block. That block subtracts $k$5 from $k$6, applies LN followed by MLP, scales by $k$7, adds a scaled copy of $k$8 by $k$9, and emits $F_{\mathrm{dif}}$0. The newly updated $F_{\mathrm{dif}}$1, and optionally $F_{\mathrm{dif}}$2, then loop back to the next MFRM iteration (Shen et al., 11 Sep 2025).

The dataflow establishes DFFM as the feedback path of the IRDFusion block. This suggests a division of labor in which MFRM performs relation-aware refinement and alignment, while DFFM converts inter-modal discrepancy into an explicit correction signal for subsequent refinement.

5. Iterative update procedure

The paper provides a step-by-step pseudocode description for the iterative interaction between MFRM and DFFM over $F_{\mathrm{dif}}$3 total iterations. At iteration $F_{\mathrm{dif}}$4, the inputs are $F_{\mathrm{dif}}$5, taken from the last DFFM output or the backbone for $F_{\mathrm{dif}}$6, and $F_{\mathrm{dif}}$7, taken from the thermal backbone or the last DFFM if a symmetric form is used (Shen et al., 11 Sep 2025).

The first step is MFRM, which computes $F_{\mathrm{dif}}$8 and $F_{\mathrm{dif}}$9 via Mutual Feature Refinement. The second step is DFFM, which computes the inter-modal difference $F_v'^{(k)}$0, normalizes and projects it with $F_v'^{(k)}$1, and then performs the feedback update $F_v'^{(k)}$2. The thermal stream may optionally be updated similarly or kept fixed. The updated features are then passed into the next MFRM if $F_v'^{(k)}$3, and the final fused feature is taken as $F_v'^{(k)}$4 (Shen et al., 11 Sep 2025).

This iterative scheme is central to how DFFM functions. Rather than producing a terminal fused tensor in one operation, it participates in a recurrent refinement loop. A plausible implication is that the quality of the fused representation depends on repeated alternation between relation-aware refinement and differential feedback, not on differencing alone.

6. Empirical effects and interpretive significance

The focused ablation results reported for Table 4 evaluate “Effects of different modules” using Co-DETR + NiNfusion as the baseline, with $F_v'^{(k)}$5, $F_v'^{(k)}$6, and $F_v'^{(k)}$7 (Shen et al., 11 Sep 2025). Adding only DFFM yields $F_v'^{(k)}$8 $F_v'^{(k)}$9, $F_t'^{(k)}$0 $F_t'^{(k)}$1, and $F_t'^{(k)}$2 $F_t'^{(k)}$3. Adding only MFRM is described as slightly less effective on its own, and the full MFRM + DFFM combination gives the best overall gains: $F_t'^{(k)}$4, $F_t'^{(k)}$5, and $F_t'^{(k)}$6 (Shen et al., 11 Sep 2025).

Configuration	mAP50	mAP75 / mAP
Co-DETR + NiNfusion baseline	84.8	44.0 / 46.9
Baseline + DFFM	87.5	46.0 / 49.2
Baseline + MFRM + DFFM	88.3	48.0 / 50.7

These results are presented as evidence that DFFM is the key “feedback amplifier” in the framework, explicitly removing feature components common to both modalities and boosting strictly differential details. The paper characterizes such details with the example of distinguishing a warm pedestrian outline in infrared from a colorful background in RGB (Shen et al., 11 Sep 2025).

The empirical pattern also clarifies a possible misconception: DFFM is not redundant with MFRM. The ablation indicates that the two modules contribute differently, with DFFM supplying a measurable improvement even in isolation and the combined design delivering the strongest performance. This suggests that relation modeling and differential feedback are complementary rather than interchangeable.

7. Conceptual interpretation and scope

Within IRDFusion, DFFM can be understood as the mechanism that converts cross-modal discrepancy into iterative supervision for feature fusion (Shen et al., 11 Sep 2025). Its defining assumptions are that not all shared information is useful, that common-mode background interference should be suppressed, and that strictly complementary information should be fed back into refinement. The combination of element-wise differencing, LayerNorm, a lightweight two-layer MLP, and residual-style update operationalizes these assumptions without additional spatial convolutions or heavy overhead.

The module’s significance is therefore architectural as well as functional. Architecturally, it provides a lightweight feedback path between successive MFRM stages. Functionally, it embodies the “relation-map difference” principle that gives IRDFusion its name. The broader framework is reported to achieve state-of-the-art performance on the FLIR, LLVIP and M$F_t'^{(k)}$7FD datasets and to consistently outperform existing methods across diverse challenging scenarios, which situates DFFM within a system-level design for robust multispectral object detection (Shen et al., 11 Sep 2025).

A plausible implication is that DFFM is most relevant in settings where complementary information is distributed asymmetrically across modalities and where naive fusion would otherwise preserve irrelevant shared content. In that sense, the module represents a specific formulation of iterative cross-modal contrast and screening rather than a general-purpose fusion primitive.