Fusion in Your Way: Aligning Image Fusion with Heterogeneous Demands via Direct Preference Optimization

Abstract: As a key technique in multi-modal processing, infrared and visible image fusion (IVIF) plays a crucial role in integrating complementary spectral information for visual enhancement and downstream vision tasks. Despite remarkable progress, existing methods struggle to flexibly accommodate heterogeneous demands. Achieving adaptive fusion that aligns with various preferences from both human and machine vision remains an open and challenging problem. To address this challenge, we propose DPOFusion, a direct preference optimization (DPO) framework integrating the property-aligned latent diffusion model (PALDM) and the preference-controllable latent diffusion model (PCLDM), enabling task-guided, preference-adaptive IVIF for both human and machine vision. The PALDM leverages a latent fusion prior and a joint conditional loss to generate diverse candidate fusion results with various properties. PCLDM is subsequently fine-tuned via instance direct preference optimization (IDPO), enabling direct control of the final fusion results with heterogeneous preference signals. Experimental results demonstrate that our framework not only attains precise preference alignment among humans, vision-LLMs, and task-driven networks, but also sets a new benchmark for adaptive fusion quality and task-oriented transferability.

• The paper introduces DPOFusion, a two-stage latent-diffusion framework that generates diverse infrared-visible fusion candidates and aligns them with human, VLM, detection, or segmentation preferences using instance-level DPO.
• The method achieves leading fusion quality in several settings, including LLVIP RLHF scores of 7.725 EN, 61.911 SD, 5.977 AG, and 57.295 MUSIQ, while improving MSRS segmentation mIoU to 55.96 and M3FD detection mAP to 43.40.
• The proposed masked preference updates preserve non-target regions and outperform vanilla DPO, but the 778.4M-parameter model remains computationally expensive and requires substantial preference annotation.

Overview

DPOFusion addresses a persistent limitation of infrared–visible image fusion (IVIF) methods: their inability to adapt a single model to heterogeneous, often conflicting demands from human observers, vision-LLMs (VLMs), and downstream task networks. Existing preference-aware fusion approaches typically require a separate model and training pipeline per demand type. The authors instead import direct preference optimization (DPO), originally developed for LLM alignment, into IVIF, constructing a two-stage framework composed of a property-aligned latent diffusion model (PALDM) that generates diverse candidate fusions, and a preference-controlled latent diffusion model (PCLDM) fine-tuned via a novel instance-level DPO objective. The framework supports four alignment modes: human feedback (RLHF), VLM feedback (RLVF, using Qwen3-Omni-Think as rater), detection-driven feedback (RLDF-OD, YOLOv11), and segmentation-driven feedback (RLDF-Seg, SegFormer).

Motivation and problem formulation

The paper identifies two obstacles to applying DPO to unsupervised fusion. First, without ground-truth supervision, the solution space of valid fused outputs is vast, and most candidates lack perceptual fidelity or semantic accuracy; preference adaptation therefore needs a controlled pool of high-quality candidates to annotate. Second, localized preference adaptation tends to perturb shared network parameters, degrading unrelated regions — so the method must balance local adaptability against global consistency.

The framework proceeds in three stages: PALDM generates multiple high-quality candidates per scene; a data collection module annotates region-specific or global preferences from humans, task models, or VLMs; and PCLDM is fine-tuned on these preference pairs using instance DPO (IDPO).

Methodology

Prior latent fusion model and PALDM. A Restormer-based network $\epsilon_\text{lfm}$ operates on concatenated VAE latents of the infrared and visible sources, trained with an intensity-plus-Sobel-gradient max loss. PALDM ($\epsilon_\text{ref}$) is then trained as a conditional denoiser over the fused latent, conditioned on source latents, CLIP ViT-L/14 text embeddings, and timestep. To create property diversity, the authors introduce a joint conditional loss: a discrete interpolation level $k$ yields a blended target latent mixing infrared, visible, and prior-fused latents, denoised under a corresponding property prompt alongside the standard fusion target. Ablations show this joint loss outperforms single- and multi-objective variants across EN, SD, AG, MUSIQ, and CNNIQA.

Preference data collection. Two collection modes are defined. Region-specific collection provides a mask $I_m$ plus preferred/rejected pairs, used for human annotation (with SAM-assisted mask extraction via a custom UI) and segmentation-driven preferences (mask = intersection of predicted masks from preferred, rejected, baseline, and ground truth). Global collection, used for VLM ranking and detection, applies a filtering pipeline and sets $I_m$ to cover extracted patches. Preference labels for tasks are derived from weighted combinations of mIoU/accuracy (segmentation) and mAP/accuracy (detection).

PCLDM and IDPO. PCLDM is initialized as a copy of the frozen PALDM, connected through zero-initialized $1{\times}1$ convolutions in ControlNet fashion, taking a preference-aligned prompt while the frozen branch takes the general prompt. The IDPO loss restricts the DPO-style preference terms $\mathcal{P}_w$, $\mathcal{P}_l$ to the masked region via element-wise masking of noise-prediction residuals, while adding pixel-level consistency penalties $\mathcal{O}_w$, $\mathcal{O}_l$ outside the mask against the frozen reference:

$\epsilon_\text{ref}$0

This design directly addresses the local-adaptation/global-consistency trade-off identified in the motivation.

Experimental results

Training uses LLVIP (256×256 crops); preference datasets are built from 100 images each of LLVIP, MSRS, and RoadScene for RLHF/RLVF, and 540 M3FD plus 1083 MSRS images for RLDF. Evaluation covers five general metrics (EN, SD, AG, MUSIQ, CNNIQA) plus five supplementary ones (VIF, SCD, SF, DF, VIFF), against nine SOTA methods including U2Fusion, DDFM, SHIP, EMMA, Text-IF, DCEvo, GIFNet, LUT-Fuse, and SAGE.

Key quantitative findings:

• General fusion quality: RLHF attains the best EN (7.725), SD (61.911), AG (5.977), and MUSIQ (57.295) on LLVIP; RLVF leads on MSRS (EN 7.203, SD 56.614, AG 5.782) and RoadScene (EN 7.574, AG 8.622). Supplementary metrics show RLVF achieving best SF, DF, and VIFF on MSRS and RoadScene.
• Downstream transfer: RLDF-Seg improves mIoU by 0.5% over the strongest baseline (55.96 vs. 55.66 for DCEvo) on MSRS, with notable gains on small classes such as traffic cones (33.09 vs. 29.29). RLDF-OD yields a 4.2% mAP improvement on M3FD (@.5:.95 of 43.40 vs. 42.89 for DCEvo).
• Ablations: IDPO outperforms both vanilla DPO and contrastive losses on both quality metrics and segmentation mIoU/mAcc (55.96/64.26 vs. 55.20/62.52 for DPO). CLIP-I/DINO similarity measurements confirm tighter alignment with ground-truth preferred images under IDPO.
• Efficiency: operating in latent space, DPOFusion infers at ~1.71 s per 256×256 image versus ~7.77 s for DDFM, though it remains far heavier than lightweight CNN methods (e.g., LUT-Fuse at 2.47 ms).

The authors also report sensitivity analyses: detection accuracy increases monotonically with $\epsilon_\text{ref}$1 up to 500 when full patches serve as preference regions, whereas $\epsilon_\text{ref}$2, $\epsilon_\text{ref}$3 performs best for RLHF. A blind study with 10 annotators rating 20 images on a 1–5 Likert scale favors RLHF for thermal-target visibility and texture detail.

Limitations and open questions

Several constraints are acknowledged or evident. RLHF underperforms on RoadScene, which the authors attribute to human evaluators prioritizing holistic coherence over local detail in complex scenes — indicating that global-preference signals may conflict with metric-based quality measures. The computational cost (778.4M parameters, 2063 GFLOPs) precludes real-time deployment; the proposed remedy, distilling preference-aligned pseudo-labels into lightweight students, is asserted rather than demonstrated. Preference annotation remains labor-intensive: human masks require SAM-assisted interactive annotation, and task-driven labels depend on fixed evaluator networks (YOLOv11, SegFormer), raising the question of whether alignment transfers to other detectors or segmenters. The choice of $\epsilon_\text{ref}$4 interpolation levels constrains the candidate distribution, and the effect of this discretization on preference coverage is not systematically explored. Finally, whether a single PCLDM can serve multiple simultaneous preference types, rather than one fine-tuned model per prompt, remains unaddressed.

Conclusion

DPOFusion demonstrates that DPO-style preference optimization can be adapted to unsupervised multimodal image fusion through two technical contributions: a joint conditional loss that produces prompt-controllable candidate diversity in a latent diffusion prior, and instance-level DPO that confines preference updates to masked regions while enforcing consistency elsewhere. The resulting framework achieves state-of-the-art fusion quality and measurable gains in detection and segmentation when aligned with task-specific feedback, establishing a unified alternative to per-demand specialized fusion models. Its practical adoption hinges on reducing annotation cost and inference overhead, questions the paper leaves open.