OCNF: Orthogonal Constraint Normalization Fusion

• OCNF is a feature fusion paradigm that enforces orthogonal projection and quality-aware weighting to preserve modality-specific cues under missing or degraded data.
• It dynamically recalibrates fusion weights using a DMQA network and softmax normalization to suppress unreliable modality features.
• Empirical studies on remote sensing datasets demonstrate improved detection performance and clearer attention maps under challenging conditions.

Orthogonal Constraint Normalization Fusion (OCNF) is a feature fusion paradigm designed for robust multimodal learning, particularly in scenarios where data from one or more modalities may be partially missing or degraded. OCNF has been proposed in the context of optical–SAR (Synthetic Aperture Radar) object detection as a core component of the Quality-Aware Dynamic Fusion Network (QDFNet). Unlike simple summation or concatenation fusion strategies, OCNF incorporates both orthogonality-enforcing projections and quality-aware normalization of fusion weights, preserving modality-specific discrimination and suppressing the influence of unreliable modality features. This dual mechanism addresses the challenges of modality noise entanglement and unreliable channel propagation, which are particularly problematic in remote sensing and other real-world multimodal tasks (Zhao et al., 27 Dec 2025).

1. Motivation and Principles

OCNF is motivated by two key failure modes in multimodal fusion under missing or degraded modalities: (i) projection subspace sharing that allows noise from one stream to corrupt the other, and (ii) fusion schemes that may assign undue weights to poor-quality modality features, leading to error propagation in the fused representation. The OCNF module directly addresses both:

• Orthogonality Constraint: By enforcing orthogonality between the learnable projection matrices for each modality, OCNF decorrelates modal features in the fused space. This ensures that modality-specific discriminative cues are preserved and prevents feature-space collapse due to imposed redundancy.
• Quality-Aware Normalization: Fusion weights are dynamically reweighted via a softmax function applied to reliability scores derived from a Dynamic Modality Quality Assessment (DMQA) network. As a result, OCNF down-weights channels with low estimated reliability, ensuring robustness against unreliable features (Zhao et al., 27 Dec 2025).

2. Architectural Design and Computational Flow

OCNF is implemented as a plug-in fusion block, operating on modality feature maps and their associated reliability scores. For optical–SAR fusion:

• Given feature maps $F_o, F_s \in \mathbb{R}^{C \times H \times W}$ (for optical and SAR, respectively) and reliability scores $R_o, R_s \in \mathbb{R}^{1 \times H \times W}$ from the DMQA module, OCNF proceeds as follows:

1. Orthogonal Projection: Two learnable matrices, $W_o, W_s \in \mathbb{R}^{d \times C}$, project the optical and SAR features into a $d$-dimensional space. Orthogonality and norm constraints are imposed: $W_o^T W_s = 0$, $\|W_o\|_F = \|W_s\|_F = 1$.
2. Reliability Mapping: Reliability maps $R_o$, $R_s$ are aggregated and passed through a shared MLP to produce channel-wise reliability vectors $\tilde{R}_o, \tilde{R}_s \in \mathbb{R}^{d}$.
3. Softmax Weighting: For each channel $c$, fusion weights are computed:

$$\gamma_o^{(c)} = \frac{\exp(\tilde{R}_o^{(c)})}{\exp(\tilde{R}_o^{(c)}) + \exp(\tilde{R}_s^{(c)})}, \quad \gamma_s^{(c)} = \frac{\exp(\tilde{R}_s^{(c)})}{\exp(\tilde{R}_o^{(c)}) + \exp(\tilde{R}_s^{(c)})}$$

1. Fused Output: The fused feature is obtained as

$$F_{fused} = \gamma_o \odot F'_o + \gamma_s \odot F'_s$$

where $F'_o = W_o \cdot \mathrm{vec}(F_o)$, $F'_s = W_s \cdot \mathrm{vec}(F_s)$ and $\odot$ denotes broadcasted elementwise multiplication.

The result is reshaped back to $d \times H \times W$ and forwarded to the detection head (Zhao et al., 27 Dec 2025).

3. Mathematical Constraints and Loss Formulation

The orthogonality and normalization of the projection matrices are jointly enforced to maintain decorrelated and norm-equalized projections:

• Orthogonality: $W_o^T W_s = 0$
• Frobenius normalization: $\|W_o\|_F = 1$, $\|W_s\|_F = 1$

A soft penalty is added to the total loss:

$$L_{orth} = \|W_o^T W_s\|_F^2 + \lambda \left( \|W_o\|_F^2 - 1 \right)^2 + \lambda \left( \|W_s\|_F^2 - 1 \right)^2$$

The overall detection objective thus includes the standard loss terms for bounding box, classification, and objectness, plus the orthogonality penalty:

$$L_{total} = \lambda_1 L_{box} + \lambda_2 L_{cls} + \lambda_3 L_{obj} + \mu L_{orth}$$

Fusion weights are normalized by a channel-wise softmax, ensuring that the relative reliability of each modality is respected per channel.

4. Training Strategy and Implementation

During training, OCNF projection matrices are updated by backpropagation. Orthogonality is maintained either through explicit weight normalization based on SVD—centralizing, performing SVD, and renormalizing columns of $[W_o; W_s]$—or by including $L_{orth}$ in the loss with coefficient $\mu$.

The typical training schedule fixes the INN module for the initial epochs, then proceeds to jointly train the backbone, DMQA, OCNF, and detection head for 120 epochs with a batch size of 16. Standard learning rate schedules are used (Zhao et al., 27 Dec 2025).

5. Empirical Performance and Analysis

Ablative results on the SpaceNet6-OTD-Fog dataset (Zero-filling corruption) are as follows:

Configuration	mAP50	mAP
Baseline	80.2	42.7
+ DMQA only	81.3	43.5
+ OCNF only	80.9	43.3
Full QDFNet	82.6	44.9

Under severe missing conditions (MR = 0.3, Zero-filling):

Configuration	mAP50	mAP
Baseline	73.8	37.5
+ OCNF only	74.5	38.1

Similar improvements are observed on OGSOD-2.0 (Table 2 of (Zhao et al., 27 Dec 2025)). Grad-CAM analyses reveal that OCNF yields sharper and less entangled attention maps compared to baseline fusion, especially in absent-modality regions. This suggests that OCNF effectively isolates high-reliability signals and suppresses noise propagation under missing modality scenarios.

6. Comparative Context and Related Fusion Methodologies

Orthogonality constraints in multimodal fusion have been used for inter-class regularization and cross-modal discrimination in prior works, such as Fusion and Orthogonal Projection (FOP) for face-voice association (Saeed et al., 2021). While FOP similarly projects modalities into a shared space with orthogonality supervision and combines embeddings via gated fusion, OCNF is distinguished by its use of dynamically computed channel-wise reliability for quality-guided normalizing of fusion weights at the feature map level, as well as its explicit application to missing data regimes in remote sensing.

A plausible implication is that the combination of orthogonality and reliability-guided weighting is broadly effective for maintaining discriminative power while suppressing unreliable features in various cross-modal and multimodal tasks.