Bidirectional Cross Fusion Block

Updated 10 July 2026

Bidirectional Cross Fusion Block is a reciprocal fusion schema that exchanges complementary information between two feature streams to preserve distinct spatial and semantic cues.
It employs dual pathways with cross-attention, adaptive gating, and target-aware positional encoding to refine shallow and deep features for classification and regression tasks.
Empirical studies in UAV tracking, medical segmentation, and dual-view analysis demonstrate its capacity to boost performance while reducing computational complexity.

“Bidirectional Cross Fusion Block” is best understood as an Editor’s term for a class of reciprocal fusion modules that exchange information in both directions between two feature streams rather than collapsing them immediately into a single shared representation. In its most explicit form, the term denotes the core fusion unit in the target-aware Bidirectional Fusion Transformer for UAV tracking, where shallow and deep correlation features are first jointly modeled and then fused along two directed paths—shallow→deep and deep→shallow—to produce task-specific outputs for classification and regression (Sun et al., 13 Mar 2025). Closely related constructions appear in unpaired CT/MRI segmentation, dual-view mammography, RGB-T tracking, audio-visual navigation, reversible multiscale pyramids, and web security models, but the underlying principle remains the same: bidirectionality is used to preserve complementary structure while allowing reciprocal conditioning across streams (V et al., 25 Dec 2025, Pirsoltan et al., 7 Jul 2026, Zeng et al., 2024, Wang et al., 11 Jan 2026, Chiley et al., 2022, Tian et al., 24 Jun 2025).

1. Conceptual definition and motivation

In the UAV-tracking formulation, the block is introduced to address a specific deficiency of single-stage fusion. Aerial tracking must cope with strong appearance changes, motion blur, occlusion, and clutter. Shallow features carry spatial locality and edges that are critical for precise localization, whereas deep features carry global semantics and invariances that are critical for discrimination. Existing multi-stage fusion trackers often reduce these heterogeneous cues to one fused representation shared by both classification and regression, which the paper characterizes as suboptimal because the shallow branch should be adjusted by semantics for localization, while the deep branch should be enriched by local details for recognition (Sun et al., 13 Mar 2025).

The resulting design principle is not limited to UAV tracking. Across the literature, reciprocal fusion is repeatedly used when two streams encode different but complementary statistics: CT and MRI boundary-versus-soft-tissue cues in A-QCF-Net, CC and MLO mammographic views in dual-view large-vision adaptation, or audio and visual embeddings in CRFN (V et al., 25 Dec 2025, Pirsoltan et al., 7 Jul 2026, Wang et al., 11 Jan 2026). This suggests a broader interpretation: bidirectional cross fusion is most useful when one stream should inform the other without destroying stream-specific structure.

A further implication, made explicit in several papers, is that bidirectionality need not imply symmetry of purpose. In BFTrans, the two directions are decoupled by downstream head assignment; in SAVA-X, Exo→Ego and Ego→Exo are both computed, yet the task is asymmetric and Exo→Ego conditioning is reported as more decisive for ego-side error detection (Sun et al., 13 Mar 2025, Li et al., 13 Mar 2026).

2. Canonical transformer formulation in BFTrans

In BFTrans, the block operates after backbone extraction with MobileViT-v2 and stage-wise pixel-wise cross-correlation. The two inputs are stage-3 and stage-4 correlation maps,

$M_k = C_k(\phi_k(z) \odot \phi_k(x)), \quad k=3,4,$

where $z$ is the template, $x$ is the search region, $\odot$ denotes pixel-wise cross-correlation, and each $C_k$ is a $1\times 1$ convolution mapping channels to $d=192$ . After backbone modifications, $M_3,M_4 \in \mathbb{R}^{H\times W\times d}$ share identical spatial size. They are fine-tuned by per-stream $1\times 1$ convolutions, concatenated into $M_c \in \mathbb{R}^{H\times W\times 2d}$ , processed by linear self-attention, then split back into $z$ 0 and $z$ 1 (Sun et al., 13 Mar 2025).

The core bidirectional operations are

$z$ 2

$z$ 3

$z$ 4

The forward path, shallow→deep, uses $z$ 5 from the shallow stream to modulate deep values $z$ 6, yielding $z$ 7 for the classification head. The backward path, deep→shallow, uses $z$ 8 from the deep stream to modulate shallow values $z$ 9, yielding $x$ 0 for two regression heads predicting center offsets and sizes. The two directions are therefore not merged; they remain task-specific throughout.

BFTrans further augments cross-attention with target-aware positional encoding (TAPE), applied to the keys. For a feature map $x$ 1,

$x$ 2

$x$ 3

$x$ 4

Here $x$ 5 and $x$ 6 are channel and spatial positional encodings, $x$ 7 is sigmoid, and $x$ 8 is a learnable scalar. In use, $x$ 9 in the forward fusion and $\odot$ 0 in the backward fusion. The stated purpose is to make positional information target-aware rather than input-agnostic, guiding attention to target-relevant locations and scales.

3. Architectural variants across domains

The same reciprocal-fusion idea appears under different module names and with markedly different mechanics. The table summarizes representative instantiations.

Paper	Paired streams	Bidirectional operation
"Target-aware Bidirectional Fusion Transformer" (Sun et al., 13 Mar 2025)	Shallow and deep UAV correlation maps	Linear cross-attention in both directions; separate outputs for classification and regression
"A-QCF-Net" (V et al., 25 Dec 2025)	CT and MRI encoder features	Quaternion Q/K/V projection, channel-wise attention, adaptive scalar gate
"Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification" (Pirsoltan et al., 7 Jul 2026)	CC and MLO token sequences	Cross-attention each way; mean-pooled fusion tokens are reinserted
"Residual Cross-Modal Fusion Networks for Audio-Visual Navigation" (Wang et al., 11 Jan 2026)	Audio and visual embeddings	Shared interaction vector with scalar-gated residual updates
"RevBiFPN" (Chiley et al., 2022)	Multi-resolution feature maps	Top-down and bottom-up reversible additive coupling
"SAVA-X" (Li et al., 13 Mar 2026)	Ego and exo video sequences	Parallel cross-attention with learnable gated residual mixing

These variants show that a bidirectional cross fusion block is not tied to a single computational primitive. In A-QCF-Net, the exchange is channel-wise rather than token-wise: MRI→CT and CT→MRI paths are computed symmetrically with quaternion $\odot$ 1 convolutions for $\odot$ 2, $\odot$ 3, and $\odot$ 4, channel-wise softmax, and a per-sample adaptive gate $\odot$ 5 derived from global statistics of the original stream and the raw cross-context (V et al., 25 Dec 2025). In CRFN, the mechanism is even simpler: a shared interaction vector

$\odot$ 6

is injected into both modalities through LayerNorm, scalar coefficients $\odot$ 7, and $\odot$ 8, without any cross-attention at all (Wang et al., 11 Jan 2026).

Other works reinterpret bidirectionality through carriers rather than direct residual mixing. In the mammography framework, bidirectional cross-attention outputs are mean-pooled into one fusion token per view, and these tokens are prepended back to the original sequences; the original view-specific tokens remain untouched at the fusion step (Pirsoltan et al., 7 Jul 2026). In CADNIF, the reciprocal interaction is implemented by two spatial attention maps $\odot$ 9 and $C_k$ 0 conditioned on the image pair, so the block remains CNN-based and unsupervised rather than transformer-based (Shen et al., 2021).

A distinct line is multiscale bidirectional fusion. RevBiFPN’s RevSilo performs top-down and bottom-up additive coupling across resolution levels and is exactly reversible, so the notion of bidirectionality is spatial-scale exchange rather than modality exchange (Chiley et al., 2022). BPNet’s bidirectional projection module performs 2D→3D back-projection and 3D→2D projection at multiple decoder levels, again preserving the reciprocal pattern while replacing attention with geometry-driven gather/scatter operations (Hu et al., 2021).

4. Core design axes

One major axis of variation is the choice of interaction carrier. BFTrans uses the paired streams themselves: one stream supplies queries and keys, the other supplies values. A-QCF-Net likewise uses the other stream’s projected values, but interposes a learned scalar gate $C_k$ 1 and quaternion convolutional output fusion (Sun et al., 13 Mar 2025, V et al., 25 Dec 2025). By contrast, the dual-view mammography model does not directly overwrite either stream; it summarizes cross-view evidence into fusion tokens, which are later refined by frozen transformer layers (Pirsoltan et al., 7 Jul 2026). This separation between shared and private information is presented there as a deliberate alternative to residual additive fusion.

A second axis is how positional or structural compatibility is imposed. BFTrans uses TAPE to modulate cross-attention keys with channel- and spatial-weighted target-aware positional bias (Sun et al., 13 Mar 2025). SAVA-X injects scene-adaptive view embeddings before bidirectional cross-attention and then uses learnable gates

$C_k$ 2

before averaging the two view-wise outputs (Li et al., 13 Mar 2026). Ovi instead uses modality-specific scaled RoPE so that video tokens and audio tokens, which have different temporal granularities, share aligned rotary phase during bidirectional cross-attention inside every DiT block (Low et al., 30 Sep 2025).

A third axis is whether the two directions remain separate or are recombined. BFTrans keeps $C_k$ 3 and $C_k$ 4 disjoint and routes them to different heads by design (Sun et al., 13 Mar 2025). SAVA-X averages view-wise fused outputs into a single sequence for the deformable detector (Li et al., 13 Mar 2026). CMDFusion performs both directions during training—3D-to-2D and 2D-to-3D—but at inference only the 2D-to-3D direction is retained because final predictions come from the 3D LiDAR branch and the camera stream is unavailable (Cen et al., 2023).

A fourth axis is the meaning of bidirectionality itself. In some cases it denotes stream reciprocity, as in BFTrans, A-QCF-Net, CRFN, and DCAT (Sun et al., 13 Mar 2025, V et al., 25 Dec 2025, Wang et al., 11 Jan 2026, Borah et al., 14 Mar 2025). In others it denotes reciprocal transfer across views or timelines, as in SAVA-X and RGB-T CFBT (Li et al., 13 Mar 2026, Zeng et al., 2024). In RevBiFPN it denotes top-down and bottom-up flow across scale levels, not across modalities (Chiley et al., 2022). The common structure is still reciprocal conditioning, but the objects being conditioned differ.

5. Optimization, complexity, and empirical behavior

In BFTrans, linear self-attention and linear cross-attention reduce standard dot-product complexity from $C_k$ 5 to $C_k$ 6, with $C_k$ 7. Together with compact $C_k$ 8, $C_k$ 9 fine-tuning convolutions, and removal of the last MobileViT-v2 block, this supports real-time deployment at 30.5 FPS on NVIDIA Jetson AGX Xavier. On UAV123, the ablation sequence is explicit: baseline without fusion or TAPE reaches Success 0.596 and Precision 0.799; forward-only 0.613/0.812; backward-only 0.610/0.807; bidirectional 0.634/0.831; bidirectional plus TAPE 0.647/0.847. Training uses AdamW, weight decay $1\times 1$ 0, 300 epochs, batch size 128, 60k pairs per epoch, and the loss

$1\times 1$ 1

with $1\times 1$ 2 and $1\times 1$ 3 (Sun et al., 13 Mar 2025).

A-QCF-Net demonstrates that bidirectional cross fusion can be effective even when data are completely unpaired. Its quaternion parameterization ties sub-kernels through the Hamilton product and reduces parameters by $1\times 1$ 4 relative to the corresponding real-valued convolution. The full jointly trained model reaches mean DSC $1\times 1$ 5; removing cross-fusion drops this to $1\times 1$ 6, removing the adaptive gate to $1\times 1$ 7, and removing quaternions to $1\times 1$ 8. The final model has 31.5M trainable parameters and achieves Tumor Dice scores of 76.7% on CT and 78.3% on MRI, exceeding the strong unimodal nnU-Net baseline by 5.4% and 4.7%, respectively (V et al., 25 Dec 2025).

The dual-view mammography study emphasizes fusion depth rather than parameter efficiency. On VinDr-Mammo binary classification, one fusion block at layer 18 yields AUC 0.8441, two blocks at layers 12 and 23 yield the best AUC 0.8593, and four blocks at 12, 18, 23, 26 reduce AUC to 0.8313. Mean pooling is the best fusion-token aggregation strategy, outperforming max pooling and attention pooling, and token-based fusion outperforms residual additive fusion on both CMMD and VinDr-5 (Pirsoltan et al., 7 Jul 2026). This is one of the clearest demonstrations that more bidirectional interaction is not automatically better.

Other papers emphasize stability or memory. CRFN reports that initializing the residual scaling factors $1\times 1$ 9 to 0.2 gives the best overall balance on Replica and Matterport3D, and its Fusion Control Module materially improves SPL and SR over a variant without it (Wang et al., 11 Jan 2026). RevBiFPN shows that reversible bidirectional fusion can reduce training activation memory from $d=192$ 0 to $d=192$ 1; RevBiFPN-S6 is reported as using up to 19.8x lesser training memory for image classification, and on COCO the family provides up to a 2.5% boost in AP over HRNet with a ~2.4x reduction in training-time memory (Chiley et al., 2022).

6. Applications, limitations, and recurrent misconceptions

The application range is broad. Bidirectional cross fusion appears in aerial object tracking, unpaired multimodal tumor segmentation, dual-view mammography classification, RGB-T tracking, audio-visual navigation, RGB–LiDAR segmentation, referring remote sensing segmentation, road segmentation, unaligned medical image fusion, and malicious URL detection (Sun et al., 13 Mar 2025, V et al., 25 Dec 2025, Pirsoltan et al., 7 Jul 2026, Zeng et al., 2024, Wang et al., 11 Jan 2026, Cen et al., 2023, Li et al., 1 Jan 2025, Li et al., 2020, Li et al., 2024, Tian et al., 24 Jun 2025). The recurring motivation is always some version of complementarity: shallow versus deep, geometry versus texture, local hand–object detail versus global scene structure, or lexical URL artifacts versus sparse DOM subgraphs.

A common misconception is that bidirectional cross fusion is synonymous with transformer cross-attention. That is not supported by the literature. CADNIF uses CNN-generated spatial attention maps in both directions (Shen et al., 2021). CRFN uses a shared interaction vector with scalar-gated residual updates (Wang et al., 11 Jan 2026). RevSilo uses invertible additive coupling across scales (Chiley et al., 2022). BPNet uses calibrated projection and back-projection between 2D and 3D representations (Hu et al., 2021). The concept is therefore architectural rather than operator-specific.

A second misconception is that both directions are always equally useful or equally weighted. SAVA-X reports unidirectional ablations in which Exo→Ego reaches Mean 20.73 and tIoU 58.71, whereas Ego→Exo reaches Mean 19.48 and tIoU 58.51, indicating a directional asymmetry tied to the task definition (Li et al., 13 Mar 2026). CMDFusion also treats the two directions differently between training and inference, retaining only 2D-to-3D fusion at test time (Cen et al., 2023). Bidirectionality thus often serves representational completeness during training even when deployment is asymmetric.

A third misconception is that reciprocal fusion requires paired, spatially aligned inputs. A-QCF-Net is trained on completely separate and unpaired CT and MRI cohorts, and its block exchanges abstract modality-complementary features through channel-wise attention and learned gates rather than spatial correspondence (V et al., 25 Dec 2025). SAVA-X handles asynchronous, length-mismatched ego and exo videos with cross-attention after adaptive sampling and view embedding injection (Li et al., 13 Mar 2026). What is required is not always alignment, but a mechanism for constrained reciprocal transfer.

Limitations are likewise recurrent. BFTrans abstracts linear attention without specifying the kernel $d=192$ 2, number of heads, or normalization details, and notes that two fusion paths add modest overhead relative to single fusion (Sun et al., 13 Mar 2025). A-QCF-Net observes potential failure cases near major vessels in CT and spurious MRI segments in heterogeneous parenchyma, and identifies modality imbalance, domain shift, and noisy labels as persistent concerns (V et al., 25 Dec 2025). The mammography study shows that overly dense late-stage fusion can degrade performance (Pirsoltan et al., 7 Jul 2026). RevBiFPN obtains large memory savings, but reversible recomputation introduces measured slowdowns from 12.73% to 25.02% depending on scale (Chiley et al., 2022).

Taken together, these works define the Bidirectional Cross Fusion Block not as a single module but as a recurring design schema: two streams interact in both directions, the exchange is usually constrained by task structure, and the outputs are either kept distinct or recombined only after reciprocal conditioning has occurred. The canonical BFTrans formulation is one especially clear instance, but the broader literature shows that the same principle can be realized through linear attention, quaternion operators, token carriers, residual controllers, reversible couplings, geometric projections, or alignment-aware warping (Sun et al., 13 Mar 2025, V et al., 25 Dec 2025, Pirsoltan et al., 7 Jul 2026, Chiley et al., 2022, Li et al., 2024).