CHMFFN: Cross-Hierarchical Multi-Feature Fusion

Updated 12 July 2026

CHMFFN is a design pattern that fuses features across different network depths and scales using repeated, structured operations rather than simple concatenation.
It improves task-specific performance by adaptively integrating complementary representations—such as local/global or shallow/deep features—in applications like deraining and medical imaging.
The model employs mechanisms like residual projection, cross-level attention, and graph-domain projection to maintain informative, hierarchy-aware feature interactions.

Searching arXiv for the papers on arXiv and closely related work on hierarchical multi-feature fusion to ground the article with citations. Cross-Hierarchical Multi-Feature Fusion Network (CHMFFN) can be understood as an architectural class in which feature interaction is performed across hierarchy levels, branches, scales, or modalities, and in which fusion is treated as a repeated, structured operation rather than a single late-stage merge. In the supplied literature, the exact term is often absent, but several methods are described as mapping naturally onto it: MH2F-Net for deraining, DMF for multimodal joint representation, HiFuse for medical image classification, FINE for pre-fusion semantic alignment, and related models in vessel segmentation, fine-grained recognition, multimodal micrograph learning, deepfake detection, super-resolution, hyperspectral–SAR classification, industrial anomaly detection, AIGC image assessment, multi-exposure fusion, and speech enhancement (Chen et al., 2021, Hu et al., 2018, Huo et al., 2022, Lee et al., 12 Jun 2026, Zhao et al., 2023, Gao et al., 18 Apr 2025, Srinivas et al., 2024, Mehta et al., 10 Jan 2025, Lyn, 2020, Luo et al., 2024, Xu et al., 21 Apr 2026, Meng et al., 23 Jul 2025, Mu et al., 2024, Mattursun et al., 2024).

1. Conceptual scope

Within this literature, “cross-hierarchical” refers to interactions among features drawn from different depths, scales, or stages rather than reliance on the final backbone output alone. In MH2F-Net, this is explicit in the use of outputs from multiple MHEBs inside the HADB, where information flows across feature depth and stage level; the final fusion then combines shallow/original features $L_o$ , deep/extracted features $L_e$ , and hierarchical/distilled features $L_d$ through RPFF rather than direct summation or concatenation (Chen et al., 2021). In DMF, the same idea appears as a stacked shared pathway $s_1,\dots,s_L$ , where each shared layer depends on current unimodal features and the previous shared layer, so higher-level fusion is conditioned on lower-level fusion (Hu et al., 2018).

“Multi-feature” denotes the coexistence of complementary representations that are not reducible to a single feature family. Depending on the task, these may be shallow/deep, local/global, private/shared, graph/grid, token/map, prompt/visual, or SSL/spectrogram features. HiFuse formalizes this as a three-branch hierarchy of local CNN features, global Swin-style features, and an HFF fusion branch (Huo et al., 2022). CHBC formalizes it as hierarchy-specific attention masks, hierarchy-specific features, and an all-level pooled representation for finest-grained prediction (Gao et al., 18 Apr 2025).

“Fusion” in this context is not synonymous with concatenation. The supplied works repeatedly distinguish simple addition or concatenation from more structured mechanisms such as residual projection, attentive distillation, orthogonal decomposition, alignment-aware modulation, graph-domain projection, query-based cross-attention, or residual multi-layer accumulation. This suggests that CHMFFN is best treated as a design pattern centered on selective, hierarchy-aware integration rather than on any single operator (Lee et al., 12 Jun 2026).

2. Recurrent architectural templates

Several recurring templates appear across the literature.

Model	Domain	CHMFFN-relevant structure
MH2F-Net (Chen et al., 2021)	Single image deraining	MHEB + HADB + RPFF
DMF (Hu et al., 2018)	Multimodal learning	Multiple shared layers linked by $s_{l-1}$
HiFuse (Huo et al., 2022)	Medical image classification	Local branch + global branch + HFF branch
FINE (Lee et al., 12 Jun 2026)	Detection necks	AATS + cross-level attention + residual modulation
Graph U-Net fusion (Zhao et al., 2023)	3D vessel segmentation	CNN–graph matched-scale exchange
MGLF/MPEF (Meng et al., 23 Jul 2025)	AIGC IQA	Per-level query fusion + joint aggregation

One template is hierarchical residual aggregation. DMF and FPFNet both fuse a sequence of hidden states across depth with additive accumulation and normalization, although DMF does so through explicit shared layers and FPFNet through parameter-free multi-layer fusion in encoder and decoder (Hu et al., 2018, Xu et al., 21 Apr 2026). A second template is cross-branch complementary fusion, as in HiFuse, HFMF, and BHF-MEF, where local/global or original/enhanced streams remain distinct long enough for branch-specific modeling before they are adaptively combined (Huo et al., 2022, Mehta et al., 10 Jan 2025, Mu et al., 2024). A third template is pre-fusion alignment or distillation, where feature incompatibility is corrected before merging. FINE aligns low-level and high-level detection features before ordinary fusion, and MH2F-Net uses HADB to recalibrate hierarchical features before RPFF (Lee et al., 12 Jun 2026, Chen et al., 2021).

A further template is heterogeneous-domain projection. In vessel segmentation, CNN features are mapped into graph space by $f(\cdot)$ , propagated by graph convolution $\Omega(\cdot)$ , and mapped back by $g(\cdot)$ at matched encoder and decoder levels (Zhao et al., 2023). In multimodal micrograph learning, HNF rebuilds both token and graph representations at multiple patch resolutions and fuses their global summaries at each level before passing fused information upward (Srinivas et al., 2024). In speech enhancement, MSCFF fuses weighted-sum SSL embeddings and spectrogram features through a gated multi-branch module before RHMA refinement (Mattursun et al., 2024).

3. Fusion operators and formal mechanisms

A central formal pattern is progressive hierarchical accumulation. DMF writes the shared-layer update as

$s_l = f\left( W_l^{x \to s} h_l^x + W_l^{y \to s} h_l^y + W_{l-1}^{s} s_{l-1} \right),$

which makes higher-level multimodal fusion depend on both current private features and previous shared fusion (Hu et al., 2018). This differs from single-bottleneck fusion because the shared stream itself becomes hierarchical.

Another pattern is residual discrepancy-guided fusion. In MH2F-Net, RPFF first forms a discrepancy term $R_{ed}=L_e-L_d$ , then projects it back into the extracted feature,

$L_e$ 0

and finally performs a back-projection-like correction relative to the original feature,

$L_e$ 1

The stated purpose is to emphasize “the features existing in one source but lacking in the others,” rather than only their shared content (Chen et al., 2021).

A third pattern is alignment-aware modulation before fusion. FINE first downsamples adjacent pyramid levels asymmetrically,

$L_e$ 2

then applies cross-level attention using low-level queries and high-level keys/values, reshapes the result into a spatial-channel modulation map $L_e$ 3, and refines the low-level feature by residual modulation,

$L_e$ 4

This is explicitly framed as interaction before fusion rather than direct addition or concatenation (Lee et al., 12 Jun 2026).

A fourth pattern is decomposition before enhancement. CHBC uses matrix orthogonal decomposition

$L_e$ 5

then constructs enhanced features and masks through residual addition. The ablation shows that this “MatOrth” strategy is stronger than simple residual addition for fine-grained classification (Gao et al., 18 Apr 2025).

HiFuse provides a representative three-input hierarchical fusion block. At stage $L_e$ 6, channel attention refines the global feature $L_e$ 7, spatial attention refines the local feature $L_e$ 8, the previous fused state is projected and downsampled to $L_e$ 9, and the final fused feature is

$L_d$ 0

with

$L_d$ 1

The fusion stream therefore has its own recursion $L_d$ 2 rather than behaving as a one-off merge (Huo et al., 2022).

4. Representative instantiations and empirical behavior

In image restoration, CHMFFN-like designs are used to preserve both fine structure and global context under difficult degradation. MH2F-Net addresses single image deraining with a stacked hourglass extractor, hierarchical attentive distillation, and RPFF, and reports $L_d$ 3 on Rain200L, $L_d$ 4 on Rain200H, $L_d$ 5 on Rain1400, and $L_d$ 6 on SPA-Data. Its ablation on feature fusion is especially diagnostic: without HADB + concatenation gives $L_d$ 7, with HADB + concatenation $L_d$ 8, with HADB + addition $L_d$ 9, and with HADB + RPFF $s_1,\dots,s_L$ 0, directly showing that branch-aware residual projection is stronger than naive operators (Chen et al., 2021). MLRN for super-resolution arrives at a related conclusion from a different direction: internal FSFblocks fuse raw multiscale features and fused multi-level features, while GFF aggregates outputs from all preceding blocks; on Set5 $s_1,\dots,s_L$ 1, the combination of GFF and RSC improves PSNR from $s_1,\dots,s_L$ 2 dB in the baseline to $s_1,\dots,s_L$ 3 dB (Lyn, 2020).

In medical and scientific imaging, the same pattern appears in heterogeneous fusion. HiFuse uses a local branch, a global branch, and an HFF branch, and reports that its proposed model on ISIC2018 is $s_1,\dots,s_L$ 4 higher than baseline, $s_1,\dots,s_L$ 5 on the Covid-19 dataset, and $s_1,\dots,s_L$ 6 on the Kvasir dataset. Its ablation on ISIC2018 shows a progression from Local Path $s_1,\dots,s_L$ 7 Acc / $s_1,\dots,s_L$ 8 F1 to full HiFuse-Tiny $s_1,\dots,s_L$ 9 Acc / $s_{l-1}$ 0 F1 as the global branch, attention, IRMLP, and shortcut are added (Huo et al., 2022). In 3D vessel segmentation, matched-scale CNN–graph fusion is performed at every encoder and decoder level; the full model reaches $s_{l-1}$ 1 DICE on ACA, $s_{l-1}$ 2 on HNA, and $s_{l-1}$ 3 on ASOCA, and removing the graph U-Net or the CNN feature injection into UNET-G degrades performance in all three datasets (Zhao et al., 2023). In multimodal electron micrograph learning, HNF fuses patch-sequence and patch-graph features across three patch resolutions and then aligns the resulting visual embedding with LLM-derived class descriptions; the full model reaches Top-1 $s_{l-1}$ 4 on SEM and outperforms variants without HNF, without LLMs, or without multi-head attention (Srinivas et al., 2024).

Detection and recognition tasks emphasize semantic inconsistency and task-guided aggregation. FINE shows that pre-fusion alignment improves a broad range of detectors, for example RT-DETRv1 R18 from $s_{l-1}$ 5 to $s_{l-1}$ 6 AP and Faster R-CNN R50 from $s_{l-1}$ 7 to $s_{l-1}$ 8 AP, with the strongest relative gains on small objects and large efficiency gains over vanilla cross-attention through AATS (Lee et al., 12 Jun 2026). CHBC applies cross-hierarchical feature enhancement and bidirectional consistency learning to FGVC, reaching species/model-level improvements from $s_{l-1}$ 9 to $f(\cdot)$ 0 on CUB, from $f(\cdot)$ 1 to $f(\cdot)$ 2 on Aircraft, and from $f(\cdot)$ 3 to $f(\cdot)$ 4 on Cars, while also improving TCR (Gao et al., 18 Apr 2025). For AIGC IQA, the multi-level extraction–hierarchical fusion–joint aggregation paradigm reaches $f(\cdot)$ 5 SRCC and $f(\cdot)$ 6 PLCC on AGIQA-3K perceptual quality and competitive correspondence performance through prompt-embedded per-level fusion (Meng et al., 23 Jul 2025).

Industrial and multi-source classification further broaden the pattern. HAPNet combines hierarchical attention over global, spectral, and local HSI features with frequency-domain HSI–SAR fusion, achieving $f(\cdot)$ 7 OA on Augsburg and $f(\cdot)$ 8 on Berlin; ablations without HAM or without PFFM reduce OA on both datasets (Luo et al., 2024). FPFNet uses a feature perturbation pool and a parameter-free multi-layer fusion strategy in both encoder and decoder, reaching $f(\cdot)$ 9 image-level AUROC and $\Omega(\cdot)$ 0 pixel-level AUROC on MVTec-AD, and $\Omega(\cdot)$ 1 image-level AUROC and $\Omega(\cdot)$ 2 pixel-level AUROC on VisA. Its ablation shows that encoder+decoder fusion is better than no fusion, encoder-only fusion, or decoder-only fusion (Xu et al., 21 Apr 2026). HFMF combines a hierarchical ViT–ResNet fusion module with a separate multi-stream local-feature module for deepfake detection, reaching mean $\Omega(\cdot)$ 3 accuracy on WildRF and $\Omega(\cdot)$ 4 on CollabDif validation, while calibrated Module 1 reduces ECE across all WildRF splits (Mehta et al., 10 Jan 2025).

5. Relation to adjacent paradigms, misconceptions, and limitations

A common misconception is to equate CHMFFN with any multiscale or multi-branch network. The supplied literature consistently argues otherwise. FINE explicitly states that naive resizing followed by addition or concatenation does not solve semantic inconsistency across pyramid levels (Lee et al., 12 Jun 2026). MH2F-Net reports that concatenation leaves rain streaks and causes detail loss, while addition suppresses more rain but introduces content blurring (Chen et al., 2021). CHBC shows that orthogonal decomposition is superior to simple residual addition for hierarchy-specific feature enhancement (Gao et al., 18 Apr 2025).

A second misconception is that CHMFFN must be attention-based. Attention is prevalent, but not universal. DMF uses stacked shared layers and multiple learning paths without transformer-style attention (Hu et al., 2018). FPFNet improves anomaly detection with parameter-free residual accumulation and normalization, introducing no additional learnable parameters or computational complexity (Xu et al., 21 Apr 2026). CCL, an older precursor in cross-modal retrieval, uses DBNs, linked two-pathway networks, and joint RBM fusion for coarse- and fine-grained features rather than attention blocks (Peng et al., 2017).

A third misconception is that CHMFFN is restricted to visual feature pyramids. The literature includes CNN–graph fusion in vessel segmentation (Zhao et al., 2023), HSI–SAR fusion in the frequency domain (Luo et al., 2024), prompt–visual fusion in AIGC assessment (Meng et al., 23 Jul 2025), class-description–visual fusion in micrograph learning (Srinivas et al., 2024), and SSL–spectrogram fusion in speech enhancement (Mattursun et al., 2024). This suggests that the defining property is not the modality but the structured interaction among heterogeneous feature hierarchies.

The limitations reported across the sources are equally consistent. Increased hierarchical depth can saturate or become costly: in MH2F-Net, increasing MHEBs from $\Omega(\cdot)$ 5 to $\Omega(\cdot)$ 6 yields minimal gain with “massive calculated cost” (Chen et al., 2021). FINE mainly targets adjacent pyramid levels and uses a coarse modulation map by construction (Lee et al., 12 Jun 2026). DMF lacks explicit cross-scale attention, gating, and richer alignment mechanisms, and its variable count grows by roughly $\Omega(\cdot)$ 7– $\Omega(\cdot)$ 8 in the MNIST experiment (Hu et al., 2018). The graph-based vessel model precomputes its graph from UNET-0, so graph generation is not end-to-end (Zhao et al., 2023). HNF lacks explicit top-down cross-scale feedback and does not formalize its internal gating equations rigorously (Srinivas et al., 2024). HFMF trains its modules separately before ensembling rather than as one monolithic fused backbone (Mehta et al., 10 Jan 2025).

6. Transferable design principles and outlook

The supplied literature repeatedly supports several design principles. First, intermediate outputs should be treated as usable representations rather than disposable by-products. This is explicit in MH2F-Net’s HADB, MLRN’s GFF, FPFNet’s encoder–decoder fusion, and the joint aggregation stage of the AIGC IQA framework (Chen et al., 2021, Lyn, 2020, Xu et al., 21 Apr 2026, Meng et al., 23 Jul 2025). Second, preserving distinct feature streams until late enough for adaptive interaction is often beneficial. HiFuse keeps local, global, and fusion branches separate; DMF keeps modality-specific and shared streams separate; BHF-MEF keeps original and gamma-corrected features separate until attention-guided completion (Huo et al., 2022, Hu et al., 2018, Mu et al., 2024).

Third, interaction often benefits from preceding fusion rather than following it. FINE states this principle directly through $\Omega(\cdot)$ 9 rather than $g(\cdot)$ 0 (Lee et al., 12 Jun 2026). Fourth, complementary discrepancy can be more informative than commonality alone, as shown by RPFF in deraining and by MOD in CHBC (Chen et al., 2021, Gao et al., 18 Apr 2025). Fifth, lightweight mechanisms remain competitive when they preserve hierarchy explicitly; FPFNet and DMF are especially clear on this point (Xu et al., 21 Apr 2026, Hu et al., 2018).

This suggests that future CHMFFN designs will likely continue to combine four ideas already present in the supplied literature: explicit retention of intermediate hierarchy states, alignment or distillation before fusion, decomposition of shared and unique components, and task-aware aggregation at the head. The exact implementation may vary—residual projection, cross-attention, graph projection, prompt embedding, perturbation-regularized accumulation, or frequency-domain modulation—but the unifying principle remains stable: hierarchical features become most useful when their interaction is explicit, selective, and preserved throughout the network rather than collapsed at a single bottleneck.