Cross-Scale Harmonic Fusion Decoder

• CSHF is a lightweight decoder that aggregates multi-scale encoder outputs to produce precise binary segmentation masks for crack detection.
• It employs content-adaptive alignment, learnable scale identity injection, and spatially varying attention for dynamic feature fusion.
• The design balances global continuity and fine boundary recovery while achieving superior performance with only 1.22M parameters.

Cross-Scale Harmonic Fusion (CSHF) is a lightweight decoder introduced as the segmentation head of SCRWKV for topological crack segmentation. Within that architecture, the Structure-Field Encoder (SFE) produces four hierarchical features, and CSHF converts those multi-scale representations into a one-channel crack mask through common-resolution alignment, learnable scale identity injection, spatially varying scale attention, and a final gated projection. In the source description, CSHF is explicitly characterized as a decoder for “precise feature aggregation,” “dynamic multi-level fusion,” and “semantic-gap bridging” between high-level context and low-level boundaries, while the term “harmonic” is not defined in a classical signal-processing sense such as Fourier or wavelet harmonics (Zhang et al., 14 May 2026).

1. Architectural position and problem role

CSHF appears in the SCRWKV architecture as the decoder paired with the SFE backbone. After patch embedding and positional encoding, features are initialized by a standalone Adaptive Multi-scale Cascaded Modulator (AMCM), refined by stacked Structure-Calibrated Insight Unit (SCIU) blocks, and then emitted as multi-scale representations $\{F_1,\dots,F_4\}$, also denoted $\{C_i\}_{i=1}^4$. These four tensors are passed to CSHF, which “culminat[es] in the generation of a precise binary segmentation mask” (Zhang et al., 14 May 2026).

The decoder is described as the only explicit decoder-style fusion block in the paper. It is not presented as a deep multi-stage top-down decoder with repeated lateral merges, as in FPN or U-Net. Instead, it is a lightweight single fusion head that first aligns all encoder outputs to a common channel dimension and common spatial resolution, then performs scale-aware weighting and gated projection. This design is task-specific: crack segmentation requires simultaneous preservation of global continuity and restoration of thin local boundaries, while also remaining compact enough to fit the paper’s “ultra-compact” objective (Zhang et al., 14 May 2026).

The target problem explains why such a decoder is needed. Cracks are described as thin and elongated, easily fragmented, variable in width and visibility, and frequently embedded in clutter such as shadows, oil stains, pavement particles, and textured surfaces. A plausible implication is that uniform fusion across encoder stages would be poorly matched to this setting, because shallow stages contain boundary-sensitive detail while deeper stages contain coarse but semantically stable crack trajectories. CSHF is introduced specifically to reconcile these scales at the final aggregation stage (Zhang et al., 14 May 2026).

2. Alignment to a common feature space

CSHF operates on four encoder outputs $\{C_i\}_{i=1}^4$. Each feature is first projected by a pointwise layer into a unified embedding dimension $D$, then resampled to a common output resolution $H\times W$ using DySample:

$$\tilde{\mathbf{F}}_i = \mathrm{DySample}_{s_i}\!\left(\mathcal{W}_{p}(C_i)\right).$$

The source description states that $\mathcal{W}_p$ is a pointwise projection and that DySample is used instead of fixed bilinear interpolation. This makes the alignment stage content-adaptive rather than purely geometric, which is particularly relevant when thin cracks can be damaged by naive resizing (Zhang et al., 14 May 2026).

After alignment, each scale receives a learnable scale embedding $\mathbf{E}_i\in\mathbb{R}^D$, broadcast spatially and added channelwise:

$$\hat{\mathbf{F}}_i = \tilde{\mathbf{F}}_i + \mathbf{E}_i.$$

The paper explains that this compensates for resampling loss and produces a “topology-aware feature.” Functionally, this means that once all features have been resized to the same tensor shape, the decoder still retains explicit information about which representation came from which encoder stage. The introduction’s description of “scale-signature binding” is consistent with this interpretation (Zhang et al., 14 May 2026).

The paper does not provide the exact per-stage resolutions or channel counts of the four input tensors, nor does it specify the value of $D$ inside CSHF. This absence is consequential for reimplementation: the decoder’s logic is fully specified at the level of operations, but not at the level of exact hidden widths or stage geometry (Zhang et al., 14 May 2026).

3. Scale-aware attention and the “harmonic” gate

Once the aligned and scale-tagged features are available, CSHF applies a scale-aware attention mechanism. The four features are concatenated along channels and passed through an attention predictor $\{C_i\}_{i=1}^4$0, followed by softmax:

$\{C_i\}_{i=1}^4$1

The harmonized representation is then computed as a weighted sum over scales:

$\{C_i\}_{i=1}^4$2

This is the core fusion step. Rather than using equal-weight addition or raw concatenation as the final representation, the decoder computes a spatially adaptive mixture over the four scales. This suggests that different spatial positions can prefer different encoder levels: shallow features where hairline boundaries dominate, deeper features where broader semantic continuity is more reliable (Zhang et al., 14 May 2026).

The final stage gives the module its name. The paper uses $\{C_i\}_{i=1}^4$3 as a gate over the concatenation of the original aligned features:

$\{C_i\}_{i=1}^4$4

where $\{C_i\}_{i=1}^4$5 is a channel expansion layer, $\{C_i\}_{i=1}^4$6 is LayerNorm, and $\{C_i\}_{i=1}^4$7 is a bottleneck convolution producing $\{C_i\}_{i=1}^4$8. The dataflow is therefore: align all scales, tag them with scale identity, compute a weighted sum $\{C_i\}_{i=1}^4$9, expand that fused representation, and use it multiplicatively to gate the full four-scale aligned feature stack before the final bottleneck projection (Zhang et al., 14 May 2026).

A common misconception is that “harmonic” here denotes harmonic basis functions, frequency-domain operators, Laplacians, or harmonic energy minimization. The source explicitly rules that out: the paper does not define “harmonic” in a signal-processing or spectral sense. In CSHF, “harmonic” is primarily a naming choice for the scale-aware fusion plus gating mechanism just described (Zhang et al., 14 May 2026).

4. Supervision and empirical evidence

CSHF is supervised only through the final segmentation map $\{C_i\}_{i=1}^4$0. The supplementary material gives the total objective as

$\{C_i\}_{i=1}^4$1

with

$\{C_i\}_{i=1}^4$2

and

$\{C_i\}_{i=1}^4$3

The best reported weight ratio is Dice:BCE $\{C_i\}_{i=1}^4$4, i.e. $\{C_i\}_{i=1}^4$5. The authors interpret Dice as improving structural consistency under class imbalance and BCE as preserving sharp pixel boundaries, which aligns with the decoder’s intended role in topology preservation and boundary recovery (Zhang et al., 14 May 2026).

The strongest direct evidence for CSHF comes from segmentation-head ablations on the TUT setting. Under the reported comparison, CSHF exceeds UNet, Ham, SegFormer, and MFS heads in both F1 and mIoU:

Head	F1	mIoU
UNet	0.8351	0.8443
Ham	0.8257	0.8376
SegFormer	0.8365	0.8453
MFS	0.8374	0.8462
CSHF	0.8428	0.8512

The paper emphasizes that CSHF, MFS, SegFormer, and Ham are all configured to keep the full model at 1.22M parameters. In the same supplementary comparison, CSHF uses 22.78G FLOPs and 28MB, slightly fewer FLOPs than MFS at 23.00G while remaining at the same parameter count. The abstract reports the corresponding SCRWKV result as “only 1.22M parameters,” with F1 score 0.8428 and mIoU 0.8512 on TUT (Zhang et al., 14 May 2026).

The evidence is comparative rather than fully isolating. There is no ablation in which CSHF is removed and replaced by no decoder, and there is no decomposition of the contribution of DySample, scale embeddings, attention, and harmonic gating separately. This means the empirical support is strongest for CSHF as a complete head, not for any single internal submechanism (Zhang et al., 14 May 2026).

5. Relation to adjacent cross-scale and harmonic research

CSHF belongs to a broader family of cross-scale aggregation modules, but it occupies a specific point in that landscape. Relative to explicitly harmonic methods, it is notably non-spectral. CHASM, for example, performs cross-frequency harmonization by forcing all frequency-indexed channel operators to share a learned basis while keeping frequency-specific positive gains, yielding a structured spectral operator family $\{C_i\}_{i=1}^4$6 (Fang et al., 14 May 2026). Harmonic Alignment, in a different domain, aligns datasets through diffusion harmonics derived from intrinsic geometry and does not require pointwise sample correspondence (III et al., 2018). CSHF does not adopt either type of harmonic formalism; its “harmonic” step is a learned scale mixture and gate in the spatial feature domain (Zhang et al., 14 May 2026).

Relative to other cross-scale fusion architectures, CSHF is closer in spirit. CoFusion uses a three-level pyramid, spatial-spectral collaboration, and DWT-based low/high-frequency processing through SpeCAM, making it strongly cross-scale and partially frequency-aware (Li, 12 Apr 2026). ECFNet emphasizes that alignment should precede cross-scale fusion and combines deformable alignment with structure-guided high-frequency refinement in MRI super-resolution (Yang et al., 2024). CSHF is lighter than both patterns: it does not build a multi-stage decoder or explicit spectral branch, but instead aligns four encoder outputs once and fuses them through attention and gated projection (Zhang et al., 14 May 2026).

CS-Mixer provides another adjacent design pattern. It uses cross-scale embedding and patch merging together with local/global aggregation and low-rank joint spatial-channel mixing, but it does not contain a harmonic, Fourier, wavelet, or basis-expansion formulation (Cui et al., 2023). This contrast is useful because it clarifies that cross-scale aggregation alone does not imply harmonic modeling. CSHF shares the cross-scale emphasis, yet remains a decoder head rather than a general-purpose token mixer (Zhang et al., 14 May 2026).

6. Interpretation, scope, and limitations

The novelty of CSHF is primarily architectural and efficiency-oriented. The paper’s distinct ingredients are content-adaptive DySample alignment, learnable scale embeddings after alignment, spatially varying scale attention over all four levels, and a final gate that uses the harmonized representation to modulate the full aligned feature spectrum. This combination makes CSHF a compact decoder variant rather than a generic FPN-style or U-Net-style fusion block (Zhang et al., 14 May 2026).

Its scope within SCRWKV is also specific. The SFE backbone is responsible for structure-aware representation formation: AMCM enriches texture and receptive fields, GBST models geometry-guided bidirectional structural interaction, Dy-WKV with DSCD suppresses noise propagation, and SCIU integrates those mechanisms. CSHF does not discover crack topology from scratch; it consumes the structure-aware outputs of the encoder and performs the final semantic-spatial reconciliation needed for a crisp crack mask. This suggests that its main contribution lies in preserving long-range continuity from deeper stages while recovering thin boundaries from shallower ones (Zhang et al., 14 May 2026).

Several limitations are explicit. The paper does not provide the exact hidden dimension $\{C_i\}_{i=1}^4$7, the kernel size of the bottleneck $\{C_i\}_{i=1}^4$8, the precise type of $\{C_i\}_{i=1}^4$9 and $D$0, or a standalone latency and parameter breakdown for CSHF itself. It does not present a formal theoretical analysis of why the module should be called “harmonic,” nor does it compare against a broad family of modern attention-based decoders. Stronger claims about decoder superiority beyond the reported head comparisons would therefore be speculative (Zhang et al., 14 May 2026).

Taken together, CSHF is best understood as a four-scale lightweight decoder that aligns encoder outputs, binds scale identity after resampling, computes per-pixel scale weights, and uses the resulting fused representation as a gate over the aligned multiscale feature stack. Its importance in the current literature lies less in a new harmonic theory than in a compact cross-scale aggregation pattern tailored to topology-sensitive segmentation under severe parameter constraints (Zhang et al., 14 May 2026).