Adaptive Approximate Wavelet Transform

• AdaWAT is a learnable, adaptive wavelet-inspired frequency decomposition module that separates low- and high-frequency sub-bands for fine-grained image fusion.
• It employs grouped convolutions and dilated filters to enhance modality-specific features across tasks like infrared-visible and medical image fusion.
• Empirical results demonstrate that AdaWAT improves fusion quality metrics by increasing structural similarity, texture preservation, and mutual information.

Adaptive Approximate Wavelet Transform (AdaWAT) is a learnable, adaptive, wavelet-inspired frequency decomposition module introduced within the AdaSFFuse framework for task-generalized multimodal image fusion. In that setting, it is the main mechanism for frequency decoupling: early feature embeddings from each modality are separated into low-frequency and multiple high-frequency sub-bands, refined by frequency-specific convolutions, passed to Spatial-Frequency Mamba Blocks for fusion, and then recoupled by the Adaptive Inverse Wavelet Approximation Transform (AdaIWAT) for deep fusion and final image reconstruction (Wang et al., 21 Aug 2025). It is adaptive because its effective filters are learned from data across Infrared-Visible Image Fusion, Multi-Focus Image Fusion, Multi-Exposure Image Fusion, and Medical Image Fusion; it is approximate because both analysis and synthesis are implemented as learnable convolutional approximations rather than as a strict analytic discrete wavelet transform (Wang et al., 21 Aug 2025).

1. Architectural role and problem setting

Within AdaSFFuse, AdaWAT is introduced to address three limitations that recur in multimodal image fusion: hand-crafted, non-learnable frequency decompositions such as classical DWT, Laplacian pyramids, and fixed Fourier filters; task-agnostic transforms that do not adapt to IVF, MFF, MEF, or MIF; and pure spatial-domain feature extractors, including CNNs, Transformers, and Mamba, that often lose high-frequency details and handle modality differences poorly (Wang et al., 21 Aug 2025). Its central function is therefore not merely transform-domain preprocessing, but scene- and task-conditioned frequency decoupling inside an end-to-end fusion network.

The forward role of AdaWAT is explicitly frequency-segmented. It takes early modality features, such as visible and infrared embeddings, and separates them into one low-frequency component and three high-frequency sub-bands. This produces a wavelet-like multi-band representation suitable for fine-grained extraction of modality-specific information and for frequency-aligned fusion, so that low-frequency structure and high-frequency detail can be handled differently (Wang et al., 21 Aug 2025). The shallow fused outputs are then processed by Spatial-Frequency Mamba Blocks, while AdaIWAT serves as the inverse recoupling stage for deep fusion and final reconstruction.

This placement makes AdaWAT the backbone of frequency-domain processing in AdaSFFuse. The module is therefore not an isolated transform block but part of an analysis–fusion–synthesis pipeline in which low- and high-frequency content are explicitly decoupled, fused, and reconstructed in a task-generalized setting. A common misconception is to treat it as a drop-in replacement for a fixed DWT layer. In the AdaSFFuse design, however, AdaWAT is coupled to learnable shallow embedding, frequency-specific enhancement, Mamba-based cross-domain fusion, and an adaptive inverse transform (Wang et al., 21 Aug 2025).

2. Mathematical formulation

AdaWAT begins from the classical wavelet decomposition recalled in the paper. For a discrete one-dimensional signal $x(t)$, the wavelet and scale functions are

$$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$

and the decomposition at scale $j_0$ is

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$

with detail and approximation coefficients

$$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$

The corresponding adaptive one-dimensional analysis vectors are introduced as

$$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$

where $\vec{u}_0$ is the low-frequency filter and $\vec{u}_1$ is the high-frequency filter. The recursive filter-bank form is

$$v_{j+1,p}=\sum_k \vec{u}_0[k-2p]a_{j,k}, \qquad w_{j+1,p}=\sum_k \vec{u}_1[k-2p]a_{j,k}.$$

For a two-dimensional feature map, for example

$$F_{m1}\in\mathbb{R}^{\frac{H}{2}\times\frac{W}{2}\times C},$$

AdaWAT constructs separable two-dimensional filters by outer products,

$$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$0

These correspond respectively to low–low, low–high, high–low, and high–high filtering, and are implemented through grouped convolutions (Wang et al., 21 Aug 2025). Convolution plus downsampling produces sub-bands

$$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$1

The paper assigns two meanings to the word approximate. First, AdaWAT is a convolutional approximation of wavelet filter banks: instead of fixed analytical filters such as Haar or Daubechies, the vectors $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$2 and $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$3 are implemented and adapted within the network through grouped convolutions. Second, after wavelet-like filtering, AdaWAT applies different dilated convolutions to low- and high-frequency sub-bands to perform learnable, task-adaptive low-pass and high-pass enhancement (Wang et al., 21 Aug 2025). The low-frequency $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$4 band uses a $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$5 dilated convolution with $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$6, while the high-frequency $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$7, $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$8, and $$\psi_{j,k}(t) = 2^{j/2}\psi(2^j t-k), \qquad \phi_{j,k}(t) = 2^{j/2}\phi(2^j t-k),$$9 bands use $j_0$0 dilated convolution with $j_0$1.

The forward decoupling stage yields frequency-segmented fusion variables

$j_0$2

$j_0$3

After shallow fusion by Spatial-Frequency Mamba, AdaIWAT performs recoupling. The inverse stage uses similar wavelet kernels and transpose convolution with adaptive weights, combines the low- and high-frequency fused features into

$j_0$4

and reconstructs the fused image

$j_0$5

with minimal information loss while preserving task-relevant features (Wang et al., 21 Aug 2025).

3. Adaptive mechanisms and optimization

AdaWAT is adaptive in several distinct senses. The first is filter adaptivity. The analysis vectors $j_0$6 and $j_0$7, together with the corresponding two-dimensional kernels, are learned from data rather than fixed by analytic construction. During joint training across IVF, MFF, MEF, and MIF, these filters are tuned to modality-specific and task-specific frequency patterns, including thermal contours in IVF, exposure-related high-frequency detail in MEF, in-focus textures in MFF, and anatomical or functional structures in MIF (Wang et al., 21 Aug 2025).

The second is supervisory adaptivity. AdaSFFuse uses a total loss

$j_0$8

with

$j_0$9

The constituent losses are

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$0

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$1

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$2

These objectives drive AdaWAT toward frequency decompositions that facilitate high structural similarity, maximal retention of gradient and texture information, and correct intensity combination (Wang et al., 21 Aug 2025).

The third is interaction with adaptive spatial-frequency fusion. After AdaWAT, the frequency-segmented features are processed by AdaD-SSD within the Spatial-Frequency Mamba Block. The spatial-aware branch is

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$3

and the frequency-filtering branch is

$$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$4

Backpropagation through these branches further shapes AdaWAT’s effective behavior, so the learned decomposition is conditioned not only by the wavelet-like filters themselves but also by the subsequent spatial-frequency selection and fusion process (Wang et al., 21 Aug 2025).

From an implementation standpoint, the module uses grouped convolutions, single-level four-subband decomposition, downsampling to $$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$5, and distinct dilation rates for low and high bands. As part of AdaSFFuse, the overall model has $$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$6M parameters, $$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$7 G FLOPs for $$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$8 input, and $$x(t)=\sum_{j>j_0}\sum_k w_{j,k}\psi_{j,k}(t)+\sum_k v_{j_0,k}\phi_{j_0,k}(t),$$9 ms latency. The paper reports that this is lower than several comparison systems, including HitFusion, EMMA, SwinFusion, and MambaDFuse, and attributes the efficiency in part to grouped convolutions and the limited decomposition depth (Wang et al., 21 Aug 2025).

4. Empirical behavior in multimodal image fusion

The empirical case for AdaWAT in AdaSFFuse is made through both ablation and task-level evaluation. In the ablation study labeled A.I–A.II, a baseline without AdaWAT obtains $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$0, $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$1, $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$2, and $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$3. Adding AdaWAT raises these values to $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$4, $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$5, $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$6, and $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$7 (Wang et al., 21 Aug 2025). Within the terms of that study, the addition of AdaWAT is therefore associated with higher structural similarity, information content, spatial frequency, and mutual information.

A second ablation compares four frequency-decoupling choices for the IVF task: FFT-based decoupling, Laplacian pyramid, fixed Wavelet Transform, and AdaWAT. The reported metrics are $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$8, $$w_{j,k}=\langle x(t),\psi_{j,k}(t)\rangle, \qquad v_{j_0,k}=\langle x(t),\phi_{j_0,k}(t)\rangle.$$9, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$0, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$1 for FFT; $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$2 for LapF; $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$3, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$4, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$5, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$6 for fixed WAT; and $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$7, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$8, $$\vec{u}_0[k]=2^{1/2}\phi(2t-k), \qquad \vec{u}_1[k]=2^{1/2}\psi(2t-k),$$9, $\vec{u}_0$0 for AdaWAT, which is best across all listed metrics (Wang et al., 21 Aug 2025). This directly supports the claim that static frequency transforms are less effective than the adaptive, convolution-based approximation used by AdaWAT.

At the full-system level, AdaSFFuse is reported to achieve state-of-the-art results across four tasks. For IVF and MEF, the paper states that AdaSFFuse is best in nearly all metrics, including IVF $\vec{u}_0$1 versus CDDFuse $\vec{u}_0$2, and MEF $\vec{u}_0$3 versus CDDFuse $\vec{u}_0$4. For MFF and MIF, it reports MFF $\vec{u}_0$5 and $\vec{u}_0$6, and MIF $\vec{u}_0$7, $\vec{u}_0$8, and $\vec{u}_0$9, all listed as best (Wang et al., 21 Aug 2025). The paper explicitly links these gains to better high-frequency preservation and low-frequency consistency enabled by AdaWAT.

The associated visualization claim is similarly frequency-specific: $\vec{u}_1$0 and $\vec{u}_1$1 capture low-frequency structures, $\vec{u}_1$2 and $\vec{u}_1$3 highlight high-frequency details, the fused low- and high-frequency maps $\vec{u}_1$4 and $\vec{u}_1$5 combine more complete information after shallow fusion, and the final fused image preserves clear textures and shapes from both modalities (Wang et al., 21 Aug 2025). This makes AdaWAT’s empirical contribution primarily one of structured decoupling prior to fusion, rather than a generic increase in network capacity.

5. Relation to classical transforms and the broader adaptive-wavelet literature

AdaWAT in AdaSFFuse is closest to classical DWT in its filter-bank-plus-downsampling structure and in its explicit $\vec{u}_1$6, $\vec{u}_1$7, $\vec{u}_1$8, and $\vec{u}_1$9 sub-band decomposition. The differences identified in the paper are learnability, task adaptation, dilated enhancement of low and high bands, and an adaptive inverse based on transpose convolutions rather than exact orthogonal inversion (Wang et al., 21 Aug 2025). The paper also states that it does not provide formal proofs of perfect reconstruction, stability bounds, or orthogonality; conceptually, AdaWAT plus AdaIWAT form a learned analysis–synthesis pair akin to a biorthogonal wavelet system, but optimized for fusion quality rather than strict mathematical properties (Wang et al., 21 Aug 2025).

The adjective approximate has a distinct meaning in another wavelet literature. For the standard Morlet wavelet, the classical admissibility integral diverges, yet exact inversion can still be obtained through formulas involving $$v_{j+1,p}=\sum_k \vec{u}_0[k-2p]a_{j,k}, \qquad w_{j+1,p}=\sum_k \vec{u}_1[k-2p]a_{j,k}.$$0, integration over scales, and, in the practical real-valued case, an expression of the form

$$v_{j+1,p}=\sum_k \vec{u}_0[k-2p]a_{j,k}, \qquad w_{j+1,p}=\sum_k \vec{u}_1[k-2p]a_{j,k}.$$1

so that non-admissible kernels can still support exact inverse transforms (Postnikov et al., 2015). This makes clear that approximate in AdaWAT does not denote the same concept as approximate wavelet in non-admissible CWT theory. A plausible implication is that the term is polysemous across subfields.

The supplied literature also shows that AdaWAT belongs to a broader methodological family of adaptive or approximate wavelet constructions. In texture classification, adaptive wavelet modules are built with Haar DWT-Split and lifting-scheme predictor/updater CNNs, and are explicitly described as task-oriented rather than strictly reconstructive (Yu et al., 2024). In time-series analysis, AdaWaveNet uses a lifting scheme-based adaptive transform and inverse transform with learnable convolutional filters for forecasting, imputation, and super-resolution (Yu et al., 2024). In graph learning, ASWT-SGNN approximates graph wavelet operators by spectrum-adapted polynomials, avoiding eigen-decomposition while retaining multiscale spatial-spectral localization (Liu et al., 2023). In two-dimensional empirical wavelet analysis, arbitrary partitions of the Fourier domain are detected from scale-space maxima and a watershed transform, and corresponding frame-based filters are then constructed (Hurat et al., 2024). Other supplied examples include region-based path-adaptive sparse image transforms (Budinich, 2017) and adaptive online refinement of wavelet bases for value-function approximation in reinforcement learning (Beukman et al., 2022). This suggests a broader family resemblance, even though AdaWAT as a proper name is explicitly attached to AdaSFFuse.

6. Constraints, misconceptions, and future directions

The principal misconception around AdaWAT is to read it as a standard wavelet transform with trainable coefficients. The paper instead presents it as a learnable grouped-convolutional approximation to wavelet analysis, followed by task-conditioned enhancement and fused within a larger spatial-frequency architecture (Wang et al., 21 Aug 2025). Its inverse is likewise adaptive and approximate, not a strict analytic inverse in the orthogonal-DWT sense.

A second misconception is that improved visual or low-level fusion metrics necessarily imply improved downstream semantics. The supplied discussion notes that improved visual and frequency representations do not always guarantee better downstream performance, including some semantic segmentation cases where fused images attenuate certain classes (Wang et al., 21 Aug 2025). This suggests that AdaWAT’s current optimization target is low-level fusion quality rather than explicit downstream task alignment.

The explicit limitations are theoretical as well as application-level. Invertibility and stability are handled empirically through AdaIWAT and loss minimization, but formal guarantees are not provided. Future work is described in two directions: end-to-end, task-aware multimodal frameworks in which the fusion process, including AdaWAT, is jointly optimized with downstream losses such as detection or segmentation losses; and more principled designs of learned wavelet-like bases with constraints on reconstruction error and frequency coverage (Wang et al., 21 Aug 2025).

In this sense, AdaWAT occupies an intermediate position between classical wavelet theory and task-specific deep architectures. It preserves the organizing logic of multi-band wavelet analysis—low/high separation, sub-band-specific processing, and analysis–synthesis recoupling—while replacing fixed bases, task-agnostic decomposition, and exact analytic inversion with learned operators optimized for multimodal fusion performance (Wang et al., 21 Aug 2025).