SAMwave: Wavelet-Driven SAM Adaptation

• SAMwave is a wavelet-driven feature enrichment method that adapts SAM by injecting localized high-frequency details via one-level 2D DWT into frozen image encoders.
• It replaces rigid Fourier masking with multi-scale, directionally selective wavelet transforms that generate complementary LH, HL, and HH detail subbands for edge and texture preservation.
• The method employs real and complex-valued adapters for parameter-efficient tuning, achieving improved performance on tasks such as camouflaged object, shadow, defocus blur, and polyp detection.

Searching arXiv for the specified paper and closely related adaptation work to ground the article with current citations. SAMwave is a wavelet-driven feature enrichment method for adapting the Segment Anything Model (SAM) and SAM2 to dense prediction and low-level vision tasks. It is introduced as a parameter-efficient alternative to prior adapter-based tuning strategies that inject high-frequency information through Fourier-domain masking. The method replaces fixed Fourier filtering with wavelet transforms that extract multi-scale, spatially localized, and directionally selective detail coefficients, and it further extends this design with complex-valued adapters coupled to complex wavelet transforms. In the reported formulation, SAMwave keeps the image encoder frozen, learns lightweight per-layer adapters together with the mask decoder, and applies this strategy across tasks including camouflaged object detection, shadow detection, defocus blur detection, polyp segmentation, and forgery-related settings (Yadav et al., 27 Jul 2025).

1. Problem setting and motivation

SAM and SAM2 are described as strong models for semantic segmentation and interactive prompting, but their direct use on dense prediction and low-level vision tasks is limited by the need for fine-grained local cues such as edges, micro-textures, and small gradients. The underlying difficulty is architectural: the ViT encoder emphasizes global context and semantics, while tasks such as camouflaged object detection, shadow detection, defocus blur detection, polyp segmentation, and forgery localization depend on subtle local structure. Without explicit high-frequency guidance, the mask decoder is reported to struggle with subtle boundaries and localized anomalies (Yadav et al., 27 Jul 2025).

The paper positions SAMwave against prior SAM adaptation methods such as Explicit Visual Prompting, SAM-Adapter, and SAM2-Adapter, which inject high-frequency content through Fourier-domain masking. In that setting, the Fourier transform is computed, a fixed band-pass or high-pass mask is applied, and the residual is fed back to the encoder. The analysis reported for SAMwave argues that this strategy provides limited benefit for three reasons. First, a single handcrafted mask ignores per-image and per-task variability in spectral distributions; second, Fourier basis functions are global and therefore have poor spatial localization; third, fixed Fourier masking lacks orientation selectivity and shift tolerance. An empirical comparison further reports that simply inverting the Fourier mask and feeding low-frequency features yields similar performance gaps, with average improvements described as marginal at approximately $0.009$ across COD metrics. This is used to argue that the main limitation is not merely “high-pass versus low-pass,” but the rigidity of the feature extraction mechanism itself (Yadav et al., 27 Jul 2025).

A common misconception addressed by these results is that any injected high-frequency signal should materially improve low-level adaptation. The reported evidence suggests instead that performance depends on whether the representation preserves spatial layout, orientation structure, and task-dependent variability.

2. Wavelet-based formulation

The central design principle of SAMwave is to replace rigid Fourier masking with wavelet transforms. In a one-level 2D discrete wavelet transform, the input is decomposed into one approximation subband, $LL$, and three detail subbands, $LH$, $HL$, and $HH$, corresponding respectively to low-frequency content and horizontal, vertical, and diagonal detail structure. SAMwave aggregates the detail subbands to form a high-frequency representation that is injected into the encoder through lightweight adapters (Yadav et al., 27 Jul 2025).

The paper presents the standard wavelet formulation in terms of a mother wavelet $\psi$ and scaling function $\phi$:

$$\psi_{j,k}(t)=2^{-j/2}\psi(2^{-j}t-k), \qquad \phi_{j,k}(t)=2^{-j/2}\phi(2^{-j}t-k).$$

Detail and approximation coefficients are written as

$$W_{j,k}=\langle x,\psi_{j,k}\rangle, \qquad A_{j,k}=\langle x,\phi_{j,k}\rangle,$$

with the inverse transform reconstructing the input by summing detail and approximation terms across scales (Yadav et al., 27 Jul 2025).

In the practical implementation, SAMwave uses a single-level decomposition, $L=1$. To preserve spatial resolution after wavelet downsampling, the input image $LL$0 is first upsampled to $LL$1, after which a one-level 2D DWT produces subbands $LL$2, $LL$3, $LL$4, and $LL$5, each at resolution $LL$6. The high-frequency map is then formed as

$LL$7

This construction is intended to supply the encoder with localized boundary and texture information while preserving the spatial granularity needed by dense prediction (Yadav et al., 27 Jul 2025).

The evaluated real-valued wavelet families include Haar, Daubechies, Coiflet, and Symlet. The paper also evaluates complex variants, including symmetric complex wavelets denoted sy-a and sy-b, as well as DT-CWT-style implementations (Yadav et al., 27 Jul 2025).

3. Real and complex adapter mechanisms

SAMwave introduces two adapter regimes: real-valued adapters based on real DWT coefficients, and complex-valued adapters that operate on complex wavelet coefficients. The latter are designed to capture complex-valued spatial-frequency information, particularly phase and directional structure, through complex wavelet transforms (Yadav et al., 27 Jul 2025).

For complex wavelets, the paper defines coefficients from two approximately Hilbert-pair trees, $LL$8 and $LL$9, and combines them as

$LH$0

These coefficients are described as approximately shift-invariant and directionally selective, while retaining the phase information $LH$1 that is relevant for edge orientation and local structure (Yadav et al., 27 Jul 2025).

Adapter integration is performed per encoder layer. Let $LH$2 denote the ViT feature map at layer $LH$3. SAMwave constructs two components: $LH$4, obtained from encoder embeddings through a small MLP, and $LH$5, obtained from the wavelet-derived high-frequency map by patchification and projection. These are concatenated channel-wise:

$LH$6

In the real-valued case, a two-layer MLP computes

$LH$7

In the complex case, with complex weights $LH$8 and complex input $LH$9, the linear block is

$HL$0

followed by split-complex activation, and the residual fusion is written as

$HL$1

or alternatively by adding both real and imaginary parts through two residuals (Yadav et al., 27 Jul 2025).

This formulation makes the fusion data-adaptive rather than mask-driven. The paper also gives an equivalent high-level expression,

$HL$2

where $HL$3 is learnable and $HL$4 is implemented by MLP-based adapters rather than an explicit attention softmax (Yadav et al., 27 Jul 2025).

4. Architectural integration and optimization

SAMwave is inserted into SAM or SAM2 image encoder blocks along the residual path after the block feature $HL$5 has been computed. The layer-wise structure described in the paper is: a standard ViT block with layer normalization, multi-head self-attention, residual addition, layer normalization, MLP, residual addition, yielding $HL$6; then a SAMwave adapter computes $HL$7 and updates the block output via $HL$8 (Yadav et al., 27 Jul 2025).

The real-valued adapter is a two-layer MLP with a tuning projection, GeLU activation, and an up-projection back to the encoder channel dimension. The paper characterizes the resulting parameter count as a small fraction of the backbone, consistent with parameter-efficient tuning. In the complex-valued variant, MLP weights are replaced with complex-valued weights, and the projection convolution used on $HL$9 can also be complex-valued to preserve phase (Yadav et al., 27 Jul 2025).

Training proceeds by freezing the SAM or SAM2 image encoder and learning only the mask decoder and the adapter parameters. The prompt encoder remains as in SAM. Implementation is reported in PyTorch on an NVIDIA A100, with images resized to $HH$0 and upsampled by $HH$1 before the one-level DWT. Boundary handling follows standard DWT library defaults such as symmetric padding (Yadav et al., 27 Jul 2025).

The task-specific losses are explicitly enumerated. Binary cross-entropy is used for defocus blur detection and forgery detection, balanced BCE for shadow detection, and BCE plus IoU for camouflaged object detection. The paper writes:

$HH$2

$HH$3

with

$HH$4

and

$HH$5

Standard optimizer choices such as AdamW with decoupled weight decay are stated to be compatible, although exact hyperparameters are not mandated in the text (Yadav et al., 27 Jul 2025).

5. Benchmarks, tasks, and empirical results

The reported evaluation covers four benchmark families: camouflaged object detection, shadow detection, defocus blur detection, and polyp detection. The datasets and metrics are summarized below (Yadav et al., 27 Jul 2025).

Task	Datasets	Metrics
Camouflaged Object Detection	CHAMELEON, CAMO, COD10K	S-measure, mean E-measure, weighted F-measure, MAE
Shadow Detection	ISTD, SBU	BER
Defocus Blur Detection	CUHK, DUT	$HH$6, MAE
Polyp Detection	Kvasir-SEG, CVC-ClinicDB, ColonDB, ETIS	mDice, mIoU

Several representative results are highlighted in the paper. On camouflaged object detection with a SAM backbone, SAMwave using Coiflet reaches $HH$7 on CHAMELEON, surpassing SAM2-Adapter at $HH$8. With a SAM2 backbone, SAMwave using Haar reaches $HH$9 on COD10K, outperforming SAM2-Adapter at $\psi$0. On shadow detection, ISTD BER is reported at $\psi$1 for SAMwave with SAM2 and Daubechies, improving over SAM2-Adapter at $\psi$2; on SBU, SAMwave with SAM and Daubechies reports $\psi$3 versus DSD at $\psi$4 and FDRNet at $\psi$5. On defocus blur detection, the DUT benchmark reports $\psi$6 and MAE $\psi$7 for SAMwave with SAM2 and complex sy-b, compared with Explicit Visual Prompting at $\psi$8 and MAE $\psi$9. On CUHK, SAMwave with SAM2 and sy-b achieves $\phi$0, described as on par with Explicit Visual Prompting, while showing lower MAE in several configurations. On polyp detection, Kvasir mDice is reported as $\phi$1 for SAMwave with SAM2 and Daubechies versus $\phi$2 for SAM-Adapter, and ETIS reports mDice $\phi$3 and mIoU $\phi$4 for SAMwave with SAM2 and sy-b, both listed as best (Yadav et al., 27 Jul 2025).

The ablation results clarify the source of these gains. Using the $\phi$5 subband degrades performance; individual detail subbands $\phi$6, $\phi$7, and $\phi$8 are beneficial, but the best results arise when all three are combined. This is presented as evidence that horizontal, vertical, and diagonal detail cues are complementary. Across wavelet families, all evaluated real wavelets improve performance, with Coiflet and Haar often strongest in camouflaged object detection and forgery-related settings, while complex symmetric wavelets often perform best on blur and polyp tasks, especially with SAM2. The paper also reports that per-block adapter placement with residual fusion is critical, and that removing the adapters substantially hurts performance relative to full SAMwave (Yadav et al., 27 Jul 2025).

6. Efficiency, interpretability, limitations, and prospective extensions

SAMwave is presented as a parameter-efficient adaptation strategy. The DWT is characterized as lightweight relative to backbone forward passes, and the adapters are described as small MLPs inserted into selected encoder blocks while leaving the encoder weights frozen. This design reduces trainable parameter count and computational overhead relative to full backbone fine-tuning (Yadav et al., 27 Jul 2025).

Interpretability is a central part of the method’s framing. Wavelet-domain features are localized in both space and frequency, and the $\phi$9, $$\psi_{j,k}(t)=2^{-j/2}\psi(2^{-j}t-k), \qquad \phi_{j,k}(t)=2^{-j/2}\phi(2^{-j}t-k).$$0, and $$\psi_{j,k}(t)=2^{-j/2}\psi(2^{-j}t-k), \qquad \phi_{j,k}(t)=2^{-j/2}\phi(2^{-j}t-k).$$1 subbands correspond to oriented edges and textures that are directly relevant to boundary accuracy. Complex wavelets add phase information, directional selectivity, and approximate shift invariance, which the paper connects to local edge orientation and subtle intensity transitions. Visualizations of detail coefficient maps are reported to reveal enhanced boundary emphasis and texture cues available to the decoder (Yadav et al., 27 Jul 2025).

The paper also states several limitations. Complex-valued operations increase implementation complexity, and framework support for complex convolutions, complex MLPs, and normalization is less mature than for real-valued counterparts. The use of a single decomposition level, $$\psi_{j,k}(t)=2^{-j/2}\psi(2^{-j}t-k), \qquad \phi_{j,k}(t)=2^{-j/2}\phi(2^{-j}t-k).$$2, may miss coarse-to-fine interactions; the text identifies learned wavelets, multi-level decomposition, and dynamic scale selection as plausible extensions. It further notes that task-specific priors, such as blur kernels or illumination models, could be integrated into the fusion mechanism, and that the modest computational overhead may still matter in real-time settings, where pruning or selective adapter placement could help (Yadav et al., 27 Jul 2025).

Taken together, SAMwave situates wavelet-domain feature injection as an alternative to Fourier-domain prompting for SAM adaptation. The reported evidence suggests that the key advantage is not simply access to high-frequency content, but access to high-frequency content that is localized, oriented, and adaptively fused within the encoder.