---
title: Multi-scale Decomposition & Reconstruction
url: https://www.emergentmind.com/topics/multi-scale-decomposition-reconstruction
type: topic
---

# Multi-scale Decomposition & Reconstruction

Searching arXiv for recent papers on multi-scale decomposition reconstruction and closely related formulations.
Multi-scale decomposition reconstruction denotes a family of procedures in which a signal, image, field, matrix, or latent representation is separated into constituents associated with different scales and then reassembled by summation, inverse transform, or composition. In the cited literature, this paradigm appears in hierarchical Tikhonov and total-variation decompositions, linear and learned image pyramids, block-wise low-rank matrix models, local multi-scale supervision for masked image modeling, coarse-to-fine diffusion priors, and multiresolution decompositions for inverse problems and dynamical systems [1301.5041] [2008.00802] [1507.08751] [2303.05251] [2601.23201] [1506.00564]. The common objective is not merely to partition data by resolution, but to align representation, regularization, and reconstruction with the scale structure of the underlying phenomenon.

## 1. Scale as a modeling variable

In the surveyed works, “scale” is defined operationally rather than abstractly. In learned image models, it may be the supervision patch size: LocalMIM defines scales \(p_1<p_2<\dots<p_P\), partitions the input \(x\in\mathbb R^{H\times W\times C}\) into non-overlapping \(p_j\times p_j\) patches, and derives supervision \(y^{(j)}\in\mathbb R^{H/p_j\times W/p_j\times D}\) from descriptors such as normalized pixels or HOG [2303.05251]. In compressive imaging, it may be a filterbank output \(x^{(s)}=D_s B_s x\), with Laplacian pyramid, Haar wavelet, or scale-space decompositions furnishing the scale channels [2008.00802]. In Laplacian-pyramid diffusion models, the bands are \(x^{(3)}=I^{(3)}\), \(x^{(2)}=I^{(2)}-U(I^{(3)})\), and \(x^{(1)}=I^{(1)}-U(I^{(2)})\) [2601.23201].

Scale also appears as a partition of time, geometry, or matrix support. In mrDMD, level \(\ell\) partitions the time axis into \(J_\ell=2^{\ell-1}\) windows and extracts slow and fast components within each window [1506.00564]. In multi-scale low-rank matrix decomposition, each scale \(i\) is associated with a partition \(P_i\) of the matrix indices into blocks \(m_i\times n_i\), and each component is locally low-rank on its own partition [1507.08751]. In AMD for time series forecasting, repeated average-pooling generates scale patterns \(\tau_i^c\), and coarse scales are mixed back into fine ones in a residual top-down fashion [2406.03751].

This diversity shows that multi-scale decomposition reconstruction is not tied to a single representation theory. A plausible implication is that the notion of scale is most useful when it is coupled to the inductive bias of the task: patch size in masked prediction, block size in low-rank modeling, time bin in dynamical decomposition, or diffusion time in PDE-based filtering.

## 2. Reconstruction operators and synthesis rules

The reconstruction step is as central as the decomposition. In additive image decompositions, the original datum is recovered as a coarse residual plus detail bands. The SwV pipeline defines \(B_k=E_{r_k,\epsilon_k}(B_{k-1})\), \(D_k=B_{k-1}-B_k\), and reconstructs by
\[
I = B_N + \sum_{k=1}^N D_k,
\qquad
I_{\mathrm{enh}} = B_N + \sum_{k=1}^N \alpha_k D_k,
\]
where \(\alpha_k\) controls detail amplification or suppression [2105.01951]. The constrained-diffusion method similarly forms dyadic bands \(C_n(x)=I_n(x)-I_{n-1}(x)\) and reconstructs
\[
f(x)=\sum_{n=1}^N C_n(x)+I_N(x),
\]
with exact reconstruction up to finite-\(dt\) error [2201.05484]. In low-rank matrix decomposition, the recovered matrix is simply recombined as \(\hat X=\sum_{i=1}^L X_i^\star\) [1507.08751].

Other settings require non-additive synthesis. For manifold-valued data, Grohs and Wallner replace vector-space addition and subtraction by \(\oplus\), \(\ominus\), and geodesic weighted averages. Analysis computes
\[
c^{(j-1)}=D(c^{(j)}), \qquad d^{(j)}=Q\bigl(c^{(j)}\ominus S(c^{(j-1)})\bigr),
\]
while synthesis reconstructs
\[
c^{(j)}=S(c^{(j-1)})\oplus R(d^{(j)}).
\]
The paper also shows a basic obstruction: for arbitrary nonlinear subdivision filters on manifolds, one cannot expect a general perfect-reconstruction theory analogous to the linear biorthogonal case; interpolating and midpoint-interpolating constructions are the cases that bypass this obstruction [1001.1517]. In image registration, the multiscale expansion is compositional rather than additive, with an optimal registration written as
\[
g^\ast=\dots\circ g_2\circ g_1\circ g_0
\]
instead of \(\sum_j u_j\) [1803.01790].

A common misconception is that multi-scale decomposition automatically implies exact invertibility. The literature is more specific: some frameworks provide exact reconstruction identities, some provide exact reconstruction up to numerical discretization error, and some admit only structured or local notions of invertibility.

## 3. Variational and hierarchical formulations

A large part of the theory begins with Tikhonov-type decompositions. Tadmor, Nezzar, and Vese’s hierarchical \((L_2,BV)\) scheme, further analyzed in subsequent work, defines
\[
u_{k+1}
=\arg\min_{u\in BV(\Omega)\cap L_2(\Omega)}
\Bigl\{\tfrac12\|f-u\|_{L_2}^2+t_k|u-u_k|_{BV}\Bigr\},
\qquad u_0=0,
\]
with residuals \(r_k=f-u_k\). The paper proves monotonicity of the data fidelity, a telescoping energy identity, and \(L_2\)-convergence \(r_k\to0\) when \(t_k\to0\) [1301.5041]. In the base Tikhonov decomposition,
\[
(u_t,v_t)=\arg\min_{u+v=f}\{\|v\|_X+t\|u\|_Y\},
\]
the component \(u_t\in Y\) captures the “good” part and \(v_t\in X\) the residual [1301.5041].

The Banach-space MHDM generalizes this structure to linear ill-posed problems with fidelity
\[
F(u;f)=\tfrac12\|Tu-f\|_H^2
\]
and regularizers \(J\) or \(J_k\). With \(x_{-1}=0\), it iterates
\[
u_k\in\arg\min_{u\in X}\{F(u;v_{k-1})+J_k(u)\},\qquad
x_k=x_{k-1}+u_k,\qquad
v_k=f-Tx_k.
\]
Under generalized triangle-inequality assumptions and geometric conditions on \(\lambda_k\), the residual obeys a decay estimate and converges to zero; in the noisy case, stopping by a discrepancy principle yields stability [2304.08332]. The same work shows that Bregman iteration can be written as an adaptive MHDM, and that coincidence with single-step Tikhonov regularization holds only under specific zero-Bregman-increment conditions; one-dimensional total-variation denoising is one such case [2304.08332].

The abstract multiscale theory of Modin, Nachman, and Rondi extends the paradigm to nonlinear inverse problems and diffeomorphic image registration. With increasing \(\lambda_n\) and decreasing \(a_n\), the iterates
\[
\sigma_n\in\arg\min_{\sigma:\tilde\sigma_{n-1}+\sigma\in E}
\left\{
\lambda_n[d_Y(\hat H,\mathcal N(\tilde\sigma_{n-1}+\sigma))]^\alpha
+a_n|\tilde\sigma_{n-1}+\sigma|^\gamma
+|\sigma|^\beta
\right\}
\]
yield convergence of the fidelities to the infimum value \(\delta_0\) and, under additional assumptions, convergence of the parameters [1803.01790].

Blind deconvolution introduces a coupled multiscale hierarchy for both image and kernel:
\[
U^\dagger=\sum_{s=0}^S u_s,\qquad K^\dagger=\sum_{s=0}^S k_s,
\]
with scale-wise energies penalized by fractional Sobolev norms and a positivity constraint \(\widehat k_s(\xi)\ge0\) to break the scale-indeterminacy \((u_s,k_s)\mapsto(u_s/c,ck_s)\). The method establishes residual convergence in the noise-free case and discrepancy-principle stability in the noisy case [2409.08734].

## 4. Neural architectures for multi-scale decomposition and reconstruction

Recent learned systems internalize decomposition and reconstruction inside the architecture rather than treating them as separate preprocessing and inverse steps. LocalMIM is exemplary: it groups encoder layers into fine-scale layers \(I_1\) and coarse-scale layers \(I_2\), attaches a tiny decoder to each selected layer, and optimizes
\[
L_{\mathrm{total}}=\alpha L_{\mathrm{fine}}+\beta L_{\mathrm{coarse}},
\]
with default \(\alpha=\beta=1.0\). Lower and upper layers reconstruct fine-scale and coarse-scale supervision signals respectively, and the default ViT-B layer set is \([2,4,10,12]\) with HOG descriptors using 18 orientation bins [2303.05251].

MS-DCI jointly learns decomposition, sampling, and reconstruction. The decomposition stage is implemented by parallel convolutions \(W_s^1\), sampling by further convolutions \(W_s^2\), and reconstruction by an initial \(1\times1\) pseudo-inverse-like layer followed by a five-layer enhancement module and a Multi-Level Wavelet CNN. The system is trained in three phases with losses \(L_{\mathrm{init}}\), \(L_{\mathrm{enh1}}\), and \(L_{\mathrm{enh2}}\) [2008.00802].

MsDCNN addresses compressed sensing reconstruction with a fully-convolutional measurement operator and a Multi-scale Feature Extraction module. In the MFE, several parallel dilated-convolution branches with rates such as \(d_1=1\), \(d_2=2\), \(d_3=3\) extract multi-scale features from the same feature map, concatenate them, and fuse them by a \(3\times3\) convolution [2209.13761].

Scale-cascaded diffusion models for medical super-resolution make the decomposition explicit. Three separate diffusion priors are trained: an unconditional prior for the coarsest band \(x^{(3)}\), a conditional prior for \(x^{(2)}\mid U(x^{(3)})\), and a conditional prior for \(x^{(1)}\mid U(x^{(2)})+U^2(x^{(3)})\). Posterior sampling proceeds from coarse to fine, using scale-specific data-consistency steps [2601.23201].

Other domains adopt analogous ideas. SurfR precomputes per-cell features on parallel multi-scale grids with scales \(\{1,4,16\}\), fuses query-time features across scales by a Transformer encoder, and regresses sign logits and magnitude for the SDF [2506.08635]. AMD decomposes each time-series channel by repeated average-pooling, performs residual coarse-to-fine mixing, models temporal and channel dependencies, and uses adaptive multi-predictor synthesis to combine expert forecasts [2406.03751]. VSRNN reconstructs unresolved physics as
\[
u'_{\mathrm{VSRNN}}(y)=\sum_{j=1}^{N_s}\alpha_j(\bar u;\alpha)\,\psi_j(y),
\]
mirroring the variational multiscale expansion of the fine scales [2101.09839].

## 5. Empirical performance across domains

The empirical record is heterogeneous but substantial. In masked image modeling, LocalMIM reports that on ViT-B/16 with 100 pre-train epochs it reaches MAE’s \(83.3\%\) fine-tune accuracy in \(\sim70\) GPU-hours versus MAE’s \(\sim220\) GPU-hours. On Swin-B with 100 pre-train epochs, it reaches equivalent SimMIM accuracy \((84.0\%)\) in 100 GPU-hours versus 360 GPU-hours. For ADE20K semantic segmentation with UperNet and ViT-B, MAE at 1600 epochs gives \(48.1\) mIoU, whereas LocalMIM-HOG at 1600 epochs and 1120 GPU-h gives \(49.5\) mIoU. For COCO detection and segmentation with Mask R-CNN and Swin-B, LocalMIM-HOG at 400 epochs gives \(50.7\) box AP and \(44.9\) mask AP [2303.05251].

In compressed sensing, MsDCNN-3 achieves mean PSNR \(26.43\) dB at MR=\(0.10\), \(23.96\) dB at MR=\(0.04\), and \(20.22\) dB at MR=\(0.01\), versus DR2-Net’s \(24.32/20.80/17.44\) dB and ReconNet’s \(22.68/19.99/17.27\) dB; SSIM is improved similarly by \(\sim0.05\) [2209.13761]. MS-DCI reports, on six \(512\times512\) test images, that SS-DCI\(^3\) reaches \(30.51/0.839\) at rate \(0.10\), \(33.69/0.901\) at rate \(0.20\), and \(35.92/0.931\) at rate \(0.30\), outperforming CSNet and S-CSNet in the reported comparisons [2008.00802].

For medical super-resolution, the three-level scale-cascaded diffusion model reports, on held-out FastMRI Brain, Knee, and Prostate slices, \(31.69\) PSNR, \(0.940\) SSIM, and \(0.16\) LPIPS for \(4\times\) SR with the 3-level cascade, compared with \(24.25/0.810/0.23\) for DiffPIR and \(23.48/0.780/0.24\) for Multi-Grid PnP; inference speed is improved by \(\approx35\%\) because coarser scales use smaller images and smaller networks [2601.23201].

In 3D full waveform inversion, TT-3DIFWI with M-SSIM reports GPU memory for INR training dropping from 98 GB to \(<17\) MB on the synthetic Overthrust tests, and achieves the lowest absolute error maps and clearest stratigraphic interfaces among the compared methods [2606.22867]. In surface reconstruction, SurfR reports average inference times on Thingi10K of \(0.2\) s at \(64^3\), \(1.8\) s at \(128^3\), and \(7.0\) s at \(256^3\); at \(256^3\), POCO takes \(\approx51\) s and P2S \(\approx232\) s, while average Chamfer is \(2.1\) for SurfR and \(1.9\) for POCO [2506.08635].

In rotating turbulence inpainting, GPOD, EPOD, and GAN exhibit a nontrivial trade-off: EPOD and GAN achieve lower RMSE than GPOD for small and medium centered-square gaps, yet “the non-linear GAN does not outperform one of the linear POD techniques” in point-wise reconstruction, whereas the GAN better reproduces multi-scale statistics, non-Gaussian tails, and extreme events [2210.11921].

## 6. Recurring limitations, artifacts, and methodological tensions

Several recurrent issues structure the field. First, band-limited decompositions may introduce artifacts at sharp transitions. In astronomical maps, wave transforms can produce negative ringing around sharp edges; the constrained-diffusion method was designed specifically so that, for \(f\ge0\), each band remains non-negative and artifacts around discontinuities are absent [2201.05484]. In detail enhancement, the SwV filter emphasizes a gradient-preserving property, claiming that no gradient-reversal or halo artifact can occur because the filter never overshoots local means [2105.01951].

Second, multiscale structure does not remove the need for careful regularization design. Multi-scale low-rank matrix decomposition notes that block partitions break translation invariance and can create block seams; cycle spinning is introduced to reduce these blocking artifacts [1507.08751]. In blind deconvolution, the MHDM approach produces comparable results to a single-step variational method and a non-blind MHDM while requiring less laborious parameter tuning “at the price of more computations” [2409.08734].

Third, theoretical equivalence across formulations is exceptional rather than generic. Grohs and Wallner show that a general manifold analogue of biorthogonal wavelets cannot be expected to possess perfect reconstruction, except for specific interpolating constructions [1001.1517]. The MHDM literature likewise shows that agreement with single-step Tikhonov regularization requires necessary and sufficient conditions and holds, for instance, in one-dimensional total variation denoising and certain \(\ell^1\)-regularized settings [2304.08332].

Finally, nonlinear models do not dominate linear ones on every criterion. In turbulent-flow reconstruction, POD-based methods remain competitive in point-wise error, while GANs better preserve statistical multi-scale properties and extreme events [2210.11921]. This suggests that “reconstruction quality” in multiscale settings is intrinsically multi-criteria: fidelity to individual samples, faithfulness of scale-resolved statistics, stability of inverse solutions, computational burden, and the interpretability of the recovered components do not necessarily align.

Source: https://www.emergentmind.com/topics/multi-scale-decomposition-reconstruction