Sparse-Plus-Smooth Decomposition

• Sparse-plus-smooth decomposition is a modeling technique that separates signals into a sparse component for high-frequency, localized features and a smooth component for broad, low-frequency trends.
• It employs convex optimization methods like ADMM and proximal gradient to efficiently decouple and recover the signal's distinct features.
• The framework offers robust theoretical guarantees and is applied in diverse areas such as image segmentation, anomaly detection, and medical imaging.

Sparse-plus-smooth decomposition refers to a family of methodologies in inverse problems, signal processing, and machine learning that aim to represent a signal, image, or tensor as the sum of two distinct components: a “sparse” part capturing high-frequency, localized, or singular features, and a “smooth” part capturing low-frequency or slowly varying structures. This paradigm supports interpretable and effective modeling of signals exhibiting structured heterogeneity—such as images containing both broad backgrounds and sharp text, or time-series with smooth baseline and sporadic anomalies. The framework is mathematically grounded, admits efficient numerical solutions, and enables statistical guarantees under appropriate structural conditions.

1. Formal Definitions and Mathematical Frameworks

Let $y$ denote observed data (vector, image, tensor) and $A$ a (possibly identity or measurement) operator. The canonical sparse-plus-smooth model posits

$$y = A(x_{\text{sparse}} + x_{\text{smooth}}) + \text{noise}$$

where

• $x_{\text{sparse}}$ is a sparse vector or function (e.g., few nonzeros, spikes, Dirac measures, $L^1$-norm regularized);
• $x_{\text{smooth}}$ is a smooth function (e.g., lies in a low-dimensional basis, penalized by Sobolev/TV/$\ell_2$-norm).

The most widely used instances are convex composite problems: $$\min_{x_{\text{sparse}}, x_{\text{smooth}}} \frac{1}{2}\|y - A(x_{\text{sparse}} + x_{\text{smooth}})\|_2^2 + \lambda_1\|L_1 x_{\text{sparse}} \|_1 + \lambda_2\|L_2 x_{\text{smooth}}\|_2^2$$ with operator choices:

• $L_1 = I$ for direct sparsity, or a higher-order difference for group/fused-sparsity,
• $L_2$ a (possibly high-order) difference/Laplacian or a kernel operator for smoothness (Jarret et al., 2024, Jarret et al., 27 Oct 2025, Debarre et al., 2021).

Generalizations further exist:

• Atomic/Banach-norm penalty for sparsity, such as TV or nuclear norm (Jarret et al., 27 Oct 2025).
• Low-rank and/or graph-smooth tensor structure (Sofuoglu et al., 2020, Peng et al., 2022).
• Mixed-data models: Poisson-likelihood for counts (Zhao et al., 2022).

2. Algorithmic Approaches and Computational Schemes

Sparse-plus-smooth decompositions are commonly optimized using first-order convex optimization techniques and operator splitting methods:

• Alternating Direction Method of Multipliers (ADMM): The data-fidelity and non-smooth terms are split, permitting efficient block-wise updates. For unconstrained forms, one commonly introduces auxiliary variables and uses soft-thresholding/shrinkage for the sparse part and quadratic solvers for the smooth part (Minaee et al., 2015, Sofuoglu et al., 2020, Peng et al., 2022, Debarre et al., 2021).
• Proximal Gradient and Accelerated First-Order Methods: Particularly effective for large-scale or compressed data settings. Soft-thresholding for $A$0-terms and closed-form/projection for quadratic (smooth) penalties dominate per-iteration cost (Jarret et al., 2024, Mou et al., 2022).
• Decoupling/Representer Theorems: Recent work shows that, for a significant class of quadratic data-fidelity + strongly convex smoothness penalty, the problem decouples: first, solve a “whitened” sparse inverse problem, then compute the smooth part in closed form (typically via a single linear solve) (Jarret et al., 27 Oct 2025, Jarret et al., 2024). This procedure yields substantial computational savings and is exact under mild conditions.
• Patch-wise or Local Block Decompositions: Employed for images and tensors; e.g., screen-content image segmentation applies blockwise DCT bases for smooth background with a pixelwise sparse model for foreground (Minaee et al., 2015), while patch-based models further permit supervised dictionary learning (Ducotterd et al., 2024).

Computational complexity is dominated by the solution of small to moderate linear systems and the cost of applying basis transforms; the separation enables almost linear scaling and accelerates compared to joint, non-decoupled solvers (Minaee et al., 2015, Jarret et al., 2024, Jarret et al., 27 Oct 2025).

3. Theoretical Guarantees and Recovery Properties

Sparse-plus-smooth decomposition frameworks admit rigorous theoretical analysis, with guarantees depending on model structure and incoherence assumptions:

• Exact Decomposition: For matrix or tensor data, if the sparse and smooth (or low-rank + TV) parts are incoherent and the sparse part is sufficiently sparse, then convex programs (with nuclear, TV, and $A$1 penalties) provably recover the true components (Peng et al., 2022). For compressed measurements, restricted isometry conditions on measurement matrices ($A$2) yield stable recovery (Mou et al., 2022).
• Representer Theorems: For continuous or infinite-dimensional variants, composite representer theorems characterize all minimizers as sums of a discrete atomic (sparse) part and a “spline” smooth part (e.g., $A$3-spline + $A$4-spline) with explicit coefficients determined by measurements (Debarre et al., 2021, Jarret et al., 27 Oct 2025).
• Statistical Risk and Model Selection: Regularization parameters ($A$5) can be tuned by cross-validation, maximum-likelihood, or risk estimation. In practice, simple scaling ($A$6) is often robust (Peng et al., 2022). Empirical studies consistently show superior support recovery and mean-squared error relative to strictly sparse or strictly smooth models (Minaee et al., 2015, Ducotterd et al., 2024, Atamturk et al., 2018).

4. Applications Across Domains

Sparse-plus-smooth decomposition is central in diverse application areas:

Domain	Example Application	Reference
Image Segmentation	Text-background separation in screen content	(Minaee et al., 2015)
Anomaly Detection	Surface defect detection, spatiotemporal urban traffic	(Mou et al., 2022, Sofuoglu et al., 2020)
Medical Imaging	Compressed-sensing MRI, denoising, super-resolution	(Ducotterd et al., 2024, Mou et al., 2022)
Time Series/Change Detection	Online change-point detection in high-dim data	(Guo et al., 2020)
Signal Deconvolution	Diracs over smooth background, continuous inverse prob.	(Jarret et al., 27 Oct 2025, Jarret et al., 2024, Debarre et al., 2021)

These methods support interpretable model decompositions and improved quantitative metrics such as precision/recall (segmentation), AUC (anomaly detection), and PSNR/SSIM (reconstruction). For example, sparse-plus-smooth segmentation substantially outperforms rule-based or clustering screen-content segmenters in both recall and precision, especially for thin strokes and low-contrast features (Minaee et al., 2015).

5. Extensions, Generalizations, and Connections

Sparse-plus-smooth modeling connects and extends several classical frameworks:

• Sparse Functional PCA: Unified approaches to principal component analysis add both sparsity and smoothness regularization to left and right singular vectors, recovering both classical functional PCA and sparse PCA as special cases (Allen et al., 2013).
• Low-Rank Plus Smooth Models: Instead of promoting only smoothness, some frameworks enforce low-rankness together with local smoothness (e.g., via correlated total variation regularization of gradient maps), yielding improved exact recovery versus low-rank-only or TV-only decompositions (Peng et al., 2022, Sofuoglu et al., 2020).
• Probabilistic and Bayesian Formulations: Variational inference with spike-and-slab priors yields online adaptive decomposition in streaming or partially observed data (Guo et al., 2020).
• Learning-Based/Dictionary Learning: Bilevel learning frameworks optimize both analysis/synthesis dictionaries and regularizer parameters through empirical risk or supervised learning, combining patch-based decomposition with end-to-end optimization (Ducotterd et al., 2024).

The “decoupling” of sparse and smooth variables for computational and theoretical tractability is a recent advance, supporting speedups and scalability for large-scale and high-dimensional problems (Jarret et al., 27 Oct 2025, Jarret et al., 2024).

6. Practical Implementation and Tuning Considerations

Effective application of sparse-plus-smooth decomposition requires informed choices:

• Basis Selection: Low-frequency DCT, B-spline, or learned dictionaries for the smooth part; identity or localized bases for sparse portion (Minaee et al., 2015, Mou et al., 2022, Ducotterd et al., 2024).
• Parameter Selection: Regularization parameters ($A$7) may be tuned via cross-validation, L-curve, or closed-form “critical $A$8” rules, often normalized w.r.t. measurement or system properties (Jarret et al., 2024, Peng et al., 2022).
• Algorithmic Scalability: For large $A$9, block-wise or patch-wise strategies, Kronecker (separable) extensions, and adaptive block coordinate descent are effective (Mou et al., 2022, Peng et al., 2022, Atamturk et al., 2018).
• Handling Constraints and Priors: Beyond direct sparsity, block/group constraints, support structure, and temporal/spatial smoothness are encoded via structured norms (group-lasso, graph-TV, functional priors) (Sofuoglu et al., 2020, Allen et al., 2013, Atamturk et al., 2018).
• Interpretability: The decomposition yields directly interpretable components (e.g., background/foreground in imaging, local anomalies in spatiotemporal data) with support and structure informed by the regularizers.

7. Summary and Outlook

Sparse-plus-smooth decomposition provides a mathematically rigorous, computationally efficient, and empirically validated framework for separating structured signal components in a wide array of domains. The combination of convex modeling, decoupled algorithms, explicit representer theorems, and robust theoretical guarantees enables performance unattainable by monolithic sparse or smooth models. Continued development occurs along axes of multi-component models, higher-order tensors, discrete-to-continuous formulations, non-convex surrogates for bias reduction, and task-specific regularization learning (Jarret et al., 27 Oct 2025, Ducotterd et al., 2024, Mou et al., 2022). The approach is central to both the theory and practice of modern signal processing, machine learning, and high-dimensional data analysis.