Fast Weighted Split Bregman

• Fast Weighted Split Bregman (FWSB) is an algorithm that efficiently solves convex optimization problems with sparsity-promoting weighted regularization.
• It decouples composite objectives into smooth quadratic and non-smooth weighted penalty components using adaptive active-set updates and fixed-point linear solvers.
• Empirical evaluations show FWSB achieves 3–200× speed-ups in portfolio selection and imaging tasks while preserving solution accuracy.

The Fast Weighted Split Bregman (FWSB) algorithm refers to a family of computational techniques designed to efficiently solve convex optimization problems with sparsity-promoting, weighted regularization penalties. These methods arise prominently in problems of portfolio selection under robust regularization (Ho et al., 2015) and in variational models for imaging such as weighted total variation (TV) denoising and compressed sensing MRI (Lazzaro et al., 2018). FWSB exploits problem structure—sparsity, separability, and diagonal dominance in system matrices—to accelerate classic Split Bregman iterations, providing significant improvements in computation without compromising solution quality.

1. Formal Problem Settings and Weighted Regularization

FWSB is utilized in minimization of composite objectives coupling quadratic data-fidelity terms to weighted $\ell_1$ (and often $\ell_2$) regularization. In the context of portfolio optimization, this takes the weighted elastic-net penalized mean-variance form (Ho et al., 2015): $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$ where $\hat\mu$ and $\hat\Gamma$ are estimated mean and covariance, and $\beta_i$, $\alpha_i$ encode asset-wise risk aversion.

In imaging applications, FWSB targets solutions of: $$\min_{u}~\frac12\|\Phi u - z\|_2^2 + \lambda\|\nabla_x^w u\|_1 + \lambda\|\nabla_y^w u\|_1$$ where $\Phi$ is a linear operator (e.g., blur or sensing matrix), and $\nabla_x^w,\nabla_y^w$ denote weighted finite differences, typically with spatially variable weights to preserve structure in images (Lazzaro et al., 2018).

Weight selection is often guided by either robust statistical uncertainty quantification (bootstrap calibration in finance (Ho et al., 2015)) or data-dependent edge detection (e.g., log–exp weighting in images (Lazzaro et al., 2018)).

2. Split Bregman Formalism and Weighted Variants

Split Bregman methods decompose composite objectives by introducing auxiliary variables to decouple non-smooth regularization from smooth (typically quadratic) terms:

• For finance (Ho et al., 2015), introduce $\ell_2$0 so $\ell_2$1, recasting the $\ell_2$2 problem via constrained minimization.
• For imaging (Lazzaro et al., 2018), employ $\ell_2$3, $\ell_2$4, reformulating the problem with respect to $\ell_2$5, $\ell_2$6, $\ell_2$7.

The augmented Lagrangian for these problems includes quadratic constraint penalties and scaled Bregman (dual) variables, and iterations alternate between smooth quadratic block minimization (e.g., solving linear systems in the main variables) and soft-thresholding (shrinkage) in the auxiliary blocks.

3. Fast Weighted Splitting Schemes

The acceleration in FWSB arises from two principal innovations:

1. Support Expansion (Finance) (Ho et al., 2015): FWSB leverages optimality conditions that link the gradient residual $\ell_2$8 to support identification. Only a small subset of variables typically attain nonzero solutions due to the weighted $\ell_2$9 penalty. By expanding the active set to only include indices $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$0 where $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$1, each Split Bregman iteration is confined to a much lower-dimensional subproblem, and convergence can be certified when all external coordinates satisfy $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$2. This is the "adaptive support" principle.
2. Fixed-Point Linear Solves (Imaging) (Lazzaro et al., 2018): In the imaging context, the principal bottleneck is solving a linear system of the form $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$3 where $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$4 (with $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$5 a weighted Laplacian). FWSB applies a splitting $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$6 (with $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$7, $$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$8) to yield a fixed-point iteration:

$$\min_{w\in\mathbb R^N}~w^T\hat\Gamma w - \hat\mu^T w + \sum_{i=1}^N \beta_i |w_i| + \sum_{i=1}^N \alpha_i w_i^2$$9

under conditions guaranteeing spectral radius $\hat\mu$0—thus enabling global linear convergence at each Split Bregman substep, and exploiting full vectorization for efficiency.

4. Algorithmic Structure and Pseudocode

The following table summarizes the salient algorithmic components of FWSB in both finance and imaging applications:

Component	Finance (Ho et al., 2015)	Imaging (Lazzaro et al., 2018)
Regularizer	Weighted elastic-net	Weighted TV (anisotropic, spatially var.)
Active set update	Residual-optimality based, batch expand	All coordinates (full or block)
Linear solve	Direct (on support)	Fixed-point via Laplacian splitting
Proximal step	Shrinkage per active coord.	Soft/cut threshold on auxiliary gradients
Termination	No violated support optimality	SB/FBS relative tolerance

Pseudocode for the adaptive support FWSB (finance) details initialization, residual computation, batch support expansion, restricted variable prolongation, inner Split Bregman iteration on support, and dual variable update, with clear stopping criteria when no external coordinate violates support optimality or maximum steps are reached. In imaging, the core steps embed an inner FWSB fixed-point solver within an accelerated Forward–Backward (FISTA-like) outer loop, with explicit formulas for coordinate updates, weighted gradient operations, and shrinkage steps.

5. Convergence Properties and Theoretical Guarantees

• Support-optimality: For the weighted elastic-net, Theorem 2.1 (Ho et al., 2015) provides quantitative conditions ensuring that approximate KKT residuals translate to objective suboptimality bounds. Practical versions permit thresholds distinguishing numerically vanishing and nonzero weights.
• Fixed-point convergence: For weighted TV, Theorem 3.2 (Lazzaro et al., 2018) establishes that the fixed-point iteration with $\hat\mu$1 converges linearly when $\hat\mu$2. The convergence rate is governed by the spectral radius $\hat\mu$3, and global convergence holds by the Householder–Johns theorem.
• Global termination: FWSB in finance guarantees at most $\hat\mu$4 outer support expansions; each strictly decreases the objective or improves the support set, ensuring finite convergence.
• Inexact inner solves: In imaging, allowing relaxed tolerances for inner FWSB iterations does not impede outer convergence of the Forward–Backward loop.

6. Empirical Performance and Complexity Analysis

FWSB demonstrates substantial computational advantages over standard and alternative Split Bregman approaches in reported applications.

• Portfolio selection (Ho et al., 2015): For $\hat\mu$5, FWSB achieves up to $\hat\mu$6 speed-up versus vanilla SB and remains competitive even as sparsity decreases. Reported MATLAB timings show that for 100–900 nonzeros, FWSB requires $\hat\mu$7–$\hat\mu$8 seconds (SB: $\hat\mu$9–$\hat\Gamma$0 sec; FISTA: $\hat\Gamma$1–$\hat\Gamma$2 sec).
• Imaging (Lazzaro et al., 2018): In image deblurring (e.g., $\hat\Gamma$3), FWSB solves inner subproblems in $\hat\Gamma$4–$\hat\Gamma$5 seconds versus $\hat\Gamma$6–$\hat\Gamma$7 for Gauss–Seidel. In MRI (e.g., $\hat\Gamma$8), FWSB completes in $\hat\Gamma$9–$\beta_i$0 seconds (GS: $\beta_i$1–$\beta_i$2).
• Scalability and resource utilization: FWSB exploits sparsity and vectorization to minimize computational burden, especially in high-dimensional or highly sparse regimes.

Observed numerical results show that FWSB achieves equivalent or slightly improved signal fidelity (e.g., peak signal-to-noise ratio in imaging) relative to baseline methods, while consistently reducing runtime by factors of 3–200, depending on application and sparsity.

7. Applications and Extensions

FWSB is directly applied to:

• Robust mean-variance portfolio design via adaptive elastic-net regularization incorporating parameter uncertainty (Ho et al., 2015).
• Weighted TV-based image restoration problems, including edge-preserving deblurring and compressed sensing MRI, where spatial adaptation of the regularizer is critical (Lazzaro et al., 2018).

The combination of weighted regularization, inner-outer algorithmic splitting, and sparsity-exploiting active-set adaptation positions FWSB as an efficient tool in high-dimensional, convex, weighted-regularized minimization. Its principles may extend to other settings where structured sparsity and fast, scalable solvers are essential.