Constrained IRLS Methods

• CIRLS is a family of optimization methods that replace complex, nonsmooth objectives with iterative, constrained weighted least-squares subproblems ensuring feasibility at every step.
• It applies to varied contexts such as matrix recovery, generalized linear models, and robust regression by leveraging adaptive reweighting strategies tailored to each problem structure.
• The approach incorporates stabilization techniques like epsilon-smoothing and weight clipping to handle singularities while offering theoretical guarantees from local convergence to exact recovery.

Searching arXiv for CIRLS and related IRLS papers to ground the article in published work. arXiv search: constrained iteratively reweighted least squares CIRLS generalized linear models matrix recovery image recovery IRLS. Constrained iteratively-reweighted least squares (CIRLS) denotes a family of optimization methods in which a nonsmooth, nonquadratic, or nonconvex objective is replaced by a sequence of weighted quadratic subproblems, with feasibility enforced inside each update rather than recovered only at the limit. In its strict form, the method solves a constrained weighted least-squares or quadratic program at every iteration; representative constraints include affine feasibility conditions such as $Ax=b$ or $\mathcal L(X)=y$, linear inequality systems $l\le C\beta\le u$, simplex constraints, and orthogonality constraints. The label is used explicitly in generalized linear models and simplicial-simplicial regression, but the same structural pattern appears earlier in matrix recovery, graph min-cut, low-rank tensor recovery, and other inverse problems, often without the acronym itself (Masselot et al., 22 Sep 2025, Tsagris, 17 Nov 2025, Fornasier et al., 2010, Zhu et al., 2015, Krämer, 2021).

1. Canonical optimization template

At the most abstract level, CIRLS alternates between two operations. First, it constructs weights from the current iterate, typically from residuals, coefficients, singular values, or fitted probabilities. Second, it solves a quadratic subproblem over a feasible set. This yields the basic constrained pattern

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$

or an equivalent weighted least-squares form, with $R^{(k)}$ and $q^{(k)}$ depending on the previous iterate.

Several important instances fit this template exactly.

Setting	Constrained objective	Per-iteration subproblem
Matrix recovery	$\min_X \|X\|_* \text{ s.t. } S(X)=M$	$\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$
$\ell_p$ regression	$\min_{x:Ax=b}\|x\|_p$	$\mathcal L(X)=y$0
Constrained GLM	$\mathcal L(X)=y$1	constrained IRLS quadratic program
Simplicial regression	maximize $\mathcal L(X)=y$2 with row-simplex $\mathcal L(X)=y$3	constrained weighted least-squares quadratic program
$\mathcal L(X)=y$4-mincut relaxation	$\mathcal L(X)=y$5	$\mathcal L(X)=y$6

In low-rank matrix recovery, the constrained step is

$\mathcal L(X)=y$7

with weights derived from the current singular spectrum, so each iterate is the weighted least-squares projection onto the affine feasible set (Fornasier et al., 2010). In affine $\mathcal L(X)=y$8 regression, the core energy is

$\mathcal L(X)=y$9

so feasibility is again enforced exactly at every iteration (Ene et al., 2 Oct 2025). In constrained GLMs, IRLS is converted into

$l\le C\beta\le u$0

which is a quadratic program rather than an unconstrained normal-equation solve (Masselot et al., 22 Sep 2025). In transformation-free simplicial-simplicial regression, the coefficient matrix $l\le C\beta\le u$1 is row-stochastic,

$l\le C\beta\le u$2

and each identity-link logistic-type update is solved under those simplex constraints (Tsagris, 17 Nov 2025). In graph min-cut, the continuous relaxation

$l\le C\beta\le u$3

induces a constrained quadratic Laplacian solve at each reweighting step (Zhu et al., 2015).

This suggests that CIRLS is better understood as an algorithmic template than as a single model class.

2. Reweighting mechanisms and surrogate construction

The central technical device in CIRLS is the replacement of a difficult objective by a weighted quadratic surrogate. The form of the weights depends on the structure being promoted.

For sparse vector models, classical IRLS replaces $l\le C\beta\le u$4 or $l\le C\beta\le u$5 by a quadratic term with coefficient

$l\le C\beta\le u$6

so small coordinates receive larger weights at the next step. This is the core mechanism in generalized sparse regularization for $l\le C\beta\le u$7, where the algorithm minimizes a smoothed surrogate and converges to a minimizer of the original convex objective (Voronin et al., 2015).

For low-rank matrix recovery, the weights are spectral rather than coordinatewise. If

$l\le C\beta\le u$8

then stabilized matrix IRLS uses

$l\le C\beta\le u$9

so reweighting occurs in singular directions rather than in ambient coordinates (Fornasier et al., 2010). Tensor IRLS-$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$0 generalizes the same idea to a family of matricizations $$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$1, with weights

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$2

corresponding to a log-det sum-of-ranks surrogate (Krämer, 2021).

In constrained GLMs, the weights are the standard IRLS working weights, but the distinction is that the weighted least-squares solve is performed over the feasible set. The quadratic subproblem is written explicitly as

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$3

so CIRLS is literally “IRLS plus constraints” (Masselot et al., 22 Sep 2025). In transformation-free simplicial-simplicial regression, the identity-link logistic reinterpretation yields weights

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$4

and the next iterate is obtained from a constrained quadratic program in $$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$5 (Tsagris, 17 Nov 2025).

Some related robust-regression variants show how reweighting can be stabilized. Stagewise truncated IRLS for robust linear regression uses clipped residual weights

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$6

and increases $$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$7 across stages, thereby globalizing a basic IRLS heuristic (Mukhoty et al., 2020). In robust kernel regression with the $$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$8-loss, the induced weight is

$$x^{(k+1)}\in\arg\min_{x\in\mathcal C} \; x^\top R^{(k)}x - 2(q^{(k)})^\top x,$$9

and the LS-SVR subproblem remains an equality-constrained weighted least-squares system (Dong et al., 2019).

3. Constraint architectures

What distinguishes CIRLS from generic IRLS is not reweighting alone but the fact that feasibility is built into the subproblem.

The simplest architecture is affine equality feasibility. In matrix recovery, every iterate lies in

$R^{(k)}$0

and the weighted step is the exact weighted projection onto that affine set (Fornasier et al., 2010). In affine $R^{(k)}$1 regression, every iterate satisfies $R^{(k)}$2, and the closed-form constrained minimizer is

$R^{(k)}$3

so exact feasibility is preserved throughout (Ene et al., 2 Oct 2025). In graph min-cut, the source/sink conditions are fixed by

$R^{(k)}$4

and each weighted step becomes a reduced Laplacian solve with those boundary values enforced exactly (Zhu et al., 2015). Tensor IRLS-$R^{(k)}$5 likewise preserves the affine constraint $R^{(k)}$6 in the full-space method (Krämer, 2021).

A second architecture consists of linear inequality and mixed equality/inequality constraints. Constrained GLMs use

$R^{(k)}$7

which includes equality constraints as the special case $R^{(k)}$8, as well as nonnegativity, monotonicity, and sum-to-zero restrictions (Masselot et al., 22 Sep 2025).

A third architecture is simplex feasibility. In transformation-free simplicial-simplicial regression, the coefficient matrix is row-stochastic: $R^{(k)}$9 The important point is that the simplex constraints are enforced inside the weighted least-squares step rather than by projecting an unconstrained estimate afterward (Tsagris, 17 Nov 2025).

A fourth architecture is manifold-type feasibility. In $q^{(k)}$0-norm PCA, the loading matrix satisfies

$q^{(k)}$1

and each reweighted subproblem is solved by an ordinary $q^{(k)}$2-PCA oracle on a weighted data matrix, so the orthogonality constraint is handled exactly by the subroutine (Park et al., 2016).

A common misconception is that any reweighted least-squares method for inverse problems is automatically CIRLS. This is not the case. The imaging methods based on weighted $q^{(k)}$3 and weighted Schatten-$q^{(k)}$4 priors solve

$q^{(k)}$5

which is a penalized variational problem, not an explicitly constrained one. Algorithmically these methods are very close to CIRLS, because they solve reweighted quadratic subproblems derived by majorization-minimization, but formally they are penalized IRLS frameworks rather than strict feasibility-constrained CIRLS (Lefkimmiatis et al., 2023, Koshelev et al., 2023).

4. Theoretical guarantees

The theory of CIRLS is heterogeneous: some settings admit exact recovery and complexity theorems, whereas others offer only descent to stationary points or local convergence.

For matrix recovery, the strongest results are phrased through null-space conditions. If the measurement operator satisfies the rank null space property (RNSP), nuclear-norm minimization recovers every rank-$q^{(k)}$6 matrix uniquely; under the stronger SRNSP, stable recovery bounds are obtained in terms of the best rank-$q^{(k)}$7 approximation error

$q^{(k)}$8

The IRLS-M iteration satisfies monotonic descent,

$q^{(k)}$9

and asymptotic regularity,

$\min_X \|X\|_* \text{ s.t. } S(X)=M$0

Under SRNSP and a suitable continuation rule for $\min_X \|X\|_* \text{ s.t. } S(X)=M$1, the iterates converge either to the unique nuclear-norm minimizer or to a minimizer of the smoothed problem (Fornasier et al., 2010).

For affine $\min_X \|X\|_* \text{ s.t. } S(X)=M$2 regression, recent primal-dual IRLS theory provides certificate-based complexity bounds rather than only asymptotic convergence. The main problem

$\min_X \|X\|_* \text{ s.t. } S(X)=M$3

is solved through repeated constrained weighted least-squares subproblems, with a dual invariant controlling energy growth. In the high-precision regime, the method solves

$\min_X \|X\|_* \text{ s.t. } S(X)=M$4

subproblems, each requiring

$\min_X \|X\|_* \text{ s.t. } S(X)=M$5

linear system solves, matching the best known asymptotic bounds for this problem class (Ene et al., 2 Oct 2025).

For constrained GLMs, the paper interprets CIRLS as a special case of sequential quadratic programming. Because the constraints are linear, the SQP constraint linearization is exact, and the method inherits fast local convergence behavior. With canonical links, the constrained log-likelihood is concave in many cases, which implies a unique optimum within the feasible region (Masselot et al., 22 Sep 2025).

For tensor sum-of-ranks minimization, the strongest statement concerns asymptotic minimizers of the log-det surrogate. If $\min_X \|X\|_* \text{ s.t. } S(X)=M$6 globally minimizes $\min_X \|X\|_* \text{ s.t. } S(X)=M$7 over $\min_X \|X\|_* \text{ s.t. } S(X)=M$8 and $\min_X \|X\|_* \text{ s.t. } S(X)=M$9 as $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$0, then $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$1 minimizes the sum of matricization ranks subject to the affine constraint. The exact IRLS-$\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$2 iteration also satisfies monotonicity and local convergence statements, but the relaxed alternating AIRLS variant has only a descent result at present (Krämer, 2021).

By contrast, several algorithmically adjacent methods have weaker guarantees. Online reweighted least squares for sparse recovery is motivated by a constrained $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$3-ball formulation but implemented through a penalized objective, and the paper provides no theorem-level convergence guarantees for the outer reweighting sequence (Mukherjee et al., 2017). Penalized imaging IRLS methods prove convergence to stationary points, not global minimizers, because the relevant weighted $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$4 and weighted Schatten-$\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$5 objectives are generally nonconvex (Koshelev et al., 2023).

5. Applications and computational structure

CIRLS has appeared in a wide range of application domains because the same reweight–solve cycle adapts to different constraint sets and structural priors.

In low-rank recovery and matrix completion, the weighted subproblem often has an explicit affine projection formula. For matrix completion, the columnwise structure allows

$\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$6

and the Woodbury identity reduces the computational burden from a large measurement-space inversion to an $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$7 system, where $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$8 is the effective rank (Fornasier et al., 2010).

In graph optimization, the undirected $\min_{S(X)=M}\|(W^\ell)^{1/2}X\|_F^2$9-mincut problem can be relaxed to

$\ell_p$0

and the CIRLS step becomes a reduced Laplacian solve. This gives an electrical-flow interpretation of each weighted least-squares update and enables parallel sparse linear algebra implementations (Zhu et al., 2015).

In generalized linear modeling, CIRLS turns each IRLS step into a quadratic program. This is especially useful when constraints encode nonnegativity, order, compositional balance, or high-collinearity structure, because the coefficient vector remains feasible throughout the optimization (Masselot et al., 22 Sep 2025).

In compositional data analysis, transformation-free linear simplicial-simplicial regression replaces the earlier EM algorithm by CIRLS. The result is a sequence of convex quadratic programs for the row-simplex constrained coefficient matrix, and the paper reports runtime improvements from roughly $\ell_p$1 to $\ell_p$2 while producing KL objective values within $\ell_p$3 of the EM solutions (Tsagris, 17 Nov 2025).

In robust kernel regression, the LS-SVR variant is particularly close to CIRLS because the underlying model already uses equality constraints. Each $\ell_p$4-loss update solves a weighted kernel LS-SVR linear system with a diagonal matrix $\ell_p$5 that depends on the current residuals (Dong et al., 2019).

In $\ell_p$6-norm PCA, the residual-based reweighting is rowwise and the constrained subproblem is handled by solving a standard $\ell_p$7-PCA problem on a weighted data matrix. This is a constrained IRLS pattern under the Stiefel constraint $\ell_p$8 (Park et al., 2016).

A broader implication is that CIRLS is often successful when the weighted quadratic subproblem inherits exploitable structure: Laplacian systems, kernel linear systems, reduced-rank spectral problems, or block-diagonal quadratic programs.

6. Limitations, terminology, and recurring numerical issues

Several limitations recur across the literature.

First, the term “CIRLS” is used inconsistently. Some papers use it explicitly for constrained GLMs and simplicial regression (Masselot et al., 22 Sep 2025, Tsagris, 17 Nov 2025). Earlier works with identical structure, such as constrained matrix IRLS and graph min-cut IRLS, do not use the acronym even though the per-iteration update is an exact constrained weighted least-squares solve (Fornasier et al., 2010, Zhu et al., 2015). Conversely, many inverse-imaging papers are very close algorithmically but remain penalized IRLS rather than strict CIRLS (Lefkimmiatis et al., 2023, Koshelev et al., 2023).

Second, feasibility preservation does not by itself eliminate bias or inferential complications. In constrained GLMs, the simulations show that constraints introduce bias while decreasing estimator variance; the resulting trade-off can improve RMSE when the constraints are chosen appropriately. The same study also reports error in variance estimation and coverage for the proposed truncated-multivariate-normal inference scheme (Masselot et al., 22 Sep 2025).

Third, weight singularities are endemic. Different papers address them by different smoothing devices: $\ell_p$9-stabilization in matrix IRLS, $\min_{x:Ax=b}\|x\|_p$0 in weighted $\min_{x:Ax=b}\|x\|_p$1 and Schatten-$\min_{x:Ax=b}\|x\|_p$2 imaging priors, $\min_{x:Ax=b}\|x\|_p$3 in online sparse recovery, and clipping $\min_{x:Ax=b}\|x\|_p$4 in robust regression (Fornasier et al., 2010, Lefkimmiatis et al., 2023, Mukherjee et al., 2017, Mukhoty et al., 2020). This suggests that numerical stabilization is not an optional refinement but a core design choice.

Fourth, large-scale variants often relax the exact CIRLS template. AIRLS-$\min_{x:Ax=b}\|x\|_p$5 for tensors replaces exact affine feasibility by a penalized residual term and restricts each step to a tensor-network subspace; the paper proves descent, but not the full convergence theory available for the exact constrained algorithm (Krämer, 2021). A similar gap appears in online reweighted least squares, where the motivating constrained formulation is converted into a penalized surrogate for computation (Mukherjee et al., 2017).

Finally, a common misconception is that CIRLS always means coordinatewise diagonal reweighting. This is true for many sparse models, but false in low-rank and tensor settings, where weights are matrix-valued, SVD-dependent, or induced by multiple matricizations (Fornasier et al., 2010, Krämer, 2021). In that sense, modern CIRLS extends well beyond the classical picture of diagonal residual weights.

Constrained iteratively-reweighted least squares is therefore best viewed as a family of structure-exploiting algorithms for constrained optimization, unified by a simple principle: replace the difficult objective by a locally weighted quadratic, solve the resulting constrained least-squares problem exactly or to high accuracy, update the weights, and repeat. The mathematical consequences of that principle depend strongly on the constraint geometry, the surrogate construction, and the stabilization strategy.