Q3R: Quadratic Reweighted Rank Regularizer

• Q3R is a non-convex rank surrogate that replaces the discontinuous rank objective with a smoothed log-determinant function, efficiently promoting low-rank structure.
• It underpins IRLS algorithms that employ quadratic majorization, ensuring rapid, locally quadratic convergence and superior statistical estimation in matrix recovery and completion.
• The method leverages SVD-based reweighting and adaptive smoothing to balance computational efficiency with robust performance in structured recovery and deep model training.

The Quadratic Reweighted Rank Regularizer (Q3R) is a non-convex rank surrogate and majorization technique designed for inducing and controlling low-rank structure in matrix optimization problems. Q3R underpins a family of iteratively reweighted least squares (IRLS) algorithms for matrix recovery, structured matrix completion, and parameter-efficient training of large-scale deep models. The approach centers on majorizing a smoothed log-determinant function of singular values with a quadratic surrogate, achieving scalable, accurate, and locally quadratic convergence properties essential for both statistical estimation and modern machine learning applications (Ghosh et al., 6 Nov 2025, Kümmerle et al., 2023, Kümmerle et al., 2018).

1. Smoothed Log-Determinant Surrogate for Rank

Q3R replaces the discontinuous matrix rank objective with the smooth but non-convex $\varepsilon$-smoothed log-determinant surrogate. For a matrix $X$ with singular values $\sigma_1(X),..., \sigma_d(X)$, and $\varepsilon>0$, the surrogate takes the form

$$\varphi_{\varepsilon}(X) = \sum_{i=1}^{d} \log(\sigma_i(X) + \varepsilon),$$

or, in alternative Q3R formulations,

$$F_\varepsilon(X) = \sum_{i=1}^d f_\varepsilon(\sigma_i(X)),$$

where $f_\varepsilon(t)$ is piecewise defined: quadratic near zero ($\frac{1}{2}t^2$ for $|t| \leq \varepsilon$), and logarithmic outside ($$\frac{1}{2}\varepsilon^2 \log(e t^2 / \varepsilon^2)$$ for $X$0). This surrogate heavily penalizes small singular values, efficiently promoting low-rank structure for optimization and learning objectives (Ghosh et al., 6 Nov 2025, Kümmerle et al., 2023, Kümmerle et al., 2018).

2. Quadratic Majorization and IRLS Schemes

Q3R deploys a quadratic majorizer for the smoothed log-determinant, making the non-convex problem tractable with iterative updates.

At each IRLS iteration $X$1, given a current point $X$2 (or $X$3), the algorithm computes a quadratic expansion at $X$4, majorizing the surrogate: $X$5 where $X$6 is the reweighting operator built from the leading $X$7 singular vectors of $X$8. The explicit form exploits the SVD structure to provide tight local curvature information. Constants and linear terms can be omitted for minimization, yielding a simple weighted least squares update centered at the current iterate (Ghosh et al., 6 Nov 2025, Kümmerle et al., 2023).

This procedure is closely aligned with majorization–minimization (MM) principles and Newton-type methods, resulting in provable monotonic decrease of the surrogate and rapid (locally quadratic) convergence under suitable conditions.

3. Algorithmic Structure and Pseudocode

The Q3R-IRLS methodology iteratively alternates between:

• Computation of the quadratic weight operator: Extract leading singular vectors of the current iterate to construct $X$9, which encodes per-mode penalties as functions of current singular values and the smoothing parameter $\sigma_1(X),..., \sigma_d(X)$0.
• Solving a quadratic subproblem: Minimize the quadratic surrogate, optionally subject to additional constraints (e.g., $\sigma_1(X),..., \sigma_d(X)$1 in recovery, or adding observed loss for deep learning).
• Updating the smoothing parameter: $\sigma_1(X),..., \sigma_d(X)$2, progressively shrinking the penalization region and enforcing rank targets.
• Repeating until convergence.

A representative pseudocode for AdamQ3R (for low-rank neural network training) involves periodically recomputing $\sigma_1(X),..., \sigma_d(X)$3 using truncated SVD, and interleaving this with adaptive moment optimization steps (Ghosh et al., 6 Nov 2025). For matrix recovery, the algorithm involves alternating quadratic minimization and reweighting, with monotonic convergence guarantees (Kümmerle et al., 2023, Kümmerle et al., 2018).

4. Theoretical Properties: Convergence and Sample Complexity

Q3R-IRLS methods possess strong theoretical guarantees:

• Majorization–minimization property: Each IRLS iteration decreases the value of the smoothed surrogate, ensuring monotonic progress toward a stationary point (Ghosh et al., 6 Nov 2025, Kümmerle et al., 2018).
• Locally quadratic convergence: If the measurement operator $\sigma_1(X),..., \sigma_d(X)$4 satisfies a restricted isometry property (RIP) and the iterate approaches the true solution $\sigma_1(X),..., \sigma_d(X)$5, the error at each step contracts quadratically: $\sigma_1(X),..., \sigma_d(X)$6 with explicit constants depending on problem details (Kümmerle et al., 2023).
• Near-optimal sample complexity: For random Gaussian measurements, recovery is possible with

$\sigma_1(X),..., \sigma_d(X)$7

matching, up to logarithmic factors, the information-theoretic limits for simultaneous rank- and row-sparsity (Kümmerle et al., 2023).

A key advantage is local quadratic convergence rate, which is not attained by convex relaxations such as the nuclear norm.

5. Applications: Structured Recovery and Deep Model Training

The Q3R methodology has demonstrated strong empirical and statistical performance across diverse domains:

• Simultaneous low-rank and sparse matrix recovery: Accurate reconstruction from minimal measurements; Q3R-IRLS achieves superior empirical success rates relative to nuclear-norm or log-sum relaxations, particularly in regimes with heterogeneous structures (Kümmerle et al., 2023).
• Structured low-rank matrix completion: StrucHMIRLS, a structured Q3R-IRLS variant, efficiently recovers low-rank Hankel and Toeplitz matrices for tasks such as spectral super-resolution, achieving high success rates near the minimal sampling limit (Kümmerle et al., 2018).
• Low-rank training of deep models: AdamQ3R applies the Q3R principle to large-scale neural network training, retaining high predictive accuracy while enforcing low target ranks. For example, ViT-Tiny models pruned to 40% parameter retention incur less than 1.3% accuracy loss on CIFAR-10. Comparable benchmarks show superior or comparable task accuracy to dense or other parameter-efficient baselines, including LoRA and LoRITa, across transformers and LLMs (Ghosh et al., 6 Nov 2025).

6. Computational Aspects and Implementation

Q3R-based IRLS methods feature efficient implementations:

• Structured matrix scenarios: Fast Hankel/Toeplitz products yield per-iteration complexity linear in the size of the input vector and quadratic in rank estimate $\sigma_1(X),..., \sigma_d(X)$8, suitable for large-scale problems (Kümmerle et al., 2018).
• Unstructured cases and deep learning: Memory requirements for the reweighting matrices scale as $\sigma_1(X),..., \sigma_d(X)$9, with each IRLS update for weight matrices of size $\varepsilon>0$0 incurring $\varepsilon>0$1 cost, where $\varepsilon>0$2 is the effective rank (Ghosh et al., 6 Nov 2025).
• Compatibility: The quadratic reweighted regularizer can be applied to any linear or convolutional weight matrix; bias terms are typically not regularized (Ghosh et al., 6 Nov 2025).
• Optimization: The quadratic surrogate integrates seamlessly with first-order optimizers (e.g., Adam), enabling practical training of large, modern architectures with small periodic overhead for recomputing SVD-based weights.

7. Comparative Performance and Empirical Outcomes

Empirical evaluations underscore the statistical efficiency and scalability of Q3R-IRLS schemes. In Hankel matrix completion, StrucHMIRLS recovers the ground truth with nearly minimal samples, outperforming atomic norm and structured nuclear-norm relaxations (Kümmerle et al., 2018). For deep learning tasks, AdamQ3R demonstrates retention of performance under aggressive parameter pruning; e.g., at 40% retention on ViT-Base for CIFAR-100, AdamQ3R matches or exceeds dense and LoRITa models in accuracy. Similar superiority is observed in GLUE benchmark fine-tuning of RoBERTa-base models (Ghosh et al., 6 Nov 2025).

In summary, Q3R regularization achieves practical, scalable, and theoretically grounded low-rank control across both signal processing and modern machine learning settings, furnishing a unifying IRLS-based mechanism with provable and empirical performance advantages (Ghosh et al., 6 Nov 2025, Kümmerle et al., 2023, Kümmerle et al., 2018).