MM-RoPE: Accelerated MM for Penalized Regression

• MM-RoPE is a class of MM algorithms that integrates SQUAREM acceleration to improve convergence in high-dimensional penalized regression and estimation.
• It employs quadratic majorization and adaptive extrapolation methods to reduce the number of iterations dramatically, often cutting evaluations by orders of magnitude.
• Its plug‐and‐play design requires minimal tuning while preserving monotonic convergence, making it broadly applicable in statistical computing and machine learning.

MM-RoPE refers primarily to Majorize-Minimize (MM) algorithms applied in conjunction with the SQUAREM acceleration framework to high-dimensional estimation and penalized regression problems, as described in the context of the SQUAREM package and related algorithmic literature (Du et al., 2018). In this setting, MM-RoPE represents a specific class of MM algorithms—penalized regression or estimation problems—where slow, monotone iterative updates can be substantially accelerated via the SQUAREM scheme, leading to significant practical speedups in convergence without sophisticated problem-specific tuning.

1. Foundations of MM Algorithms and Penalized Estimation

MM algorithms solve optimization problems by iteratively constructing and minimizing a surrogate function that locally majorizes the actual objective. At each iteration, given the current parameter estimate $\theta^{(k)}$, the algorithm generates a new estimate via

$$\theta^{(k+1)} = \operatorname{argmin}_\theta\, g(\theta | \theta^{(k)})$$

where $g(\cdot | \theta^{(k)})$ is a surrogate function satisfying $g(\theta | \theta^{(k)}) \geq f(\theta)$ (the original objective), and $g(\theta^{(k)} | \theta^{(k)}) = f(\theta^{(k)})$. In penalized regression (e.g., Lasso, logistic regression with penalty, or high-dimensional regularized problems), the MM update often takes the form of quadratic majorization, leveraging second-order approximations or uniform bounds for efficient computation.

Such iterative updates commonly exhibit fixed-point structure and monotone convergence, making them suitable in principle for direct acceleration.

2. SQUAREM Acceleration: Principle and Update Scheme

SQUAREM (Squared Iterative Method) is a generic algorithmic acceleration framework tailored for fixed-point iterations that are contractive and monotone. The central idea is to transform the sequence of iterates $\theta_{k+1} = F(\theta_k)$ into an accelerated sequence that reduces convergence error much faster, often by “squaring” the contraction factor in the error recurrence.

The SQUAREM update for any fixed-point mapping $F$ proceeds as follows:

• Compute two consecutive updates:

$r = F(\theta_k) - \theta_k \ v = (F(F(\theta_k)) - F(\theta_k)) - r \$

• Calculate the adaptive step length:

$\alpha_k = -\frac{\|r\|}{\|v\|}$

• Propose the extrapolated (squared) update:

$$\theta_{sq} = \theta_k - 2\alpha_k r + \alpha_k^2 v$$

• Accept or reject this candidate using the objective function to preserve monotonicity: If $f(\theta_{sq})$ does not increase by more than a small tolerance (objfn.inc), accept $\theta_{sq}$; otherwise, fall back to the usual or a conservative step.

This strategy is immediately applicable to MM algorithms—including the penalized estimation settings described as MM-RoPE—so long as the MM update is encapsulated as a mapping and a corresponding objective function is provided.

3. Application to Penalized Regression and High-Dimensional Problems

A representative MM-RoPE problem is penalized logistic regression using quadratic majorization. Here, the negative log-likelihood

$$f(\beta) = -\sum_{i} \left[ y_i \log p_i(\beta) + (N_i - y_i) \log(1 - p_i(\beta)) \right], \quad p_i(\beta) = \frac{1}{1 + \exp(-x_i^\top \beta)}$$

is majorized by a quadratic surrogate, allowing efficient minimization. The resulting update can be directly cast as a fixed-point step:

$$\beta^{(k+1)} = \beta^{(k)} - B^{-1} \nabla f(\beta^{(k)})$$

where $B$ is a uniform upper bound on the negative Hessian, such as $B = \tfrac{1}{4} X^T N X$. This iteration is precisely the situation for which SQUAREM yields dramatic gains.

In the SQUAREM framework, the MM step is implemented as fixptfn, and the negative log-likelihood as objfn in software. SQUAREM’s interface allows these to be wrapped with no further tuning:

result <- squarem(par=beta_init, fixptfn=mm_update, objfn=negloglik, control=list(tol=1e-8))

For high-dimensional problems and expensive M-steps, the acceleration is especially significant: trial results demonstrate orders-of-magnitude reductions in the number of fixed-point evaluations—down to 54 from over 2,600 in Poisson mixture models, for instance—translating to similar reductions in computing time (Du et al., 2018). In penalized regression with logistic loss, SQUAREM compresses hundreds or thousands of MM updates into a handful of accelerated steps.

4. Convergence Guarantees and Objective Safeguards

SQUAREM is designed specifically for monotone, contractive mappings: convergence is assured provided that the original MM algorithm is monotone and globally convergent. The innovation of SQUAREM is to maximize step length adaptively and to test candidate updates against the original objective, thereby retaining global convergence with higher practical efficiency.

A core acceptance safeguard is:

$f(\theta_{sq}) - f(\theta_k) \leq \text{objfn.inc}$

where $\text{objfn.inc}$ is a small, user-controlled tolerance. If the extrapolated step is not accepted, SQUAREM falls back on a basic MM update or a more conservative acceleration.

5. Practical Advantages, Limitations, and Implementation Summary

Advantages:

• Plug-and-play acceleration: SQUAREM requires only that the MM iteration and objective be coded as functions, with no algorithmic redesign or problem-specific tuning.
• Sharp reduction in computation: Each SQUAREM iteration typically involves two or three function evaluations, but achieves quadratic improvement in the convergence error, as opposed to single-step linear contraction in vanilla fixed-point/EM/MM.
• Stability: The safeguard based on merit function increase ensures robust monotonicity.
• Scalability: Especially effective for high-dimensional penalized regression and mixture models, where unaccelerated MM/EM may otherwise be impractical.

Limitations:

• Expensive objective evaluation: In cases where the objective (e.g., negative log-likelihood) is costly to compute, the acceptance check may dominate computation.
• Sensitive in non-contractive settings: If the base MM mapping is not at least contractive (or exhibits numerical instability), SQUAREM’s acceleration may be less effective or may require reduced step length.
• Inapplicable to non-monotone, non-fixed-point iterations: Accelerated convergence is not guaranteed outside these regimes.

6. Comparison to Alternative Acceleration Strategies

SQUAREM is contrasted with strategies such as ECME, EM-ICM, and problem-specific accelerations:

Method	Generality	Tuning Required	Applicability	Example Speedup
SQUAREM	Universal (EM, MM)	No	Any smooth contraction map	up to ~50x
ECME/ICM	Problem-specific	Yes	Specialized EM variants	variable
Basic MM	Universal	No	Any MM iteration	1x (baseline)

SQUAREM’s unique position is to serve as the default, domain-agnostic accelerator for slow, monotonic MM algorithms, including those deployed in modern penalized high-dimensional estimation (MM-RoPE settings).

7. Implementation Workflow

A typical MM-RoPE workflow with SQUAREM is as follows:

1. Write MM update as a function:

   mm_update <- function(par) { # code for MM update step }

2. Write objective function:

   negloglik <- function(par) { # code to evaluate negative log-likelihood or merit function }

3. Run SQUAREM:

   result <- squarem(par=init, fixptfn=mm_update, objfn=negloglik, control=list(tol=...))

4. Inspect convergence and, if necessary, adjust tolerances or fall back on base MM method.

This approach is directly applicable to penalized regression, mixture models, factor analysis, genetics admixture, and related high-dimensional estimation problems, enabling substantial practical acceleration without requiring bespoke algorithmic innovations.

---

In summary, MM-RoPE denotes the use of SQUAREM-based acceleration for MM algorithms particularly in penalized regression and high-dimensional estimation contexts. SQUAREM’s ability to reliably “square” the improvement at each step, coupled with plug-and-play usability and robust convergence safeguards, makes it a principal tool for improving MM algorithm efficiency in research and large-scale applications (Du et al., 2018).