Progressive Power Homotopy (Prog-PowerHP)

• Progressive Power Homotopy (Prog-PowerHP) is a homotopy-based optimization paradigm that sequentially deforms complex non-convex problems into tractable surrogates using power transformations and Gaussian smoothing.
• It employs adaptive scheduling in both zeroth- and first-order regimes to ensure iterates remain in the basin of attraction, enhancing robustness against ill-conditioning and multi-modality.
• Empirical results show state-of-the-art performance in tasks like adversarial attacks, phase retrieval, and constrained optimization in power systems.

Progressive Power Homotopy (Prog-PowerHP) is a homotopy-based optimization paradigm for navigating high-dimensional non-convex objectives. The central principle is to deform a target (potentially non-convex and ill-conditioned) problem into a sequence of intermediate, more tractable surrogates through the simultaneous application of power transformations and smoothing, or related structured relaxations. As the homotopy path is traversed, algorithmic iterates are tracked in the parameter space, ensuring they remain in the basin of attraction of progressively less regularized, more challenging surrogates, culminating close to the true global optimum. Prog-PowerHP has been developed in both zeroth- and first-order settings, with substantive theoretical justification and empirical validation in machine learning and scientific optimization tasks (Xu, 17 Nov 2025, Xu, 22 Jan 2026).

1. Power-Transformed Gaussian-Smoothing: Surrogate Construction

For unconstrained maximization $f:\mathbb{R}^{d}\to\mathbb{R}$, Prog-PowerHP introduces the power-transformed, Gaussian-smoothed surrogate: $$F_{N, \sigma}(\mu) = \mathbb{E}_{x \sim \mathcal{N}(\mu, \sigma^2 I_d)} \left[ e^{N f(x)} \right]$$ where $N>0$ is the power parameter and $\sigma>0$ is the smoothing scale (Xu, 17 Nov 2025). The power transform exponentially "tilts" the distribution toward regions of high objective value, causing the maxima of $F_{N, \sigma}$ to cluster near that of $f$ as $N \to \infty$. The Gaussian smoothing regularizes the landscape, suppressing spurious local maxima and facilitating robust gradient estimation.

A key insight is the contrast to directly smoothing $f$. Smoothing $e^{N f}$, even with modest $\sigma$, yields surrogate maxima that are globally attracted to $x^*$ (the global maximizer of $f$), rather than merely local optima. This mechanism is robust to ill-conditioning and multi-modality in $f$, provided $N$ and $\sigma$ are appropriately scheduled.

2. Homotopy/Continuation and Scheduling Strategies

Prog-PowerHP employs a single-loop homotopy scheme in which either the smoothing parameter ($\sigma$) is geometrically decayed toward a fixed lower bound (zeroth-order regime (Xu, 17 Nov 2025)), or both the power parameter ($N$) and smoothing scale ($\sigma$) are progressively adjusted (first-order regime (Xu, 22 Jan 2026)):

• Zeroth-Order: $N$ fixed; $\sigma_{t+1} = \sigma_0 \cdot \beta^{t+1} + b$, for decay rate $\beta \in (0,1)$ and floor $b>0$.
• First-Order: $N_t = N_0 + \phi_t \Delta$; $\sigma_t = b + \sigma_0 \beta^t$ for schedule $\phi_t$ with $\sum_t \phi_t=1$.

This scheduling forms the Prog-PowerHP "homotopy path"—iterates are initialized in the tractable regime (large $\sigma$, small $N$), then tracked as the surrogate sharpens, ensuring solutions at each stage are in the basin of attraction for the next surrogate.

In constrained or structured settings such as AC-OPF, the homotopy includes both objective and constraint relaxations parameterized by a schedule $\lambda_t\in[0,1]$, interpolating from convex, relaxed subproblems to the original non-convex instance (Li et al., 11 Nov 2025). This allows policy learning procedures to traverse from trivial to challenging regimes under self-supervision.

3. Gradient Estimation and Algorithmic Framework

Zeroth-Order Optimization

The method employs the Gaussian-smoothing gradient estimator: $$\hat{\nabla} F_{N, \sigma}(\mu) = \frac{1}{K} \sum_{k=1}^K (x_k - \mu) e^{N f(x_k)}$$ with $x_k \sim \mathcal{N}(\mu, \sigma^2 I_d)$ (Xu, 17 Nov 2025). This estimator satisfies $$\mathbb{E}[\hat{\nabla} F_{N, \sigma}(\mu)] = \sigma^2 \nabla F_{N, \sigma}(\mu)$$, so dividing by $\sigma^2$ yields an unbiased estimate. Variance is controlled and bounded (Lemma 4.2 in (Xu, 17 Nov 2025)).

First-Order Stochastic Optimization

For models with parametric structure $f_w(x)$ and explicit gradients: $$\hat{G}_t = \frac{1}{KJ} \sum_{k=1}^K \sum_{j=1}^J N_t e^{N_t f_{w_k}(x_j)} \nabla_w f_{w_k}(x_j)$$ where $w_k \sim \mathcal{N}(\mu_{t-1}, \sigma_t^2 I_d)$, $x_j \sim \mathcal{D}$ (Xu, 22 Jan 2026). Iterates are updated via $\mu_t = \mu_{t-1} + \alpha_t \hat{G}_t$ with carefully chosen step-sizes ($\sum \alpha_t = \infty, \sum \alpha_t^2<\infty$).

Constrained Policy Learning (AC-OPF)

The policy $\pi_\Theta(\xi)$ is trained to minimize a homotopically evolving penalty objective: $$L(\Theta; \lambda) = \mathbb{E}_{\xi \sim D} \left[ f_\lambda(\pi_\Theta(\xi); \xi) + \sum_i w_i p_i(h_{\lambda,i}(\pi_\Theta(\xi); \xi)) + \sum_j w_j p_j(g_{\lambda,j}(\pi_\Theta(\xi); \xi)) \right]$$ where $h,g$ denote (relaxed) equality/inequality constraints, and penalties $p_{eq}(h) = h^2$, $p_{in}(g)=\textrm{ReLU}(g)^2$ (Li et al., 11 Nov 2025).

4. Theoretical Properties and Global Behavior

Theoretical analysis establishes strong convergence behavior under mild regularity assumptions:

• For appropriate homotopy schedule and sufficiently large $N$, stationary points of $F_{N, \sigma}$ can be made to lie arbitrarily close to the global maximizer $x^*$ (Xu, 17 Nov 2025, Xu, 22 Jan 2026).
• Under stepsizes $\alpha_t \sim t^{-1/2-\gamma}$, the iteration complexity to reach $\mathbb{E} \|\nabla F \|^2 < \epsilon$ is $O(d^2 \epsilon^{-2})$ (for $\gamma \to 0$).
• In constrained settings, empirical evidence demonstrates strong feasibility restoration and generalization to unseen instances, though the NP-hardness of global optimization in the general AC-OPF setting precludes formal global guarantees (Li et al., 11 Nov 2025).

Phase transition analyses in tensor PCA show a bifurcation in landscape geometry along the homotopy path: at sufficiently large smoothing, the problem becomes strictly convex with a unique global maximizer, while as smoothing decreases, secondary local maxima emerge, justifying the necessity for a carefully managed homotopy schedule (Anandkumar et al., 2016).

5. Empirical Performance and Benchmark Results

Prog-PowerHP has demonstrated:

• Robust top-three ranking on standard non-convex test functions (e.g., Ackley, Rosenbrock) among a pool of eight zeroth-order and homotopy algorithms (Xu, 17 Nov 2025).
• State-of-the-art results in adversarial black-box attacks (ImageNet, $d=150,528$), outperforming GS-PowerOpt, ZO-SLGH, CMA-ES, Square Attack, etc., in terms of both success rate and perturbation size.
• Significant improvement in phase retrieval at sample-to-dimension ratios approaching the information-theoretic limit, with success rates as high as 92% ($d=100$) compared to 41–56% for strong first-order baselines (Xu, 22 Jan 2026).
• In training of under-parameterized two-layer ReLU networks, lower test error and higher success rates than SGD, Adam, and various specialized landscape-smoothing methods (Xu, 22 Jan 2026).
• For parametric AC-OPF, sustained feasibility above 98% on unseen load settings (75–150% of nominal), and improved objective values relative to penalty-based baselines (Li et al., 11 Nov 2025).

Scenario	Prog-PowerHP Result	Reference
ImageNet Attack	1st place; best trade-off (d=150,528)	(Xu, 17 Nov 2025)
Phase Retrieval	0.07 rel. error (92% succ., d=100)	(Xu, 22 Jan 2026)
AC-OPF (30-bus)	1.7e–3 mean eq, 0.00 mean ineq violation	(Li et al., 11 Nov 2025)

6. Variants, Extensions, and Limitations

While originally formulated in a zeroth-order framework for black-box optimization, Prog-PowerHP admits generalization to first-order and policy-learning contexts. Notable aspects include:

• The power-homotopy principle (progressive exponentiation of the objective) and Gaussian smoothing (mollification) are jointly responsible for global-attracting surrogates.
• Adaptive scheduling (progressively increasing $N$ or decaying $\sigma$) enhances numerical stability and empirical recovery rates.
• Puiseux series and tropical-algebraic preprocessing allow extension to singularities and multiple root tracking in algebraic homotopy (Bliss et al., 2016).

The principal limitations are the need for hyperparameter tuning ($N_0,\,\Delta,\,\sigma_0,\,b,\,\beta$), sampling cost scaling as $O(KJd)$ per iteration, and, in some settings, lack of formal global guarantees due to intractability of the underlying problem (notably in constrained or high-dimensional combinatorial regimes).

7. Connections, Impact, and Outlook

Prog-PowerHP unifies a variety of homotopy- and smoothing-based approaches:

• In tensor PCA, it provides a rigorous phase transition analysis and nearly-linear time global recovery for spiked tensors at the optimal statistical threshold (Anandkumar et al., 2016).
• In policy learning for control and power systems, it achieves scalable constraint-aware optimization without labeled solutions via structured objective and constraint homotopies (Li et al., 11 Nov 2025).
• The method matches or surpasses the empirical performance of state-of-the-art global optimization heuristics, with superior robustness in high dimensions and cluttered landscapes (Xu, 17 Nov 2025, Xu, 22 Jan 2026).

A plausible implication is that Prog-PowerHP, when appropriately tailored, provides a principled and flexible toolkit for diverse non-convex search problems, and sets a template for future research integrating power-based homotopies, smooth surrogate design, and model-aware continuation schemes.