RandOpt: Random Directions Optimization

• RandOpt is a family of stochastic optimization methods that uses random directional perturbations to estimate gradients and optimize noisy simulation objectives.
• It implements both first-order and Newton-like second-order updates using minimal function evaluations while controlling estimator bias and variance.
• Theoretical and empirical results confirm RandOpt's convergence and efficiency, outperforming traditional methods like 2SPSA in various simulation benchmarks.

The RandOpt algorithm refers to a family of stochastic optimization methods that utilize random directions for adaptive, simulation-based optimization, particularly in settings where explicit gradients are unavailable, expensive, or unreliable. The canonical formulation and analysis are provided in “Adaptive system optimization using random directions stochastic approximation” (A. et al., 2015), which establishes the theoretical foundation and practical performance of RandOpt across first-order (gradient-based) and second-order (Newton-like) stochastic approximation.

1. Algorithmic Foundations and Methodological Framework

RandOpt algorithms address black-box or simulation optimization problems by iteratively updating a parameter vector $x_n \in \mathbb{R}^N$ via measurements of a noisy objective $f(x) + \text{noise}$. At each iteration, a random direction $d_n \in \mathbb{R}^N$ is sampled, and the algorithm probes $f$ along $x_n \pm \delta_n d_n$ to construct stochastic approximations to the gradient and, in second-order variants, the Hessian.

The two principal versions are:

• First-order RandOpt (1RDSA): Estimates $\nabla f(x_n)$ using two loss function measurements along $d_n$.
• Second-order RandOpt (2RDSA): Estimates both $\nabla f(x_n)$ and $\nabla^2 f(x_n)$ using three measurements, enabling stochastic Newton updates.

Perturbation vectors $d_n$ may be drawn from i.i.d. uniform, Gaussian, or asymmetric Bernoulli distributions, with normalization chosen to optimize bias–variance tradeoff.

1.1 First-order RDSA Update

Given $x_n$, direction $d_n$, and scalar step/perturbation sizes $a_n$ and $\delta_n$: $$\hat{g}_n = \frac{f(x_n + \delta_n d_n) - f(x_n - \delta_n d_n)}{2\delta_n} d_n$$

$x_{n+1} = x_n - a_n \hat{g}_n$

1.2 Second-order (Newton) RDSA Update

With additional measurement at $x_n$: $$\hat{H}_n = A_n \cdot \frac{f(x_n+\delta_n d_n) + f(x_n-\delta_n d_n) - 2f(x_n)}{\delta_n^2}$$ where $A_n$ encodes a moment-based normalization on $d_n$. The Hessian estimate $\bar{H}_n$ is smoothed over time, spectral-projected to positive-definite, and used for a Newton update: $$x_{n+1} = x_n - a_n \, [\Upsilon(\bar{H}_n)]^{-1}\hat{g}_n$$

This framework yields $O(1)$ loss measurements per iteration for first-order methods and $O(N)$ for Hessian estimation, with no need for explicit gradient computation (A. et al., 2015).

2. Perturbation Schemes and Normalization

RandOpt supports both continuous-valued and discrete-valued perturbations:

• Uniform: $$d_n^i \stackrel{\text{i.i.d.}}{\sim} U[-\eta, \eta]$$, with estimator scaling $\sim 3/\eta^2$.
• Asymmetric Bernoulli: $d_n^i=-1$ w.p. $(1+\varepsilon)/(2+\varepsilon)$, $d_n^i=+1+\varepsilon$ w.p. $1/(2+\varepsilon)$, yielding controlled second and fourth moments.

The estimator bias is $O(\delta_n^2)$ under smoothness and noise independence assumptions. Asymmetric Bernoulli achieves estimator Mean Squared Error (MSE) arbitrarily close to that of Simultaneous Perturbation Stochastic Approximation (SPSA), and continuous uniform perturbations, while more general, can introduce higher variance.

3. Theoretical Guarantees and Asymptotic Properties

RandOpt admits rigorous probabilistic convergence guarantees under classical stochastic approximation hypotheses:

• Unbiasedness: $$\mathbb{E}[\hat{g}_n \mid x_n] = \nabla f(x_n) + O(\delta_n^2)$$, $$\mathbb{E}[\hat{H}_n \mid x_n] = \nabla^2 f(x_n) + O(\delta_n^2)$$.
• Strong Consistency: $x_n \rightarrow x^*$ almost surely, with $x^*$ a locally asymptotically stable point of $\nabla f=0$.
• Asymptotic Normality: $$n^{\beta/2} (x_n-x^*) \Rightarrow \mathcal{N}(\mu,\Sigma)$$, with $\mu$ governed by the step/perturbation decay rates and $\Sigma$ depending on the Hessian spectrum and noise variance.

Second-order methods can strictly outperform 2SPSA in asymptotic MSE per function evaluation, as shown analytically and supported by numerical experiments (A. et al., 2015).

4. Empirical Performance and Benchmarks

Numerical experiments include quadratic and higher-order polynomials in $N=10$ dimensions:

• With quadratic test functions and moderate noise ($\sigma=10^{-3}$), first-order RandOpt (1RDSA-Unif) achieves NMSE $\approx 4.53 \times 10^{-2}$; 1RDSA-AsymBer matches 1SPSA performance.
• For second-order, 2RDSA-Uniform and 2RDSA-AsymBer obtain NMSE of $9.61 \times 10^{-5}$ and $8.39 \times 10^{-5}$ respectively, consistently outperforming 2SPSA ($1.05 \times 10^{-3}$).

Budgeted for a fixed number of objective calls, these results remain robust across quadratic and quartic test functions, with especially large gains for 2RDSA over 2SPSA for higher-order landscapes (A. et al., 2015).

5. Practical Guidelines and Tuning

Parameterization of RandOpt involves several key choices:

• Step-size schedule: $a_n \sim n^{-\alpha}$ ($\alpha \approx 1$ for gradient, $0.6 \ldots 0.8$ for Newton).
• Perturbation-size: $\delta_n \sim c_0 n^{-\gamma}$, with $\gamma=1/6$ balancing estimator bias and variance.
• Perturbation distribution: Asymmetric Bernoulli with $\varepsilon \approx 10^{-3}$ is near-optimal for variance and simplicity.
• Hessian smoothing (Newton case): Update $\bar{H}_n$ as an exponential moving average; eigenvalue-lift (spectral shifting) stabilizes inversion for noisy estimates.
• Box constraints: Projection of $x_n$ into a bounding region is recommended for numerical stability.
• Burn-in: For second-order schemes, a burn-in period (5–20% of the computational budget) of first-order RDSA improves basin attraction before switching to Newton steps.

6. Relations to Other Randomized Optimization Methods

RandOpt is a generalization and extension of random-direction stochastic approximation algorithms. It is distinct from purely zeroth-order methods such as random search or policy search by providing provable finite-difference gradient and Hessian estimation with low sample complexity. Compared to 2SPSA, RandOpt-based Newton schemes reduce the number of function evaluations per iteration (3 vs. 4) and achieve lower mean-squared error per simulation under broad conditions.

The framework is sufficiently general to accommodate a range of perturbation schemes and step-size decays, providing adaptability across application domains ranging from simulation-based control to stochastic optimization in high-dimensional, noisy environments (A. et al., 2015).

7. Applications and Extensions

RandOpt has been empirically applied to unconstrained simulation optimization, noisy objective minimization, and scenarios where model-based gradients are unavailable or unreliable. The underlying methodologies are extensible to multi-agent and swarm-based settings, as indicated by related developments in swarm-based random descent and distributed simulation optimization. The algorithmic principles also subserve recent gradient-free training approaches for deep learning, albeit such variants (e.g., RSO for DNNs) often employ more localized coordinatewise search or greedy update schemes that diverge from the canonical RandOpt architecture (Tripathi et al., 2020).

A plausible implication is that ongoing methodology development will focus on further lowering sample complexity and variance in high-dimensional, black-box contexts, building on the analytical backbone established for RandOpt.

---

The precise mathematical formulation, estimator construction, asymptotic results, and comprehensive empirical evaluation are detailed in “Adaptive system optimization using random directions stochastic approximation” (A. et al., 2015).