PySLSQP: Research-Oriented SQP Solver

• PySLSQP is a Python package wrapping the SLSQP algorithm, providing enhanced diagnostics and research tools for nonlinear constrained optimization.
• It features advanced auto-differentiation, independent scaling, and live visualization to support precise and inspectable optimization workflows.
• The solver enables robust warm/hot restart capabilities and detailed iteration logging for transparent performance analysis and reproducibility in research.

PySLSQP is a Python package providing a transparent, research-oriented interface to the SLSQP (Sequential Least Squares Quadratic Programming) algorithm for nonlinear constrained optimization. It wraps Kraft’s original Fortran SLSQP source—identical to the SciPy implementation—while furnishing a suite of modern utilities including auto-differentiation via finite differences, independent variable/function scaling, comprehensive access to internal optimization state, live visualization, persistent iteration logs, and flexible warm/hot restart workflows. This architecture positions PySLSQP as a robust, extensible, and highly inspectable SQP solver for the scientific Python ecosystem (Joshy et al., 2024).

1. Mathematical and Algorithmic Foundations

PySLSQP targets the solution of nonlinear programs (NLPs) of the form: $$\begin{aligned} & \min_{x \in \mathbb{R}^n} f(x) \ & \text{s.t. } g_i(x) = 0, \quad i = 1, \dots, m, \ & \hspace{2.2em} h_j(x) \geq 0, \quad j = 1, \dots, p, \ & \hspace{2.2em} l \leq x \leq u, \end{aligned}$$ where $f : \mathbb{R}^n \to \mathbb{R}$ is smooth. SLSQP operates by sequentially solving QP subproblems. At each major iteration $k$, it linearizes the constraints at $x_k$ and solves: $$\begin{aligned} d_k = \arg\min_{d \in \mathbb{R}^n} \; \nabla f(x_k)^\top d + \tfrac12 d^\top B_k d \ \text{s.t.} \quad \nabla g_i(x_k)^\top d + g_i(x_k)=0, \ \qquad \nabla h_j(x_k)^\top d + h_j(x_k)\geq0, \ \qquad l - x_k \leq d \leq u - x_k, \end{aligned}$$ where $B_k$ is the current (quasi-Newton) Hessian approximation. The step $d_k$ is accepted or rejected based on a merit function condition, and a BFGS-type update of the Hessian is performed on acceptance. Convergence is certified at stationarity points satisfying the Karush-Kuhn-Tucker (KKT) conditions: $$\begin{aligned} & \nabla f(x^*) + \sum_{i=1}^m \lambda_i \nabla g_i(x^*) + \sum_{j=1}^p \mu_j \nabla h_j(x^*) = 0, \ & g_i(x^*) = 0, \quad h_j(x^*) \geq 0, \ & \mu_j \geq 0, \quad \mu_j h_j(x^*) = 0. \end{aligned}$$ For numerical stability in degenerate cases, SLSQP implements a relaxation (modified Powell) introducing a penalized scalar slack variable and, if necessary, resets Hessian approximations (Ma et al., 2024).

2. PySLSQP Researcher-Focused Enhancements

PySLSQP extends beyond the underlying Fortran implementation by presenting several facilities aimed at research rigor and workflow flexibility:

• Auto-Generation of Derivatives: If explicit Jacobians/gradients are not provided, PySLSQP computes them via user-tunable first-order finite differences (central and forward schemes). This ensures robustness in black-box or legacy-code scenarios:

$$\frac{\partial f}{\partial x_i} \approx \begin{cases} \frac{f(x + h e_i) - f(x - h e_i)}{2 h} & \text{(central)} \ \frac{f(x + h e_i) - f(x)}{h} & \text{(forward)} \end{cases}$$

• Independent Scaling: Each variable, objective, and constraint can be rescaled internally by user-supplied scaling factors. PySLSQP reformulates the NLP accordingly—for example, minimizing $f_s f(x_s x')$ in scaled variables—while transparently mapping bounds and derivative calculations. This is essential for improving condition numbers and controlling step sizes in ill-scaled problems.
• Access to Internal State: At each major iteration, PySLSQP provides access to:
  • Primal variables $f : \mathbb{R}^n \to \mathbb{R}$0
  • Objective and constraint values $f : \mathbb{R}^n \to \mathbb{R}$1
  • Gradients and Jacobians
  • Hessian approximation $f : \mathbb{R}^n \to \mathbb{R}$2
  • Search direction $f : \mathbb{R}^n \to \mathbb{R}$3, step length $f : \mathbb{R}^n \to \mathbb{R}$4
  • Lagrange multipliers ($f : \mathbb{R}^n \to \mathbb{R}$5)
  • Norms of optimality and feasibility
  • These are archivable in HDF5 or CSV formats, indexed for easy post hoc analysis (Joshy et al., 2024).

3. Visualization and Iteration Data Management

PySLSQP is distinguished by its tight integration of live visualization and fine-grained iteration logging:

• Live Visualization: During optimization, a separate process or thread can plot trajectories of scalar or vector monitors per iteration, including objective value, maximum constraint violation, step norm $f : \mathbb{R}^n \to \mathbb{R}$6, or user-selected variables. This is implemented using matplotlib and supports rapid diagnosis of convergence behavior and stagnation.
• Data Persistence: All relevant quantities per iteration (or at user-specified frequency) are stored in HDF5 files. A summary text file logs scalar diagnostics (numbers of function/gradient evaluations, max constraint violation, optimality measures, step sizes). The logged data enables full post-processing and “reanimation” of optimization runs.

4. Flexible Restart Mechanisms

PySLSQP introduces structured restart capabilities to facilitate large, computationally demanding optimization work:

• Warm Restart: Loads only the most recent iterate $f : \mathbb{R}^n \to \mathbb{R}$7 as the new $f : \mathbb{R}^n \to \mathbb{R}$8, while all other algorithmic state (Hessian, multipliers) is reinitialized.
• Hot Restart: Recovers the full solver state—current iterate, gradients, Jacobians, Hessian approximation, and multipliers—enabling seamless continuation as though the optimization had never been interrupted. This is particularly effective in time-limited batch job settings or when exploring neighboring optima (Joshy et al., 2024).

5. Example of Usage and API Design

PySLSQP exposes a minimal, direct Python interface. A canonical usage scenario is:

$f : \mathbb{R}^n \to \mathbb{R}$9 After execution, all iterates, internal states, and summary metrics are archived and accessible for subsequent analysis (Joshy et al., 2024).

6. Comparative Performance and Limitations

Empirical benchmarks, including a spacecraft landing trajectory optimization with ~200 decision variables, demonstrate that PySLSQP is unique among SNOPT, IPOPT, and TrustConstr in converging within 200 function evaluations in this challenging regime. The underlying Fortran code is identical to SciPy’s SLSQP; the research-oriented differentiators are transparency, live monitoring, and restartability (Joshy et al., 2024).

In direct comparison with I-SLSQP/I-SQP enhancements (Ma et al., 2024):

Problem	Instances Solved (of 6)	Best vs. I-SLSQP	Time Ratio (PySLSQP / I-SLSQP)
PSD	6/6	identical	0.8
EDWC-EW	5/6	5–15% worse	1.3

A key limitation is that PySLSQP, like classical SLSQP, relies on a single-ζ Powell relaxation for subproblem feasibility, and all LSQ subproblems are solved using Lawson–Hanson dual NNLS. On highly ill-conditioned problems, this can trigger serious cancellation, inaccurate directions, or premature termination. PySLSQP does not implement Nowak’s multi-ζ or QP hybrid fallback; empirically this means optimality can be compromised or progress lost on difficult instances, where I-SLSQP achieves better robustness via hybrid and QP-relaxation machinery (Ma et al., 2024). For well-conditioned small- to medium-scale NLPs, PySLSQP remains efficient and easy to use.

7. Context in the SQP Ecosystem and Recommendations

PySLSQP preserves the reliability of Kraft’s original SLSQP—trusted and widely deployed—while exposing the full internal optimization trace and research-friendly utilities. It is competitive with improved SQP variants on well-conditioned problems and offers Python-centric usability exceeding legacy Fortran/SciPy wrappers. However, for pathologically ill-conditioned or large-scale, simulation-based process optimizations, variants such as I-SLSQP with hybrid relaxations and explicit QP fallback are currently superior in both robustness and final optimality (Joshy et al., 2024, Ma et al., 2024).

A plausible implication is that PySLSQP’s modular design and f2py/Meson build system may enable rapid prototyping of future algorithmic advances, including more sophisticated relaxations or integration with external QP solvers. Its combination of transparency, live diagnostic monitoring, flexible checkpointing, and ease of modification distinguishes it in academic and industrial research workflows.