Primal-dual dynamics featuring Hessian-driven damping and variable mass for convex optimization problems

Abstract: This paper deals with a new Tikhonov regularized primal-dual dynamical system with variable mass and Hessian-driven damping for solving a convex optimization problem with linear equality constraints. The system features several time-dependent parameters: variable mass, slow viscous damping, extrapolation, and temporal scaling. By employing the Lyapunov analysis approach, we obtain the strong convergence of the trajectory generated by the proposed system to the minimal norm solution of the optimization problem, as well as convergence rate results for the primal-dual gap, the objective residual, and the feasibility violation. We also show that the convergence rates of the primal-dual gap, the objective residual, and the feasibility violation can be improved by appropriately adjusting these parameters. Further, we conduct numerical experiments to demonstrate the effectiveness of the theoretical results.

• The paper introduces a novel Tikhonov regularized primal-dual dynamical system featuring variable mass and Hessian-driven damping to improve convergence in convex optimization under linear constraints.
• It leverages Lyapunov techniques to establish strong convergence and explicit rate bounds for the primal-dual gap, objective residual, and feasibility violation.
• Numerical experiments confirm that variable mass and aggressive time-scaling accelerate convergence while Hessian-driven damping effectively suppresses oscillatory behavior.

Primal-Dual Dynamics with Hessian-Driven Damping and Variable Mass for Convex Optimization

Overview and Motivation

The paper "Primal-dual dynamics featuring Hessian-driven damping and variable mass for convex optimization problems" (2603.29124) presents a new Tikhonov regularized primal-dual dynamical system tailored to convex optimization under linear equality constraints. The authors address gaps in prior work, notably the lack of variable mass and Hessian-driven damping integration in primal-dual continuous-time dynamics for constrained optimization. The proposed system incorporates multiple time-dependent parameters—variable mass, slow viscous damping, Hessian-driven damping, extrapolation, and temporal scaling—yielding improved convergence guarantees and control over trajectory oscillations. The analysis builds upon and significantly extends prior results on inertial and primal-dual mechanisms [Zhu2024S, He2023C, cs24].

System Formulation

The target problem is: $$\min_{x \in \mathcal{X}} f(x) \quad \text{s.t.} \quad Ax = b$$ where $f$ is a continuously differentiable convex function, and $A$ is a linear operator between Hilbert spaces. The authors introduce a generalized, Tikhonov-regularized primal-dual dynamic system: $\begin{split} &m(t)\ddot{x}(t) + \frac{\alpha}{t^q}\dot{x}(t) + \gamma \frac{d}{dt}\nabla_x \mathcal{L}_t(x(t),\lambda(t)) + t^s \nabla_x \mathcal{L}_t(x(t),\lambda(t)) = 0 \ &\dot{\lambda}(t) - (\alpha-1)(t^{q+s}-\gamma q t^{q-1})\nabla_\lambda \mathcal{L}_t(x(t)+\theta(t)\dot{x}(t),\lambda(t)) = 0 \end{split}$ where $\mathcal{L}_t$ is an augmented Lagrangian with Tikhonov regularization, $m(t)$ is a variable mass function, $\gamma$ is the Hessian-driven damping coefficient, and $\theta(t)$ is an extrapolation parameter explicitly constructed from system variables.

The system generalizes existing primal-dual dynamical systems by including variable coefficients and Hessian feedback, permitting fine-grained control of convergence rates and trajectory behavior.

Theoretical Results

Convergence and Rate Analysis

Utilizing Lyapunov techniques, the authors establish:

• Strong convergence of the trajectories to the minimal-norm solution of the constrained problem, extending beyond weak convergence or inferior-limit convergence typically found in prior literature [2026zhu, lihl, zhujcam, sunjota, Bot2021T, cs24k].
• Explicit rate bounds for primal-dual gap, objective residual, and feasibility violation, all of which depend on the choice of variable mass, viscous damping, and scaling parameters. For suitably chosen $m(t)$ and parameters, convergence rates outperform prior formulations, with error terms decaying at rates:
  • $\mathcal{O}(\sqrt{m(t)}\, t^r + \frac{1}{t^p})$ for primal-dual gap, objective residual, and feasibility violation (with $f$0 determined by system parameters).
• Parameter sensitivity: Detailed conditions are provided for selecting $f$1, $f$2, $f$3, $f$4, and the decay properties of $f$5, directly relating them to attainable rates and robustness of convergence.

Role of System Components

• Variable mass ($f$6): Enables acceleration and preservation of convergence rates and can improve performance, as validated by numerical experiments.
• Hessian-driven damping ($f$7): Suppresses trajectory oscillations and stabilizes convergence, particularly in high-dimensional and ill-conditioned settings.
• Slowly viscous damping ($f$8): Ensures strong convergence to minimal-norm solution, even without restrictive assumptions on the trajectory location.
• Time scaling ($f$9): Allows further acceleration, with empirical confirmation that rapid scaling enhances objective and feasibility convergence.

Extension and Improvement

The system strictly generalizes prior work in several directions:

• Admits variable mass, contrasting with fixed-mass approaches [csim2024].
• Incorporates Hessian-driven feedback, previously limited to unconstrained optimization [Attouch2023A, Bot2021T].
• Removes strong assumptions on the trajectory required for strong convergence, a key limitation of earlier Tikhonov regularized approaches [2026zhu, lihl, zhujcam].

Numerical Validation

Two quadratic programming examples demonstrate:

1. Impact of mass function: Decreasing mass improves convergence speed of both the objective residual and feasibility violation.
2. Impact of time-scaling: Aggressive scaling (larger $A$0) accelerates convergence.
3. Impact of Hessian-driven damping: The addition of $A$1 stabilizes convergence and eliminates oscillatory behavior across solution trajectories, a frequent issue in inertial primal-dual systems.

Implications and Future Directions

Practical Implications

The system provides a robust framework for algorithmic design in convex-constrained optimization, particularly for applications requiring:

• Fast convergence with explicit rate guarantees.
• Smooth, non-oscillatory trajectories in numerical discretization.
• Adaptability via parameter tuning.

Possible application domains include large-scale optimization, saddle-point problems in variational inequalities, and constrained learning tasks.

Theoretical Advances

The removal of strong proximity assumptions for strong convergence advances the state-of-the-art in continuous-time convex optimization. The integration of variable mass and Hessian-driven damping into primal-dual frameworks opens avenues for further analysis of dynamic parameter adaptation and stability.

Future Developments

Key open directions include:

• Explicit discretizations: Translating continuous-time dynamics into effective numerical algorithms, preserving theoretical rates and stability properties.
• Time-dependent Hessian-driven damping: Extending the analysis to non-constant $A$2, potentially yielding adaptive or more resilient dynamics.
• Generalization to broader constraint structures: Extending to nonlinear or more general convex constraints, and multi-block formulation.

Conclusion

This paper introduces and analyzes a primal-dual dynamical system incorporating variable mass and Hessian-driven damping for convex optimization with linear constraints (2603.29124). The framework achieves strong trajectory convergence and fast, tunable rates for primal-dual gap, objective residual, and feasibility violation. The results generalize prior work, relax key assumptions, and provide a technical foundation for algorithmic advances in continuous-time and discretized convex optimization. Future research will focus on discretization strategies and broader generalizations in constraint handling.