Neural Co-State Projection Regulator

• NCPR is a learning-based optimal control framework that predicts a projected co-state sequence to compute control inputs via convex QP-based Hamiltonian minimization.
• It bridges nonlinear model predictive control and reinforcement learning by encoding adjoint dynamics into self-supervised and model-based training objectives.
• Demonstrated improvements include computational efficiency, robustness on out-of-distribution states, and potential scalability to higher-dimensional systems.

Searching arXiv for the cited NCPR and related co-state control papers to ground the article in current literature. Neural Co-State Projection Regulator (NCPR) denotes a class of learning-based optimal control frameworks in which a neural network predicts a co-state-derived quantity and a lightweight optimization layer computes the control input by minimizing the Hamiltonian under constraints. In the formulation that explicitly uses the term, NCPR is a model-free framework for finite-horizon quadratic regulator problems on nonlinear control-affine systems with input constraints: a Co-state Projection Neural Network (CPNN) maps the current state to a finite-horizon trajectory of projected co-states, and only the first element is used in a quadratic program (QP) that enforces input constraints while approximating Pontryagin’s Minimum Principle (PMP) first-order optimality conditions (Lian et al., 1 Aug 2025). Closely related formulations appear under the names co-state neural network (CoNN), neural co-state regulator (NCR), adjoint-based neural regulator (ANR), and neural co-state policies; together they define a broader research program in which optimality is encoded through adjoints or co-states rather than through a directly learned control law (Lian et al., 1 Mar 2025, Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026, Leeftink et al., 6 May 2026).

1. Definition and scope

In its explicit formulation, NCPR addresses nonlinear control-affine systems of the form

$$\dot{\mathbf{z}}(t) = f(\mathbf{z}(t)) + g(\mathbf{z}(t))\mathbf{u}(t), \qquad \mathbf{u}(t)\in\mathcal{U},$$

with finite-horizon quadratic cost

$$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$

The neural component does not directly output the control. Instead, it predicts either a co-state trajectory, a projected co-state trajectory, or a recurrent latent state interpreted as a co-state surrogate; the control is then obtained from Hamiltonian minimization, typically through a convex QP (Lian et al., 1 Aug 2025, Lian et al., 16 Jul 2025).

Within this literature, the term “projection” has three distinct but related uses. In NCPR proper, the learned object is the projected co-state $\lambda^\top g(z)$, i.e. the co-state premultiplied by the input gain, because this is the quantity that enters the control-dependent part of the Hamiltonian (Lian et al., 1 Aug 2025). In CoNN, NCR, and ANR, projection refers to projecting the unconstrained Hamiltonian-minimizing control onto a feasible input set by solving a constrained QP (Lian et al., 1 Mar 2025, Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026). In neural co-state policies for recurrent reinforcement learning, projection refers to aligning the hidden state with a critic-derived co-state direction on the recurrent representation manifold through a cosine loss (Leeftink et al., 6 May 2026).

This family of methods occupies an intermediate position between nonlinear model predictive control (NMPC) and conventional reinforcement learning. Relative to NMPC, the objective is to amortize the computational burden of repeated two-point boundary value problem (TPBVP), nonlinear program, or shooting-based solves into an offline learning stage. Relative to Bellman-style RL, the objective is to learn an adjoint structure grounded in PMP and then recover control from Hamiltonian minimization rather than directly approximating a policy or value-induced policy (Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026, Lian et al., 1 Aug 2025).

2. Pontryagin structure and the role of projected co-states

The common mathematical substrate is PMP. For an optimal control problem with Hamiltonian

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$

PMP yields the canonical equations

$$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$

together with the pointwise minimization condition

$$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$

For quadratic regulation of a control-affine system,

$$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$

the unconstrained stationary condition gives

$$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$

while the constrained case becomes

$$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$

These identities explain why projected co-states are sufficient in the model-free NCPR setting: the control law depends on $g^\top \lambda$, or equivalently on $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$0, rather than on $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$1 alone (Lian et al., 1 Aug 2025).

The projected co-state formulation is therefore a structural reduction. Instead of identifying $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$2, $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$3, and the full adjoint field, the learner approximates the linear coefficient of the control-dependent Hamiltonian term. In NCPR this quantity is represented as

$$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$4

and only the first row is used online (Lian et al., 1 Aug 2025).

A complementary theoretical development appears in recurrent reinforcement learning. There, the hidden state $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$5 of a continuous-time recurrent policy,

$$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$6

is interpreted as a neural representation of the PMP co-state $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$7, while the readout layer is interpreted as Hamiltonian minimization. In the Neural Co-state Policy class,

$$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$8

which directly mirrors adjoint dynamics and pointwise control minimization (Leeftink et al., 6 May 2026). This does not define NCPR in the narrow sense of (Lian et al., 1 Aug 2025), but it places recurrent policy structure and co-state-based regulation under the same optimal-control interpretation.

3. Architectural realizations

The literature contains several concrete realizations of neural co-state regulation.

Framework	Learned output	Runtime control computation
NCPR / CPNN	Projected co-state trajectory $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$9	QP with input constraints
CoNN	Co-state trajectory $\lambda^\top g(z)$0	Unconstrained PMP law or constrained QP
NCR	Norm-optimal co-state trajectory $\lambda^\top g(z)$1	QP minimizing Hamiltonian over $\lambda^\top g(z)$2
ANR	Finite-horizon adjoint trajectory	QP with input bounds and HOCBF constraints
Neural Co-state Policies	Recurrent hidden state as co-state proxy	Actor head interpreted as Hamiltonian minimizer

CoNN is a feedforward network trained on offline TPBVP solutions. For each initial state $\lambda^\top g(z)$3, it outputs a horizon of co-states, and the first element $\lambda^\top g(z)$4 is inserted into the unconstrained control law

$\lambda^\top g(z)$5

or into the constrained QP. This parameterizes a family of TPBVP solutions over initial states rather than learning only a policy $\lambda^\top g(z)$6 (Lian et al., 1 Mar 2025).

NCR is also feedforward but unsupervised and model-based. Its CoNN predicts a co-state trajectory $\lambda^\top g(z)$7, and training minimizes stage cost, terminal cost, and an $\lambda^\top g(z)$8-norm co-state regularizer over rollouts generated with the unconstrained PMP control law. The paper refers to this as a norm-optimal co-state solution, and the online controller couples the neural co-state predictor with a small QP (Lian et al., 16 Jul 2025).

ANR generalizes the same structure to safety-constrained control. Its CoNN maps the current state to a finite-horizon co-state sequence, but the online controller solves a convex QP that includes both actuator bounds and Higher-Order Control Barrier Function (HOCBF) constraints. The nominal control remains the Hamiltonian minimizer induced by the learned adjoint; feasibility is handled separately by convex projection (Agboola et al., 15 Jun 2026).

The recurrent formulation differs architecturally but not conceptually. In neural co-state policies, an encoder, recurrent core, and actor-critic heads are trained with PPO plus a co-state alignment loss. The core update can be instantiated either as a standard GRU or as a continuous-time recurrent neural network with discrete-time leaky integration,

$\lambda^\top g(z)$9

and the actor mean is

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$0

This places co-state structure in the recurrent dynamics rather than in an explicit output trajectory (Leeftink et al., 6 May 2026).

4. Training objectives and projection mechanisms

The training objectives differ substantially across the variants, and these differences largely determine what “projection” means operationally.

In NCPR, training is self-supervised and model-free in the sense that the learner does not assume explicit knowledge of $H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$1 and $H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$2. The loss combines accumulated quadratic stage cost,

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$3

terminal cost

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$4

and a regularization term on the predicted projected co-state sequence. Two regularizers are reported: a uniform penalty

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$5

and a discounted penalty that weights later projected co-states more heavily. During constrained training, inputs are simply clipped to the admissible range (Lian et al., 1 Aug 2025).

In CoNN, training is supervised by TPBVP data. The prediction loss is the mean squared error between predicted and optimal co-state sequences,

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$6

and a continuity loss enforces consistency with both state and co-state dynamics by comparing integrated trajectories against ground-truth optimal trajectories one step later. This directly biases the network toward physically and adjoint-consistent outputs (Lian et al., 1 Mar 2025).

In NCR and ANR, the training is unsupervised but model-based. NCR minimizes stage cost, terminal cost, and the co-state regularizer

$H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$7

with no explicit co-state labels. ANR uses an analogous rollout-based objective with stage cost, terminal cost, and co-state regularization, and backpropagates through the analytic Hamiltonian minimizer and the system rollout. In both cases, the adjoint equation $H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$8 is not imposed as an explicit residual term; instead, co-state structure is induced implicitly by cost minimization under the system dynamics (Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026).

The recurrent policy formulation introduces a different projection mechanism. There, the critic $H(x,u,\lambda,t)=L(x,u,t)+\lambda^\top f(x,u,t),$9 yields a target

$$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$0

and the hidden state is aligned directionally via cosine loss,

$$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$1

Because hidden states are bounded while co-states may not be, the loss aligns directions rather than magnitudes. The paper explicitly interprets $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$2 as a projection of the true co-state onto the representational manifold of the RNN (Leeftink et al., 6 May 2026).

A broader projection-based antecedent exists outside optimal control proper. “Learning Physical Constraints with Neural Projections” introduces a learned constraint function $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$3 and an iterative projection operator

$$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$4

where $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$5 is a Lagrange-multiplier-like dual variable. A plausible conceptual implication is that projection-based neural correction and co-state-based neural regulation share a common primal-dual structure, even though this work targets physical simulation rather than Hamiltonian optimal control (Yang et al., 2020).

5. Online regulation, constraints, and safety

At runtime, NCPR and related regulators operate in receding-horizon feedback form. The current state is measured, a neural network predicts a co-state-related quantity over a short horizon, the first element is extracted, and a small QP produces the control applied at the current sampling instant.

In NCPR, the online QP is

$$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$6

where $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$7 is the first row of the CPNN output. Since the Hessian is $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$8, the QP is convex, and for box-constrained inputs it is extremely small: its dimension equals the input dimension $$\dot{x}(t)=\frac{\partial H}{\partial \lambda}, \qquad \dot{\lambda}(t)=-\frac{\partial H}{\partial x},$$9 (Lian et al., 1 Aug 2025).

CoNN and NCR use the same structure. In CoNN, the constrained controller solves

$$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$0

which reduces to saturation in the scalar example because $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$1, $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$2, and the unconstrained optimum is $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$3. NCR uses the same Hamiltonian-minimizing QP, and in the unicycle example the optimization variables are $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$4 with bounds $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$5, $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$6 (Lian et al., 1 Mar 2025, Lian et al., 16 Jul 2025).

ANR extends the same mechanism from input constraints to explicit safety constraints. For the unicycle with state $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$7 and control $$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$8, each obstacle defines a barrier

$$u^*(t)=\arg\min_{u(t)\in\mathcal{U}} H(x^*(t),u(t),\lambda^*(t),t).$$9

Because the control appears at relative degree two, the controller imposes HOCBF constraints

$$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$0

which become affine inequalities in $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$1. The resulting QP minimizes the Hamiltonian term

$$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$2

subject to both actuator bounds and HOCBF constraints (Agboola et al., 15 Jun 2026).

The recurrent co-state formulation implements Hamiltonian minimization more implicitly. During training, the actor head is stochastic,

$$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$3

but the readout is interpreted as a Hamiltonian-minimizing control law, and for quadratic control cost the closed form

$$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$4

makes the same point directly. The provided interpretation notes that in a regulator design one can drop the sampling and use $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$5 as deterministic control (Leeftink et al., 6 May 2026).

6. Empirical behavior, limitations, and research directions

The empirical record across the cited papers is consistent on three points: near-NMPC regulation quality, large computational gains, and materially better out-of-distribution behavior than direct RL baselines on the reported tasks.

For the explicit NCPR formulation, the unicycle reference-tracking experiments use $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$6, $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$7, $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$8, $$H(\mathbf{z},\mathbf{u},\boldsymbol{\lambda},t) = \mathbf{z}^\top Q \mathbf{z} + \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^\top \big(f(\mathbf{z})+g(\mathbf{z})\mathbf{u}\big),$$9 s, and horizon $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$0. On a seen initial state $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$1, the reported convergence errors are $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$2 for $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$3, $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$4 for $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$5, $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$6 for PPO, and $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$7 for MPC. On the out-of-distribution state $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$8, the errors are $$\mathbf{u}^*(t) = -\frac{1}{2}R^{-1}g^\top(\mathbf{z}(t))\boldsymbol{\lambda}^*(t),$$9, $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$0, $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$1, and $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$2, respectively. Online runtime is about $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$3 ms per step for NCPR versus $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$4–$$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$5 ms for nonlinear MPC. In the pendulum swing-up task, NCPR is trained on 100 states from $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$6, yet on out-of-distribution initial conditions it reports convergence errors of $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$7 or $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$8, whereas PPO reports $$\mathbf{u}^*(t) = \arg\min_{\mathbf{u}(t)\in\mathcal{U}} \left( \mathbf{u}^\top R \mathbf{u} + \boldsymbol{\lambda}^{*\top}(t) g(\mathbf{z}(t)) \mathbf{u} \right).$$9, despite requiring $g^\top \lambda$0 training steps versus $g^\top \lambda$1 for the CPNN setup (Lian et al., 1 Aug 2025).

The supervised CoNN controller is evaluated on the scalar nonlinear system

$g^\top \lambda$2

with 101 training initial conditions in $g^\top \lambda$3, $g^\top \lambda$4 s, $g^\top \lambda$5 s, and output horizon $g^\top \lambda$6. The constrained controller closely matches a direct collocation solution for state and control, and under disturbances of $g^\top \lambda$7 at $g^\top \lambda$8 s, $g^\top \lambda$9 at $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$00 s, and $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$01 at $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$02 s, the disturbed trajectories are driven back to the target $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$03 by $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$04 s (Lian et al., 1 Mar 2025).

NCR evaluates a unicycle with $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$05, $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$06, $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$07, $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$08 s, and horizon $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$09. Reported average times per step are $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$10 ms for NCR versus $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$11 ms for nonlinear MPC in Case A, $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$12 ms versus $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$13 ms in Case B, and $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$14 ms versus $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$15 ms in Case C. In the out-of-training-domain cases, NCR reports better absolute convergence error and substantially smoother inputs than MPC according to the Mean Squared Derivative metric; for example, in Case B the absolute convergence errors are $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$16 for NCR and $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$17 for MPC, while MSD(input) is $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$18 versus $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$19 (Lian et al., 16 Jul 2025).

ANR reports similar runtime advantages with additional state-safety constraints. For the obstacle-free unicycle, the average control computation time is about $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$20 ms for ANR, about $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$21 ms for SAC with a CBF-QP filter, and $$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$22–$$J = \int_{0}^{t_f} \left( \mathbf{z}^\top Q\mathbf{z} + \mathbf{u}^\top R\mathbf{u} \right) dt + \phi(\mathbf{z}(t_f)).$$23 ms for NMPC depending on the scenario. Success rates are reported as 100% for ANR and 30% for RL on in-distribution obstacle-free trials, 82% for ANR and 38% for RL on in-distribution obstacle trials, and 98% success for ANR on 100 randomized out-of-distribution obstacle trials. With obstacles, NMPC remains slightly more accurate in absolute convergence error, but ANR is faster by more than two orders of magnitude and smoother in the reported challenging scenarios (Agboola et al., 15 Jun 2026).

The recurrent neural co-state policy work addresses a different regime—partial observability rather than explicit input-constrained regulation—but supplies evidence that co-state-structured hidden dynamics improve robustness. On partially observable DMControl tasks with 50% sensor blackout, all methods solve Cartpole Swingup and Ball in Cup; Finger Turn Hard remains difficult; NC-GRU significantly improves asymptotic return and stability over standard GRU on locomotion tasks such as Walker Stand, Walker Walk, Walker Run, and Cheetah Run; and under zero-shot evaluation at 75% masking, all models degrade, but the NCP variants retain their relative advantage (Leeftink et al., 6 May 2026).

The limitations are equally consistent across the literature. The explicit NCPR formulation is presently restricted to finite-horizon quadratic regulation, low-dimensional systems, and input constraints; no rigorous closed-loop stability or convergence proof is given, and training is model-free only in the sense that it avoids explicit parametric system identification while still relying on a simulator or environment rollout (Lian et al., 1 Aug 2025). CoNN, NCR, and ANR require accurate models for rollout or TPBVP generation, do not furnish general Lyapunov or suboptimality guarantees, and have been demonstrated primarily on low-dimensional examples such as scalar systems and unicycles (Lian et al., 1 Mar 2025, Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026). The recurrent policy approach is explicitly model-free but critic-gradient targets are noisy, co-state magnitudes are only approximated directionally, and performance gains are task-dependent, helping locomotion more than some contact-heavy tasks (Leeftink et al., 6 May 2026).

The most immediate research directions already identified in these papers are explicit incorporation of state constraints into neural co-state regulators, better enforcement of adjoint dynamics through residual losses, extension beyond quadratic regulation, scaling to higher-dimensional systems such as robotic manipulators and aerial vehicles, and hybrid designs that use learned co-states to warm-start or regularize MPC or trajectory-optimization solvers (Lian et al., 1 Mar 2025, Lian et al., 16 Jul 2025, Agboola et al., 15 Jun 2026, Lian et al., 1 Aug 2025). Across these variants, the central claim remains the same: learning the co-state or its projection provides a structured alternative to direct policy learning, and the QP or equivalent Hamiltonian-minimization layer separates optimality encoding from feasibility enforcement.