Neural Co-State Regulator (NCR)

• Neural Co-State Regulator (NCR) is a real-time optimal control framework that uses a neural network to predict finite-horizon co-state trajectories and enforce input constraints via a lightweight QP.
• It leverages unsupervised, PMP-informed training to bypass the computationally intensive TPBVP, combining offline learning with efficient online optimization.
• Empirical studies, such as on a unicycle model, show NCR delivers smoother control and up to 100× faster computation compared to traditional nonlinear MPC.

Searching arXiv for the cited NCR paper and closely related work to ground the article in current literature. Neural Co-State Regulator (NCR) is a learning-based real-time optimal control framework for nonlinear optimal control problems with input constraints in which a neural network predicts a finite-horizon co-state trajectory and a small quadratic program (QP) extracts the control input by minimizing the Hamiltonian subject to actuator bounds. In the formulation introduced in "Neural Co-state Regulator: A Data-Driven Paradigm for Real-time Optimal Control with Input Constraints" (Lian et al., 16 Jul 2025), NCR combines a co-state neural network with an online control-input QP solver, thereby replacing the online two-point boundary value problem (TPBVP) associated with Pontryagin’s Minimum Principle (PMP) by offline learning plus lightweight online optimization.

1. Problem formulation and control objective

NCR is posed for a nonlinear control system with state $x(t)\in\mathbb R^p$ and control $u(t)\in\mathbb R^q$,

$\dot x(t)=f\bigl(x(t),u(t)\bigr),$

subject to input constraints

$u(t)\in\mathcal U\quad\text{for all }t.$

The objective is a finite-horizon quadratic regulation problem over $[0,T]$,

$$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$

where $Q\succeq0$, $R\succ0$, and $\phi(x(T))$ is a quadratic terminal cost (Lian et al., 16 Jul 2025).

This formulation places NCR within the class of indirect optimal-control methods grounded in PMP rather than direct policy approximation. The central design choice is to learn the co-state trajectory rather than the control law itself. In the NCR presentation, the co-state trajectory $\lambda(t)$ is described as carrying full optimality information, while direct online solution of the TPBVP is characterized as too slow for real-time use (Lian et al., 16 Jul 2025).

A key practical implication is that actuator feasibility is separated from co-state prediction. Input constraints are always enforced online by the QP, whereas the co-state predictor is trained without embedding those constraints directly into the network output. This separation is one of the defining structural features of NCR.

2. Pontryagin structure and the role of the co-state

NCR is explicitly derived from Pontryagin’s Minimum Principle. The Hamiltonian is defined as

$u(t)\in\mathbb R^q$0

with co-state $u(t)\in\mathbb R^q$1. For an optimal triple $u(t)\in\mathbb R^q$2, PMP yields the necessary conditions

$u(t)\in\mathbb R^q$3

with terminal boundary condition

$u(t)\in\mathbb R^q$4

and pointwise Hamiltonian minimization

$u(t)\in\mathbb R^q$5

(Lian et al., 16 Jul 2025).

In NCR, the co-state is not treated as an auxiliary variable used only in analysis; it is the primary learned object. The neural network, denoted CoNN$u(t)\in\mathbb R^q$6, maps the current state $u(t)\in\mathbb R^q$7 to a finite-horizon co-state sequence

$u(t)\in\mathbb R^q$8

At runtime, only the first element $u(t)\in\mathbb R^q$9 is used to compute the control. This receding-horizon use of the co-state prediction preserves the feedback character of the controller while avoiding online TPBVP solution (Lian et al., 16 Jul 2025).

The conceptual distinction from conventional neural control policies is therefore structural: NCR learns an adjoint object that encodes first-order optimality and then reconstructs the input by constrained Hamiltonian minimization. This suggests a closer affinity to indirect optimal control and adjoint-based methods than to end-to-end policy learning.

3. Network architecture and unsupervised PMP-informed training

The NCR architecture is a fully connected feed-forward neural network with input $\dot x(t)=f\bigl(x(t),u(t)\bigr),$0 and output given by a flattened co-state sequence $\dot x(t)=f\bigl(x(t),u(t)\bigr),$1. The paper describes representative architectures with $\dot x(t)=f\bigl(x(t),u(t)\bigr),$2–$\dot x(t)=f\bigl(x(t),u(t)\bigr),$3 hidden layers of width $\dot x(t)=f\bigl(x(t),u(t)\bigr),$4–$\dot x(t)=f\bigl(x(t),u(t)\bigr),$5 and ReLU activations. Training data consist of $\dot x(t)=f\bigl(x(t),u(t)\bigr),$6 samples of states uniformly drawn from a region $\dot x(t)=f\bigl(x(t),u(t)\bigr),$7, and the horizon $\dot x(t)=f\bigl(x(t),u(t)\bigr),$8 is chosen to match the desired control look-ahead (Lian et al., 16 Jul 2025).

Training is unsupervised and PMP-informed. For each sampled state $\dot x(t)=f\bigl(x(t),u(t)\bigr),$9, the network predicts $u(t)\in\mathcal U\quad\text{for all }t.$0, and an analytic unconstrained control sequence is reconstructed from the stationarity condition $u(t)\in\mathcal U\quad\text{for all }t.$1: $u(t)\in\mathcal U\quad\text{for all }t.$2 Using this sequence, the dynamics are rolled out to define three loss components: $u(t)\in\mathcal U\quad\text{for all }t.$3

$u(t)\in\mathcal U\quad\text{for all }t.$4

and the co-state regularizer

$u(t)\in\mathcal U\quad\text{for all }t.$5

The total loss is

$u(t)\in\mathcal U\quad\text{for all }t.$6

The NCR description states that, by penalizing the true stage cost and terminal cost for any predicted co-state, this objective implicitly enforces agreement with the co-state dynamics $u(t)\in\mathcal U\quad\text{for all }t.$7 and Hamiltonian minimization (Lian et al., 16 Jul 2025).

The training loop consists of sampling $u(t)\in\mathcal U\quad\text{for all }t.$8 from $u(t)\in\mathcal U\quad\text{for all }t.$9, predicting $[0,T]$0, rolling out the dynamics with the analytic $[0,T]$1, adding the regularization term, and backpropagating to update $[0,T]$2. Input constraints are not enforced during training; the co-state prediction is described as agnostic to them, while the online QP always enforces them (Lian et al., 16 Jul 2025).

This training protocol contrasts with the earlier Co-state Neural Network (CoNN) method, which used data generated from numerical solutions of unconstrained TPBVPs together with a supervised prediction loss and a continuity loss (Lian et al., 1 Mar 2025). Relative to that predecessor, NCR is distinguished by its unsupervised formulation.

4. Online control extraction and constrained optimality

Once $[0,T]$3 is available, NCR computes the constrained control input by solving a small quadratic program: $[0,T]$4 In standard QP form, the decision variable is $[0,T]$5, the quadratic cost matrix is $[0,T]$6, the linear term is $[0,T]$7, and the constraints may be box or polyhedral, written as $[0,T]$8 (Lian et al., 16 Jul 2025).

This QP is the online mechanism by which NCR enforces both actuator feasibility and the PMP Hamiltonian minimization condition. The framework summary states that solving this QP at each step yields the constrained, PMP-optimal input in real time (Lian et al., 16 Jul 2025).

The runtime workflow is therefore compact. The current state is measured, CoNN$[0,T]$9 predicts the finite-horizon co-state sequence, the first co-state vector is extracted, and a low-dimensional convex QP produces the control applied to the plant. In the summary of the framework, this arrangement is described as replacing the online TPBVP or large-scale nonlinear program of model predictive control (MPC) with a lightweight co-state prediction network plus a small QP, while guaranteeing satisfaction of input bounds and preserving optimality in the PMP sense (Lian et al., 16 Jul 2025).

A recurrent misconception is that NCR removes optimization from the feedback loop entirely. The available descriptions do not support that interpretation. The neural network removes the online co-state boundary-value solve, but the controller still performs online optimization through the QP that enforces constraints and Hamiltonian minimization.

5. Unicycle case study and empirical characteristics

The principal empirical demonstration in the NCR paper is a unicycle model with dynamics

$$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$0

The cost uses $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$1, $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$2, and terminal weight $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$3. Input constraints are

$$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$4

with sample time $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$5 and horizon $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$6 (Lian et al., 16 Jul 2025).

The NCR controller is compared with an expert nonlinear MPC solver. Averaged over three test cases, the reported performance metrics are:

Metric	MPC	NCR
Case B convergence error $$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$7	0.14	0.06
Case B control smoothness (mean squared derivative)	17.6	5.3
Computation time per step	$$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$8–$$J=\int_{0}^{T}\bigl[x(t)^\top Q\,x(t)+u(t)^\top R\,u(t)\bigr]\,dt+\phi\bigl(x(T)\bigr),$$9 ms	$Q\succeq0$0–$Q\succeq0$1 ms

The reported conclusions are specific. NCR matches or improves upon MPC in convergence error; cases B and C show markedly smoother inputs under NCR; and the computation time reduction is approximately $Q\succeq0$2, described in the abstract as two orders of magnitude less computational time than nonlinear MPC (Lian et al., 16 Jul 2025).

The same study states that, because NCR training does not rely on expert nonlinear control solvers that are often suboptimal, NCR can produce solutions that outperform the nonlinear MPC solver in convergence error and input trajectory smoothness even for system conditions outside the original training domain (Lian et al., 16 Jul 2025). The summary further reports that even for initial states outside the training set and nonzero target references, NCR reliably drives the unicycle to the goal with performance on par or better than MPC (Lian et al., 16 Jul 2025).

A plausible implication is that the generalization mechanism is tied to the learned mapping from current state error to co-state guess, rather than to memorization of nominal trajectories. That interpretation is consistent with the explicit statement that CoNN directly maps the current state error to a co-state guess.

6. Related frameworks, terminology, and conceptual lineage

NCR belongs to a small family of co-state-based neural controllers grounded in PMP. The lineage in the supplied literature includes a supervised predecessor, a model-free projected variant, and a state-constrained extension.

Framework	Core learned quantity	Distinguishing feature
Co-state Neural Network (CoNN)	optimal co-state trajectory	trained from numerical TPBVP solutions
Neural Co-State Regulator (NCR)	finite-horizon co-state sequence	unsupervised PMP-informed training + online QP
Neural Co-state Projection Regulator (NCPR)	projected co-state sequence $Q\succeq0$3	model-free, self-supervised
Adjoint-based Neural Regulator (ANR)	neural co-state policy	runtime projection with actuator limits and CBF safety constraints

The CoNN method of "Co-state Neural Network for Real-time Nonlinear Optimal Control with Input Constraints" learns the mapping from a state to its optimal co-state trajectory over a finite window, trains on numerical TPBVP solutions for unconstrained optimal control problems, and enforces constrained control by a QP (Lian et al., 1 Mar 2025). NCR preserves the co-state-plus-QP decomposition but replaces supervised TPBVP labels with unsupervised cost-based training (Lian et al., 16 Jul 2025).

The Neural Co-state Projection Regulator (NCPR) modifies the learned target itself: instead of $Q\succeq0$4, it predicts a finite-horizon sequence of projected co-states $Q\succeq0$5, then solves a small QP online. The NCPR paper characterizes this as a model-free paradigm, reports tests on unicycle reference tracking and pendulum swing-up, and compares against reinforcement learning baselines and, for context, a model-based controller in the unicycle setting (Lian et al., 1 Aug 2025).

The Adjoint-based Neural Regulator (ANR) extends the same basic idea to hard state-safety constraints enforced by a runtime convex projection with control barrier function constraints. Its summary states that ANR is also referred to as the Neural Co-State Regulator (NCR), indicating acronym overlap in the literature rather than a wholly separate conceptual family (Agboola et al., 15 Jun 2026).

A separate use of the term appears in "Neural Co-state Policies: Structuring Hidden States in Recurrent Reinforcement Learning," where the NCR idea denotes interpreting recurrent hidden states as discrete approximations of PMP co-states and regularizing them with a co-state loss inside actor-critic or PPO training (Leeftink et al., 6 May 2026). That usage concerns partially observable reinforcement learning rather than the feed-forward co-state-regression-and-QP architecture of NCR for nonlinear regulation. The shared terminology reflects a common PMP interpretation of latent adjoint structure, but the training setting and controller realization differ.

7. Generalization, limitations, and open problems

The NCR summary attributes generalization to the fact that CoNN directly maps the current state error to a co-state guess, and reports empirical robustness to mild model mismatches, with errors in the $Q\succeq0$6-model degrading performance gracefully (Lian et al., 16 Jul 2025). The same source states that the method generalizes to unseen initial conditions and changing references.

The limitations identified in the NCR presentation are explicit. The framework requires known dynamics $Q\succeq0$7 and $Q\succeq0$8. The training horizon $Q\succeq0$9 must cover the effective “stopping time.” There is no explicit handling of state constraints, which would require an additional QP or barrier. The co-state regularization weight $R\succ0$0 and network size are hyperparameters that must be tuned. Extension to stochastic or time-varying dynamics remains future work (Lian et al., 16 Jul 2025).

These limitations clarify the scope of the original NCR formulation. In particular, it is not a model-free method in the sense later claimed for NCPR, and it does not natively address state-safety constraints in the manner later formalized by ANR with higher-order control barrier functions (Lian et al., 1 Aug 2025, Agboola et al., 15 Jun 2026). This suggests a branching research program: one branch relaxes model dependence through projected co-state learning, while another augments feasibility through safety-constrained projection.

Within that program, NCR occupies a specific methodological position: a data-driven but model-based, unsupervised, PMP-grounded regulator that predicts co-states offline and resolves constrained Hamiltonian minimization online. Its significance lies in showing that the adjoint trajectory can function as the learned intermediate representation through which real-time constrained optimal control is reconstructed.