CBF-CLF-Informed Loss for Safe Control

• The paper demonstrates how augmenting imitation loss with soft penalties from CBF and CLF certificates embeds safety and stability directly into the learning objective.
• It leverages control barrier and Lyapunov functions to enforce constraints like keep-out zones and approach corridors using DAgger refinement and curriculum weighting.
• Empirical results show reduced filter interventions and real-time control performance comparable to NMPC experts while significantly lowering online computation.

Searching arXiv for the cited work and closely related CBF/CLF papers. CBF-CLF-informed loss denotes a training objective in which imitation loss is augmented by soft penalties derived from Control Barrier Function (CBF) and Control Lyapunov Function (CLF) conditions, so that safety and stability certificates enter the learning problem directly rather than only appearing at run time. In the formulation introduced for spacecraft close proximity operations, CBFs provide safety certificates, CLFs provide stability as unified design principles across data generation, training, and deployment, and the learned controller is paired at deployment with a lightweight one-step CBF-CLF quadratic program that minimally adjusts the learned control input to satisfy hard safety constraints while encouraging stability (Meinert et al., 19 Mar 2026). A related but distinct use of CBF-CLF-informed objectives appears in reinforcement learning under model uncertainty, where the learning objective penalizes estimation errors in CLF, CBF, and other control-affine constraints inside an RL-enhanced CBF-CLF-QP (Choi et al., 2020).

1. Certificate-theoretic basis

The construction begins from control-affine dynamics

$$\dot x = f(x) + g(x)u,\qquad x\in\mathbb{R}^n,\;u\in\mathbb{R}^m.$$

A continuously differentiable scalar $h(x)$ defines the zero-superlevel safe set

$S \coloneqq \{x\mid h(x)\ge 0\}.$

For a relative-degree-2 constraint, the extended barrier function is defined as

$H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$

with time derivative under input $u$

$\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$

where $\alpha(H)=\gamma H$ with $\gamma>0$. The single-step CBF condition is written compactly as

$$\mathcal{H}(x,u)\coloneqq L_fh(x)+L_gh(x)u+\gamma H(x)\ge 0.$$

Whenever $\mathcal{H}(x,u)\ge 0$, the next state is guaranteed to stay inside the corresponding safe set (Meinert et al., 19 Mar 2026).

In the spacecraft scenario, two barrier constraints are used. The spherical keep-out-zone is encoded by

$h(x)$0

and the conical approach corridor by

$h(x)$1

Each yields $h(x)$2 via the same construction. The paper states that the relative-degree-2 setting arises from the second-order Clohessy-Wiltshire dynamics.

The stability certificate is a quadratic Lyapunov function centered at a decision point $h(x)$3,

$h(x)$4

where $h(x)$5 solves the discrete algebraic Riccati equation for the linearized dynamics. A standard CLF condition of relative degree one reads

$h(x)$6

for some $h(x)$7. To concentrate the stability enforcement only in a neighborhood of $h(x)$8, a state-dependent decay rate is introduced through

$h(x)$9

The combined CLF residual is then

$S \coloneqq \{x\mid h(x)\ge 0\}.$0

Whenever $S \coloneqq \{x\mid h(x)\ge 0\}.$1, the next step is guaranteed to reduce $S \coloneqq \{x\mid h(x)\ge 0\}.$2, driving $S \coloneqq \{x\mid h(x)\ge 0\}.$3.

2. Analytical form of the loss

In plain behavior cloning, the mean-squared imitation loss is

$S \coloneqq \{x\mid h(x)\ge 0\}.$4

where $S \coloneqq \{x\mid h(x)\ge 0\}.$5 is the nonlinear Model Predictive Control expert and $S \coloneqq \{x\mid h(x)\ge 0\}.$6 is the neural policy. The CBF-CLF-informed construction augments this objective with two soft penalties (Meinert et al., 19 Mar 2026).

The CBF penalty is

$S \coloneqq \{x\mid h(x)\ge 0\}.$7

and the CLF penalty is

$S \coloneqq \{x\mid h(x)\ge 0\}.$8

Both penalties are zero whenever the corresponding constraint is satisfied at the current sample. The full training objective is

$S \coloneqq \{x\mid h(x)\ge 0\}.$9

In the reported experiments, the weights are chosen as $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$0, $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$1, and $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$2 so that all three loss terms initially lie in the same order of magnitude. This choice is important because the informed loss is not merely a regularized imitation objective; it is designed so that certificate violations appear explicitly in the optimization landscape. The practical consequence reported in the paper is that training internalizes information about future safety-filter activations and local stability near decision points rather than learning only the expert’s nominal action map.

3. Integration with DAgger and curriculum weighting

The training procedure is divided into two phases. In pre-training, $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$3 is trained for $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$4 on $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$5 only, using the initial dataset $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$6 collected by the NMPC + CBF expert. In DAgger refinement, the method performs $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$7 iterations, each consisting of mixed-policy rollouts, dataset aggregation, one epoch of training on the enlarged dataset, and an update of the certificate weights (Meinert et al., 19 Mar 2026).

At iteration $H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$8, the rollout mixing coefficient is set to

$H(x)=h(x)+\|\dot h(x)\|\,\dot h(x)/(2u_{\max}),$9

The closed-loop rollouts use the mixed policy

$u$0

for $u$1 trajectories, with new states and expert labels collected into $u$2. The policy is then trained for one epoch on the enlarged dataset by minimizing $u$3. After each refinement round, the curriculum updates are

$u$4

The stated purpose of this schedule is that the network gradually transitions from pure imitation to a strictly certificate-respecting policy, while the curriculum weights ensure the network first learns the coarse mimicry, then refines safety and stability. This organization places the informed loss inside a broader data-aggregation scheme rather than treating it as a static supervised objective. A plausible implication is that the loss is intended to shape closed-loop behavior under distribution shift, not only per-sample action matching.

4. Optimization mechanics and convergence behavior

All three loss terms are smooth except for the hinge at zero. In implementation, $u$5 is represented by the standard ReLU, whose sub-gradient is well defined. Gradients $u$6 are computed by automatic differentiation through the neural network, the barrier and Lyapunov residuals $u$7 and $u$8, and the Lie derivatives $u$9 and $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$0 obtained in closed form for the CW dynamics (Meinert et al., 19 Mar 2026).

The optimizer is AdamW with learning rate $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$1, weight decay $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$2, batch size $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$3, gradient clipping to $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$4, and dropout $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$5 to avoid overfitting. The reported empirical convergence pattern is staged: imitation loss rapidly decreases in early epochs, and the certificate losses $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$6 and $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$7 decrease subsequently once their weights grow. The paper attributes stable convergence to the combination of curriculum weighting and DAgger.

This optimization structure is technically notable because the certificate quantities are not used only as post hoc diagnostics. They are differentiated through during training, making the Lie-derivative structure part of the gradient path. In that sense, CBF-CLF-informed loss belongs to a class of objectives in which formal control certificates are converted into trainable residual penalties.

5. Role at deployment and interpretation of safety guarantees

The deployment architecture retains a lightweight one-step CBF-CLF quadratic program that minimally adjusts the learned control input to satisfy hard safety constraints while encouraging stability (Meinert et al., 19 Mar 2026). The overall framework therefore separates two functions. The learned policy provides runtime-efficient control that is intended to reduce certificate violations before filtering. The one-step quadratic program provides hard safety enforcement at execution time.

The numerical experiments are reported for ESA-compliant close proximity operations, including fly-around with a spherical keep-out zone and final approach inside a conical approach corridor, using the Basilisk high-fidelity simulator with nonlinear dynamics and perturbations. The paper states that numerical experiments indicate stable convergence to decision points and strict adherence to safety under the filter, with task performance comparable to the NMPC expert while significantly reducing online computation. A runtime analysis further demonstrates real-time feasibility on a commercial off-the-shelf processor, supporting onboard deployment for safety-critical on-orbit servicing.

This reported behavior helps distinguish two notions that are often conflated. The training loss promotes satisfaction of CBF and CLF conditions on sampled data, whereas strict adherence to safety is reported under the runtime filter. This suggests that the informed loss is presented as a mechanism for reducing the burden on the safety filter rather than eliminating the need for certified online correction.

6. Empirical validation, ablation structure, and related formulations

The ablation study compares five controllers: NMPC with naïve position constraints (infeasible), LQR + CBF-CLF safety filter (safe but chattering), NMPC + CBF (optimal but too slow), pure BC (violates certificates, unstable near decision points), and the full CBF-CLF-informed IL + filter (safe, stable, fast) (Meinert et al., 19 Mar 2026). Runtime results in Table I report that the full controller runs in approximately $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$8 ms on a desktop and $\dot H(x,u)=L_fh(x)+L_gh(x)u+\alpha(H(x)),$9 ms on an ESP32, well under the $\alpha(H)=\gamma H$0 ms control period, versus $\alpha(H)=\gamma H$1 ms for LQR + filter and $\alpha(H)=\gamma H$2 ms for pure NMPC. Trajectory quality in Fig. 4a is reported as closely tracking the NMPC expert while strictly avoiding the KOZ and staying inside the approach cone.

The same section also isolates the effect of the informed penalties on safety-filter usage. When training solely on imitation loss, the magnitude of CBF violations, and hence filter activations, decreases slowly over DAgger iterations. By contrast, adding the $\alpha(H)=\gamma H$3 penalties yields a dramatic, order-of-magnitude reduction in both CBF loss and filter intervention magnitude already after one or two DAgger rounds. The paper summarizes the resulting policy as one that requires far fewer expert queries to eliminate unsafe behavior, minimizes reliance on the run-time safety filter, and meets real-time on-board computation constraints without sacrificing the performance of the original NMPC expert.

A related formulation in reinforcement learning uses CBF-CLF information differently. In that setting, model-plant mismatch introduces uncertainty terms in $\alpha(H)=\gamma H$4, each $\alpha(H)=\gamma H$5, and each additional control-affine constraint $\alpha(H)=\gamma H$6, and the RL objective is constructed as a weighted sum of the negative estimation errors plus a terminal failure penalty (Choi et al., 2020). The actor outputs the $\alpha(H)=\gamma H$7- and $\alpha(H)=\gamma H$8-terms that enter the RL-CBF-CLF-QP constraints linearly, and training is carried out with Deep Deterministic Policy Gradient, experience replay, and soft target updates. The distinction is substantive: the imitation-learning formulation penalizes direct CBF and CLF residual violations of the learned action, whereas the RL formulation penalizes errors in learned models of certificate derivatives and constraint functions. This suggests that “CBF-CLF-informed loss” names a broader design pattern in which safety and stability certificates are encoded directly in the optimization objective, even though the exact loss semantics depend on whether the learner is imitating an expert or correcting uncertainty inside a QP-based controller.