Retrospective-Cost Adaptive Control (RCAC)

• RCAC is a model-free, data-driven adaptive control method that computes controller gains in real time by minimizing a cost function based on historical tracking errors and control efforts.
• It employs a recursive least-squares update that integrates error regressors and regularization terms, making it suitable for digital, sampled-data, and continuous-time applications.
• RCAC has been validated in applications such as UAV autopilots, PID autotuning for multicopters, and thrust regulation, demonstrating significant improvements in tracking performance and robustness.

Retrospective-Cost Adaptive Control (RCAC) is a model-free, data-driven adaptive control methodology that adjusts controller parameters online by minimizing a cost function constructed from historical tracking errors and control efforts. Unlike model-based direct adaptive control, RCAC leverages the plant's own input–output history, requiring no a priori plant model. At each sampling instant, the algorithm computes the controller gains that would have minimized a retrospective, task-relevant cost function over actual observed data, yielding an efficient recursive least-squares-type parameter update. RCAC is now established as a robust framework for digital, sampled-data, and continuous-time adaptive controller synthesis, with validated applications ranging from unmanned aerial vehicle (UAV) autopilots to quantum system control, and solid-fuel ramjet thrust regulation.

1. Core Principles and Mathematical Formulation

The defining feature of RCAC is its retrospective cost function, which evaluates how well a candidate controller parameter vector θ would have performed in the immediate past. For a digital controller parameterized as $u_k = \phi_k \theta_k$—where $\phi_k$ is the regressor based on past errors, integrals, and feedforward terms—the retrospective prediction error is formulated as

$$\hat z_k(\theta) = z_k + \sigma \left( \phi_{k-1}\theta - u_{k-1} \right)$$

with $z_k$ the tracking error and $u_{k-1}$ the previous control action; $\sigma$ is set to the sign of the leading numerator coefficient of the plant (Lee et al., 2021). The quadratic retrospective cost aggregates these errors along with regularization and control magnitude penalties:

$$J_k(\theta) = \sum_{i=0}^k \hat z_i(\theta)^T R_z \hat z_i(\theta) + (\phi_k\theta)^T R_u (\phi_k\theta) + (\theta - \theta_0)^T P_0^{-1} (\theta - \theta_0)$$

where $R_z \succ 0$ and $R_u \succ 0$ are user-chosen weights, and $P_0 \succ 0$ is the regularization matrix encoding prior trust in initial gains.

The minimization of $J_k(\theta)$ at each $k$ yields a closed-form, recursive least-squares update:

$$\theta_{k+1} = \theta_k - \sigma P_{k+1} \phi_{k-1}^T R_z [z_k + \sigma (\phi_{k-1}\theta_k - u_{k-1})] - P_{k+1} \phi_k^T R_u (\phi_k \theta_k)$$

with $P_{k+1}$ updated via a matrix-inversion-lemma-based recursion (Lee et al., 2021, Goel et al., 2020). This obviates the need for an explicit forgetting factor, as adaptation speed and numerical conditioning are governed by $R_z$, $R_u$, and $P_0$.

RCAC generalizes to multiple-input multiple-output (MIMO) and continuous-time settings by filtering regressors and formulating analogous cost integrals, with real-time update laws derived via differentiation under the integral sign (Mirtaba et al., 6 Jun 2025, Delgado et al., 2024). The approach remains based on direct minimization of retrospective cost evaluated on measured input–output signals.

2. Integration into Control Architectures and Algorithmic Variants

RCAC is typically embedded as an augmentation to fixed-gain architectures, such as autopilots for PX4-based UAVs or thrust regulators for propulsion systems. In nested-loop PX4 autopilots, RCAC augments fixed-gain controllers (P, PI, or PID) by adaptively generating corrective signals $u_\Theta$, $u_\Phi$, or $u_{\omega,PI}$, with each adaptive signal produced by a dedicated RCAC instance using the corresponding error, regressor, and prior control command (Lee et al., 2021, Spencer et al., 2021, Goel et al., 2020). The overall commanded control to the actuator is the sum of fixed and adaptive components.

Key variants include:

• Attitude and rate adaptation: Adaptive augmentation is applied separately in bank-rate, pitch-rate, and angular-acceleration (PI/FF) loops.
• Digital PID autotuning: In multicopter applications, all PID (and feedforward) blocks are handled via RCAC, with regressor stacks selected to cover proportional, integral, and derivative actions plus feedforward terms (Spencer et al., 2021).
• Data-driven RCAC (DDRCAC): RCAC is combined with recursive system identification (often via RLS with variable-rate forgetting) to capture nonminimum-phase zeros and other plant characteristics lacking a priori (Islam et al., 2021).
• Continuous-time RCAC (CT-RCAC): The algorithm is extended to continuous-time plants, introducing stable filtering of regressors and leveraging passivity properties for stability claims (Mirtaba et al., 6 Jun 2025, Delgado et al., 2024).

3. Tuning Parameters and Practical Implementation

The RCAC algorithm is governed by a small set of hyperparameters:

• $P_0$ (initial covariance): Sets initial adaptation speed and regularization—larger $P_0$ yields faster adaptation but higher noise sensitivity.
• $R_u$ (control regularization): Penalizes aggressive control actions; used to enforce smooth control.
• $R_z$ (error penalty): Typically set to identity or a scalar, penalizes tracking error.
• $\sigma$ (sign): Matches plant's leading numerator coefficient; crucial for correct adaptation direction.
• Forgetting factor ($\lambda$): Optional, controls weighting of recent data in cost (most RCAC implementations are finite-memory via $P_0$, but forgetting can be included) (Khokhar et al., 6 Nov 2025).

Selection guidelines universally recommend grid search over plausible ranges for $P_0$ and (when relevant) filter time constants, evaluating performance via integral-squared or RMS error metrics (Mirtaba et al., 6 Jun 2025, Khokhar et al., 6 Nov 2025). The same RCAC hyperparameters have been successfully utilized across gain-detuning and actuator-fault scenarios without retuning (Lee et al., 2022, Lee et al., 2021).

Computational requirements are modest; each RCAC block typically updates a low-dimensional covariance and gain vector (e.g., $2 \times 2$ for PI, $3 \times 3$ for PID), with $O(n_\theta^2)$ per update, suitable for real-time deployment on embedded processors (Spencer et al., 2021, Goel et al., 2020).

4. Representative Applications and Performance

RCAC's efficacy has been established through extensive simulation and experimental campaigns:

• Fixed-wing aircraft autopilots: RCAC augmentation recovers nearly nominal flight trajectories in the presence of large gain detuning ($\alpha_g = 0$, $0.5$) and restores path-tracking performance under actuator faults (e.g., a stuck aileron) (Lee et al., 2021, Lee et al., 2022). Quantified, RCAC achieves 40–80% reductions in RMS tracking errors versus detuned baselines (Lee et al., 2021), and cuts cross-track error RMS by roughly 40% in faulted cases (Lee et al., 2022).
• Multicopter PID tuning: A single “learning trajectory” excites all feedback channels, allowing the RCAC autotuner to outperform hand-tuned PX4 autopilots by 30–50% in RMS position-tracking error, across both simulation and hardware tests, and robustly across variations in vehicle mass (Spencer et al., 2021).
• Quadcopter autopilots with unknown dynamics: RCAC-based loops, initialized at zero gains, quickly recover tracking performance even with severe inertia changes; tracking errors match or outperform stock PX4 within seconds (Goel et al., 2020).
• Solid-fuel ramjet thrust regulation: Adaptive PI/PID controllers trained using RCAC achieve $<2\%$ overshoot and reduce rise-time by $50\%$ relative to fixed-gain PI, robustly across large range of thrust command and inlet conditions, without offline retuning (Khokhar et al., 6 Nov 2025, DeBoskey et al., 9 Jun 2025).
• Quantum systems: Continuous-time RCAC drives adaptive PID control of quantum LGKS systems using Uhlmann fidelity as the error, ensuring error convergence under dominance of regularization (Delgado et al., 2024).

5. Stability, Convergence, and Theoretical Guarantees

RCAC ensures several robustness properties under broad conditions:

• The retrospective cost $J_k(\theta_k)$ is nonincreasing along the adaptive update, guaranteeing boundedness of gain vector $\theta_k$ provided $R_z \succ 0$ or persistent excitation in the regressor (Lee et al., 2021, Goel et al., 2020).
• Global asymptotic convergence to the optimal parameter requires persistent excitation of all regressor channels and accurate sign selection for $\sigma$. These conditions are typically satisfied in sufficiently rich maneuvers or by designed excitation trajectories (Spencer et al., 2021, Lee et al., 2022).
• For continuous-time RCAC, passivity-based Lyapunov arguments demonstrate that all signals remain globally bounded and the filtered error converges to zero in the $L_2$ sense (Mirtaba et al., 6 Jun 2025, Delgado et al., 2024).
• In the presence of actuator faults or deeply detuned gains, RCAC adaptively reallocates control authority and stabilizes the system without explicit fault diagnosis or gain scheduling (Lee et al., 2021, Lee et al., 2022).
• No empirical instability or parameter drift has been observed in any experimental implementation, even under aggressive adaptation or system uncertainties (Lee et al., 2022, Spencer et al., 2021).

6. Extensions, Limitations, and Open Problems

RCAC has been extended to accommodate:

• Data-driven target-model shaping: Combining online system identification (via RLS with variable-rate forgetting) with RCAC enables handling of nonminimum-phase zeros and plant transitions, preventing dangerous pole-zero cancellations (Islam et al., 2021).
• Neural network integration: RCAC acts atop machine-learned estimators, such as neural network-based thrust predictors for SFRJ, allowing “sensorless” adaptive control regimes (DeBoskey et al., 9 Jun 2025).
• Hyperparameter optimization: Particle Swarm Optimization and grid search are used in complex plants to robustly select adaptation rates and filter time constants (Mirtaba et al., 6 Jun 2025).

Limitations include:

• Complete theoretical stability in the presence of strong plant nonlinearities remains an open research topic (Lee et al., 2022).
• Explicit robustification against measurement noise or unmodeled fast dynamics requires augmentation of the adaptation law (e.g., $\sigma$-modification) (Mirtaba et al., 6 Jun 2025).
• RCAC depends on the persistent excitation of regressors; in unexcited channels or stagnant flight regimes, adaptation may be slow.

7. Summary Table: RCAC Core Update (Digital, SISO)

Symbol	Definition	Reference
$u_k$	Control input at step $k$	$u_k = \phi_k \theta_k$
$\phi_k$	Regressor (errors, integrals, differences, etc.)	Model structure
$\theta_k$	Adaptive gain vector	Time-varying, updated online
$\hat z_k(\theta)$	Retrospective error	$z_k + \sigma (\phi_{k-1} \theta - u_{k-1})$
$J_k(\theta)$	Retrospective cost function	See above equation
$P_k$	Covariance (gain) matrix	Recursive update

At each $k$, $J_k(\theta)$ is minimized in closed-form for $\theta_{k+1}$ using an RLS-style recursion; all regressor and error signals are measured or constructed from previous outputs and references (Lee et al., 2021, Spencer et al., 2021, Lee et al., 2022, Goel et al., 2020, Mirtaba et al., 6 Jun 2025).

• "An Adaptive Digital Autopilot for Fixed-Wing Aircraft with Actuator Faults" (Lee et al., 2021)
• "Experimental Flight Testing of a Fault-Tolerant Adaptive Autopilot for Fixed-Wing Aircraft" (Lee et al., 2022)
• "An Adaptive PID Autotuner for Multicopters with Experimental Results" (Spencer et al., 2021)
• "Experimental Implementation of an Adaptive Digital Autopilot" (Goel et al., 2020)
• "Adaptive Digital PID Control of a Quadcopter with Unknown Dynamics" (Goel et al., 2020)
• "Data-Driven Retrospective Cost Adaptive Control for Flight Control Application" (Islam et al., 2021)
• "Continuous-Time Output Feedback Adaptive Control for Stabilization and Tracking with Experimental Results" (Mirtaba et al., 6 Jun 2025)
• "Model-free, Learning-based Control of LGKS Quantum System" (Delgado et al., 2024)
• "Computational Modeling and Learning-Based Adaptive Control of Solid-Fuel Ramjets" (Khokhar et al., 6 Nov 2025)
• "An In-situ Solid Fuel Ramjet Thrust Monitoring and Regulation Framework Using Neural Networks and Adaptive Control" (DeBoskey et al., 9 Jun 2025)