A Bayesian Perspective on the Data-Driven LQR

Abstract: The data-driven linear quadratic regulator (ddLQR) is a widely studied control method for unknown dynamical systems with disturbance. Existing approaches, both indirect, i.e., those that identify a model followed by model-based design, and direct, which bypasses the identification step, often rely on the certainty-equivalence principle and therefore do not explicitly account for model uncertainty. In this paper, we propose a Bayesian formulation for both indirect and direct ddLQR that incorporates posterior uncertainty into the control design. The resulting expected cost decomposes into a certainty-equivalence term and a variance-dependent term, providing a principled interpretation of regularization. We further show that the indirect and direct formulations are equivalent under this perspective. The resulting direct method admits a tractable semidefinite program whose size is independent of the data length. Numerical simulations demonstrate improved optimality gap and closed-loop stability, particularly in low-data regimes.

• The paper introduces a Bayesian formulation for ddLQR that integrates posterior uncertainty into control design for enhanced robustness.
• It presents both direct and indirect Bayesian formulations, with the direct approach yielding a tractable semidefinite program independent of data length.
• Numerical results demonstrate improved closed-loop stability and reduced optimality gaps, especially in low-data or high-noise settings.

Bayesian Framework for Data-Driven Linear Quadratic Regulator Design

Introduction

The paper "A Bayesian Perspective on the Data-Driven LQR" (2604.09074) provides a unified Bayesian formulation for the design of data-driven linear quadratic regulators (ddLQR) in scenarios where the underlying state-space model is unknown but data and prior knowledge are available. The work addresses a critical limitation of prevailing ddLQR approaches—neglect of model uncertainty—which typically leads to overconfident or unstable controllers, especially in data-limited or low SNR regimes. The authors develop both direct and indirect Bayesian ddLQR formulations that propagate posterior model uncertainty into the control design via a variance-driven regularization term, and rigorously establish the equivalence of these approaches. Importantly, the resulting direct formulation leads to a tractable semidefinite program (SDP) of dimensions independent of data length, facilitating practical deployment.

Bayesian Formulation of Data-Driven LQR

Traditionally, both indirect (model identification followed by model-based LQR synthesis) and direct (bypassing explicit system identification) ddLQR methods rely on the certainty-equivalence principle. This neglects distributional uncertainty in the parameter estimates, leaving the controller vulnerable to noise and limited data. The Bayesian perspective adopted in this work models the unknown system matrices $(A, B)$ as random variables subject to a matrix normal prior, updating this posterior given observed data $\mathcal{D}$.

Given posterior $p(A,B|\mathcal{D})$ and defining the closed-loop map $A_\mathrm{cl} = A + BK$, the expected infinite-horizon LQR cost can be decomposed as the sum of a nominal (certainty-equivalence) cost and a variance-dependent term arising from parameter uncertainty, yielding a decomposition that fundamentally interprets regularization as a consequence of posterior parameter variance.

Indirect and Direct Bayesian ddLQR Formulations

The indirect Bayesian formulation incorporates the posterior parameterization into the standard LQR synthesis framework. The expected cost naturally decomposes into:

1. Certainty-equivalence cost: Nominal cost under the posterior mean model parameters.
2. Variance regularization: An explicit penalty proportional to the posterior covariance of the estimated parameters, weighted by the steady-state covariance.

The resulting optimization objective is

$$\min_{K,\Sigma \succeq 0}~ \operatorname{Tr}\left((Q + K^\top R K)\Sigma\right) + \lambda \operatorname{Tr}\left(\begin{bmatrix} K^\top & I_n \end{bmatrix}\Psi^{-1} \begin{bmatrix} K \ I_n \end{bmatrix}\Sigma\right)$$

subject to

$$\Sigma = \sigma_w^2 I_n + \hat{A}_{\mathrm{cl}}\Sigma \hat{A}_{\mathrm{cl}}^\top$$

where $\Psi$ encodes the regularized data covariance, and $\lambda$ is a function of noise and data size.

The direct Bayesian formulation leverages a covariance-based parameterization of the feedback gain that eliminates explicit model identification. The associated optimization can be equivalently cast as an SDP whose number of decision variables is independent of the dataset size $T$. This is in contrast to standard direct approaches whose complexity typically scales with data, presenting clear computational advantages for large-scale or online applications.

Regularization Induced by Posterior Uncertainty

A central claim of the paper is that the variance-driven regularization term aligns with ad hoc regularization techniques previously used in ddLQR literature, but does so in a principled and data-dependent manner. This not only removes the need for heuristic coefficient tuning but ensures that the regularization automatically contracts as the available data increases and posterior uncertainty is reduced. In the limiting regime of infinite data (when $\Psi^{-1} \rightarrow 0$), the Bayesian and certainty-equivalent formulations coincide, analytically justifying existing practices.

Numerical Results

The Bayesian ddLQR approach is evaluated on a second-order spring-mass-damper system with stochastic dynamics, both in scenarios of varying regularization strength and different data sizes. Empirical results focus on the optimality gap (relative performance to the theoretical optimum) and closed-loop stability rate.

Figure 1: Effects of the regularization for the covariance-parametrized LQR and for the Bayesian ddLQR.

The stability rate and median optimality gap are notably improved by the Bayesian approach, particularly in low-data regimes or with higher values of regularization. Excessive regularization, however, can degrade performance, especially for non-Bayesian covariance-based regularization schemes. The Bayesian regularizer effectively balances robustness and performance by being informed by actual uncertainty in the parameters.

Figure 2: Effects of the data size for the covariance-parametrized LQR and the Bayesian ddLQR for datasets of length $\mathcal{D}$0.

As the amount of data increases, both Bayesian and covariance-parametrized controllers converge in both stability and optimality gap, reflecting the reduction in posterior uncertainty. This demonstrates that the Bayesian method gracefully interpolates between robust regulation under high uncertainty and nominal LQR as knowledge about the system improves.

Theoretical and Practical Implications

The Bayesian ddLQR formulations address persistent issues of robustness and stability that have limited the adoption of purely data-driven control in practical, safety-critical systems. By providing a principled means of quantifying and acting upon parameter uncertainty, this approach supports the deployment of controllers in domains where only limited experimental data are available or where model identification is unreliable due to noise.

The ability to encode prior knowledge via matrix-normal priors, and the efficient solution via data-size-independent SDPs, make the proposed frameworks well-suited to embedded and online applications. Moreover, the demonstrated equivalence between direct and indirect Bayesian ddLQR formulations establishes a unifying theoretical underpinning for future research in data-driven optimal control.

Future Directions

The methodological framework and empirical findings suggest several avenues for further investigation:

• Adaptive and Online Control: Extension of the Bayesian ddLQR framework to adaptive or online learning settings. This would allow continuous controller refinement and uncertainty quantification as new data arrives, aligning with recent developments in data-enabled policy optimization [zhao2025data].
• Generalization to Nonlinear Systems: While the current approach is tailored to linear systems with quadratic costs, analogous Bayesian formulations could be constructed for more general nonlinear or non-Gaussian settings.
• Robust Performance Guarantees: Derivation of explicit bounds for closed-loop performance and stability under formal robustness guarantees, leveraging the distributional information inherent in the Bayesian approach.
• Integration with Reinforcement Learning: Synergistic exploitation of Bayesian regularization and exploration-exploitation trade-offs in reinforcement learning contexts.

Conclusion

This work provides a comprehensive Bayesian treatment of data-driven LQR synthesis, integrating posterior parameter uncertainty into the control design process and delivering a practical direct formulation with scalable computational complexity. The Bayesian variance-regularized ddLQR framework offers improved empirical performance, interpretability, and robustness in limited data scenarios relative to existing certainty-equivalent approaches. These advances point toward broader adoption of truly reliable and principled data-driven optimal control policies in domains where model learning from data is required.