Direct Data Driven Natural Gradient Descent for Control

Abstract: This paper introduces a novel direct data-driven control framework based on Natural Gradient Descent (NGD) to design interpretable and robust closed-loop policies without requiring explicit model identification. We propose two data-driven NGD formulations that incorporate the closed-loop covariance matrix through the Fisher Information Matrix (FIM), allowing gradient updates to be preconditioned according to the system's intrinsic uncertainty. Leveraging two distinct data-based parameterizations of the closed-loop system, our method enables stability-guaranteed policy synthesis directly from data. We provide theoretical guarantees for contraction and convergence using semidefinite programs (SDPs) and validate our framework in both simulations and on hardware on a ROSbot XL platform. The results demonstrate intuitive features compared to linear-quadratic regulator (LQR) and standard data-driven baselines, particularly in terms of convergence speed, robustness, and control interpretability. This work bridges the gap between trajectory-oriented natural gradient methods and practical data-driven control design.

• The paper presents a novel, data-driven NGD method that synthesizes robust and interpretable controllers directly from input-state data.
• It develops two parameterizations—direct and covariance-based—each guaranteeing closed-loop stability via LMIs and providing explicit convergence bounds.
• Experimental validation on ROSbot XL demonstrates practical benefits, including simplified tuning and effective sim-to-real transfer compared to traditional methods.

Direct Data-Driven Natural Gradient Descent for Control

Introduction

This work formulates a novel framework for synthesizing robust and interpretable feedback controllers using direct data-driven Natural Gradient Descent (NGD). The methodology circumvents explicit model identification and cost tuning by embedding the geometry of the parameter space—expressed via the Fisher Information Matrix (FIM)—into the closed-loop synthesis directly from input-state trajectory data. Two algorithmic instances are developed based on distinct data-based parameterizations, guaranteeing stability and contraction of the closed-loop via Linear Matrix Inequalities (LMIs) and providing complexity bounds on both convergence and data richness.

Data-Driven Control Problem

The foundation is a stochastic, discrete-time, linear time-invariant (LTI) system with unknown $(A,B)$, governed by

$x_{k+1} = Ax_k + Bu_k + \omega_k,$

where $\omega_k$ is i.i.d. Gaussian noise. The goal is robust policy synthesis without model identification or manual cost-weight tuning. The standard control design paradigm—especially LQR—constructs controllers by solving Riccati equations derived from a user-selected cost, typically requiring extensive trial-and-error for acceptable behavior in the presence of uncertainty.

Instead, the NGD approach seeks to shape the entire closed-loop trajectory by enforcing contraction dynamics directly. Data collected from the physical or simulated system (state-input sequences under persistently exciting maneuvers) are used to build parameterizations of the system's transition matrix, either through direct concatenation or sample covariance. This enables end-to-end synthesis without resorting to system identification.

Natural Gradient Descent in Control

NGD modifies gradient descent updates by using the FIM to precondition steps, thereby adjusting updates in directions of high and low uncertainty optimally. Given the system’s stochasticity, the FIM for the mean parameterization is the inverse state covariance, so

$$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$

for expected quadratic cost $V(x_k)$. This yields geometry-aware contraction of state trajectories and ensures robustness to uncertainty, as directions with higher state variance receive larger corrections.

By enforcing closed-loop mean dynamics that follow the NGD flow, the framework achieves interpretable behavior: the single step-size $\alpha$ (as opposed to the multi-parameter $Q$ and $R$ in LQR) directly controls the aggressiveness/stability trade-off.

Data-Based Parameterizations

Two system parameterization routes are considered, both enabling controller design “from data to gain”:

1. Direct Parameterization: System data matrices $U_0$, $X_0$, $x_{k+1} = Ax_k + Bu_k + \omega_k,$0 are used such that a closed-loop parameterization $x_{k+1} = Ax_k + Bu_k + \omega_k,$1 is built by solving a regression equation for $x_{k+1} = Ax_k + Bu_k + \omega_k,$2. The propagated uncertainty through $x_{k+1} = Ax_k + Bu_k + \omega_k,$3 is accounted for explicitly when computing the stationary covariance.
2. Covariance-Based Parameterization: Using the sample covariance $x_{k+1} = Ax_k + Bu_k + \omega_k,$4, with $x_{k+1} = Ax_k + Bu_k + \omega_k,$5 stacking inputs and states, yields a mapping for controller synthesis via an alternative transformation $x_{k+1} = Ax_k + Bu_k + \omega_k,$6. This is well-suited to streaming and large-scale data regimes and exposes the statistical structure of the data more directly.

The two parameterizations correspond to two NGD controller algorithms (Theorem 1 and Theorem 2). Both guarantee $x_{k+1} = Ax_k + Bu_k + \omega_k,$7-contractiveness (contraction in an appropriately chosen Lyapunov function) by solving a tractable SDP.

Theoretical Guarantees and Complexity Bounds

The principal results establish that feasible solutions to the given LMI problems yield robust NGD data-driven controllers with explicit contraction factors $x_{k+1} = Ax_k + Bu_k + \omega_k,$8 and iteration complexity bounds. Specifically, for step-size $x_{k+1} = Ax_k + Bu_k + \omega_k,$9 and stationary covariance $\omega_k$0, the rate of contraction and sample complexity required for the collected data to be rich enough (i.e., for empirical covariance to be positive definite) are explicitly quantified.

For the mean state norm, the following holds:

$\omega_k$1

with the number of iterations to achieve an $\omega_k$2 ball scaling as $\omega_k$3. Data length $\omega_k$4 scales as $\omega_k$5 to ensure sufficient richness with high probability.

Experimental Validation

Experiments are conducted on a four-wheeled Mecanum ROSbot XL robot, both in Gazebo simulation and on hardware. Input-state data are generated using structured, persistently exciting wheel-velocity patterns to guarantee identifiability and model validity within the local linear regime.

A real platform and its simulated counterpart are visualized below:

Figure 1: The physical (left) and simulated ROSbot XL platform (right) used in our simulations and experiments.

The quality of collected data (state space coverage and input diversity) is depicted in:

Figure 2: Data collected from the Gazebo simulation. Left: robot trajectory in the $\omega_k$6 plane during the data collection phase. Right: the corresponding heading angle $\omega_k$7 over time.

Corresponding control inputs used for data collection exhibit structured diversity and respect the linearity regime:

Figure 3: Control inputs (wheel speeds) $\omega_k$8 to $\omega_k$9 applied to the robot in Gazebo during data collection.

Numerical and Empirical Results

Robustness, interpretability, and control effort/convergence trade-offs are systematically benchmarked:

• Step size tuning ($$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$0): The direct NGD parameterization (Theorem 1) allows for highly interpretable and monotonic tuning of closed-loop behavior. Small $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$1 leads to cautious, low-variance, smooth closed-loop behavior; large $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$2 leads to aggressive convergence at the expense of control effort and increased variance.

This is evident from Monte Carlo trajectories at extreme $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$3 regimes:

Figure 4: Monte Carlo $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$4-$$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$5 trajectory and heading for $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$6.

Figure 5: Control inputs $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$7–$$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$8 for $$\mu_{k+1} = \mu_k - \alpha \Sigma_k^{-1} \nabla_{\mu_k} \mathbb{E}[V(x_k)],$$9.

Figure 6: Monte Carlo $V(x_k)$0-$V(x_k)$1 trajectory and heading for $V(x_k)$2.

Figure 7: Control inputs $V(x_k)$3–$V(x_k)$4 for $V(x_k)$5.

• Direct vs. Covariance Parameterization: The covariance-based NGD controller (Theorem 2) demonstrates higher sensitivity to hyperparameter selection and data quantity. For small $V(x_k)$6 or poor conditioning, controller performance may degrade sharply, especially at small $V(x_k)$7.
• Comparison with DDLQR: Tuning the direct data-driven LQR (DDLQR, [ESMZAD2025112197]) via $V(x_k)$8 was found to be less interpretable and required extensive search for satisfactory behavior, compared to the single-parameter $V(x_k)$9 of NGD.
• Real-World Deployment: Policies synthesized purely from input-state data, without model identification, exhibited reliable sim-to-real transfer under the direct NGD parameterization. The results demonstrate that robust, interpretable closed-loop behavior and explicit control over speed-effort trade-offs are achieved.

A typical real-world point-to-point trajectory and heading is shown below:

Figure 8: Real-world trajectory and heading.

Implications and Future Directions

The results demonstrate that data-driven NGD control achieves stable, robust, and highly interpretable policies for LTI systems without explicit model identification. The direct parameterization yields superior practical performance, control authority, and tuning simplicity compared to cost-shaped LQR and other data-driven variants. Explicit higher-order noise propagation in the covariance recursions accounts for closed-loop uncertainty and improves robustness in low-data and high-variance regimes.

Theoretically, this work demonstrates that geometric optimization principles (e.g., NGD) can be merged with direct data-driven feedback design in control, providing precise guarantees and interpretable behavior with minimal tuning. Practically, it supports the adoption of NGD principles in robotics and broader autonomous systems, given their adaptivity, robustness, and ease of deployment.

Open directions include generalization to nonlinear and time-varying systems (via kernel embeddings or local linearization), explicit constraint handling (e.g., integration with MPC frameworks), and scalability improvements for high-dimensional or resource-constrained real-time deployments. Investigating online/streaming updates and nonstationary environment adaptation will further align this approach with the requirements of next-generation adaptive AI systems.

Conclusion

Direct data-driven NGD control establishes a powerful framework for robust, interpretable, and tractable synthesis of feedback policies from input-state data. By leveraging the geometry of the FIM, it provides a principled alternative to classical cost-driven methods, closing the gap between model-based geometric optimization and practical data-driven control. The technical guarantees, empirical findings, and clear interpretability of trade-offs make this technique highly relevant for both theoretical control research and deployment in AI-driven, safety-critical robotic systems.