Distributed Linear Quadratic Regulator

• Distributed LQR is a control strategy that decentralizes the centralized LQR approach by leveraging local and neighboring state information for optimal feedback.
• The κ-distributed architecture applies a tunable locality constraint, yielding an exponential decay in the optimality gap as the communication radius increases.
• Practical implementations balance communication overhead and performance by selecting a truncation radius that scales logarithmically with the desired suboptimality.

A distributed Linear Quadratic Regulator (LQR) is a control strategy for large-scale, networked systems in which the computation and execution of optimal control laws are performed in a decentralized or distributed manner across multiple agents, each with access only to limited local or neighboring information. Distributed LQR theory addresses the synthesis, analysis, and performance limits of such controllers, particularly in the context of communication-constrained, sparsely coupled, or privacy-sensitive agent networks.

1. Problem Formulation and Standard Centralized LQR

Consider a networked, discrete-time linear system with $N$ agents, described by a global state $x(t)\in\mathbb{R}^{n_x}$ and input $u(t)\in\mathbb{R}^{n_u}$ evolving as

$x(t+1) = A x(t) + B u(t),$

where $A$ and $B$ have a block structure consistent with the edge set of an undirected graph $G=(V,E)$. Each node $i\in V$ has access to a local state $x_i$ and applies control input $u_i$. The canonical infinite-horizon quadratic cost is

$x(t)\in\mathbb{R}^{n_x}$0

where $x(t)\in\mathbb{R}^{n_x}$1, $x(t)\in\mathbb{R}^{n_x}$2 encode performance and effort penalties. Under stabilizability and detectability, the centralized LQR synthesizes a unique optimal state-feedback $x(t)\in\mathbb{R}^{n_x}$3, with $x(t)\in\mathbb{R}^{n_x}$4 derived from the discrete algebraic Riccati equation: $x(t)\in\mathbb{R}^{n_x}$5

$x(t)\in\mathbb{R}^{n_x}$6

This solution assumes global state measurements and all-to-all communication, which is impractical for large-scale, distributed systems.

2. κ-Distributed LQR Architecture and Exponential Performance Decay

The $x(t)\in\mathbb{R}^{n_x}$7-distributed LQR architecture introduces a tunable locality constraint by truncating the global gain $x(t)\in\mathbb{R}^{n_x}$8 to a radius-$x(t)\in\mathbb{R}^{n_x}$9 neighborhood on the graph: $u(t)\in\mathbb{R}^{n_u}$0 where $u(t)\in\mathbb{R}^{n_u}$1, with $u(t)\in\mathbb{R}^{n_u}$2 the shortest-path distance on $u(t)\in\mathbb{R}^{n_u}$3. Equivalently, $u(t)\in\mathbb{R}^{n_u}$4 if $u(t)\in\mathbb{R}^{n_u}$5, zero otherwise.

The optimality gap of the $u(t)\in\mathbb{R}^{n_u}$6-distributed LQR relative to the centralized solution is quantified by

$u(t)\in\mathbb{R}^{n_u}$7

where constants $u(t)\in\mathbb{R}^{n_u}$8 and $u(t)\in\mathbb{R}^{n_u}$9 depend only on control-theoretic and graph growth parameters. Specifically, $x(t+1) = A x(t) + B u(t),$0 decays exponentially in the locality radius $x(t+1) = A x(t) + B u(t),$1, provided the number of nodes at distance $x(t+1) = A x(t) + B u(t),$2 from any node grows subexponentially. This ensures near-centralized performance can be attained with modest communication radii, a key result for scalable distributed control (Shin et al., 2022).

3. Assumptions and Performance Gap Analysis

The analytic guarantees for distributed LQR structure are predicated on:

• Uniform boundedness and well-conditioning of system and cost matrices;
• Stabilizability: existence of $x(t+1) = A x(t) + B u(t),$3 such that $x(t+1) = A x(t) + B u(t),$4 for $x(t+1) = A x(t) + B u(t),$5;
• Detectability (dual conditions on $x(t+1) = A x(t) + B u(t),$6);
• Subexponential growth condition for the communication/interaction graph: the number of nodes at distance $x(t+1) = A x(t) + B u(t),$7 increases no faster than a subexponential function $x(t+1) = A x(t) + B u(t),$8.

Under these, the constants controlling the exponential decay—specifically the decay rate $x(t+1) = A x(t) + B u(t),$9—are determined by the worst-case stability margins and the graph growth function. As stabilizability or detectability degrades ($A$0 or $A$1), convergence to centralized performance is slower. For typical low-dimensional meshes (e.g., grids), the exponential decay in $A$2 dominates any polynomial growth of $A$3.

4. Algorithmic Implementation and Communication Structure

The distributed LQR controller with truncation radius $A$4 is implemented as follows:

1. Centralized DARE solution: Compute $A$5 and $A$6 offline (system-wide or via distributed policy-iteration).
2. Local exchange: Each agent transmits its local state $A$7 to all agents within $A$8 hops (performed via $A$9 synchronous communication rounds, e.g., gossip protocols).
3. Local control: Agent $B$0 collects neighbor states within $B$1 and applies $B$2.
4. Synchronization burden: Each time step requires $B$3 communication rounds and matrix-vector multiplication involving only the local $B$4-neighborhood.

To achieve any prescribed suboptimality $B$5, it suffices to choose

$B$6

demonstrating logarithmic scaling of communication depth with desired accuracy.

5. Trade-Offs, Locality, and Practical Implications

The $B$7-distributed LQR architecture interpolates between fully decentralized ($B$8) and fully centralized ($B$9 graph diameter) controllers. The design enables explicit trade-off characterization:

• Communication vs. performance: Increasing $G=(V,E)$0 reduces the optimality gap exponentially at a cost of increased communication and computational burden.
• Guidelines: Select acceptable suboptimality $G=(V,E)$1, then compute the minimal $G=(V,E)$2.
• Graph-theoretic constraints: For networks with subexponential graph growth and moderate degrees, the necessary constants for performance guarantees remain practical even in large-scale systems.
• Empirical insight: In high-dimensional or sparse networks, exchanging states out to a dozen hops recovers $G=(V,E)$3 of the centralized performance.

6. Extensions and Connections to Broader Distributed Control Literature

The analytic paradigm established by the $G=(V,E)$4-distributed LQR framework is a foundation for more general distributed and localized LQR synthesis—encompassing, for instance:

• Distributed Riccati recursions and consensus-based policy iteration;
• Block-decreasing structure of $G=(V,E)$5 as established by block-decay theorems;
• Outward links to system-level synthesis (SLS) approaches that parameterize constrained distributed LQR via affine and subspace constraints, enabling convex formulations of infinite-horizon, locality-constrained optimal control (Kjellqvist et al., 2022);
• Robustness extensions to dropouts, privacy constraints, information attacks, and dynamic topology modifications.

Such frameworks also motivate the development of algorithms for distributed model-free learning of LQR (e.g., using local policy-gradient, Q-learning, or RL approaches), with guarantees on stability, regret, or learning efficiency under distributed information constraints.

7. Limitations and Future Directions

While near-optimal performance is attainable with $G=(V,E)$6-locality under mild and verifiable assumptions, several challenges remain:

• Computing or approximating the global Riccati solution or $G=(V,E)$7 in a scalable, fully distributed, and communication-efficient manner for very large networks;
• Handling time-varying, directed, or weighted graphs and nonlinearly coupled agent dynamics;
• Integrating output-feedback, partial observability, or adversarial uncertainty.
• Extending to finite-horizon, receding-horizon, and event-triggered distributed LQR variants, as well as ensuring resilience to faults and communication errors.

The $G=(V,E)$8-distributed LQR framework is a central analytic milestone, establishing that moderate locality suffices for high-performance distributed stabilization and regulation in complex networked systems (Shin et al., 2022).