Multitask Linear Quadratic Regulation

• Multitask LQR is a unified framework encompassing multiple formulations of LQR problems across various objectives, systems, or organizational layers.
• It employs methods such as Pareto tradeoffs, common controller design, meta-learning, and hierarchical decompositions to address complex control scenarios.
• The approach preserves the classical Riccati backbone while enabling advanced techniques for stability, performance tradeoffs, and distributed control.

Searching arXiv for the cited multitask LQR papers to ground the article in current literature. Search query: (Jadbabaie et al., 2024) multitask LQR Pareto linear scalarization Multitask Linear Quadratic Regulation (LQR) denotes a family of control and learning problems in which quadratic regulation is posed across multiple objectives, multiple systems, or multiple levels of organization rather than for a single linear system with a single quadratic cost. In recent arXiv work, the term covers at least four distinct formulations: multi-objective LQR on one plant with Pareto tradeoffs (Jadbabaie et al., 2024), heterogeneous multitask LQR in which a single controller is evaluated across many task instances (Stamouli et al., 23 Sep 2025), meta-learning and representation-learning schemes that exploit shared structure across LQR tasks (Toso et al., 2024, Lee et al., 2024), and hierarchical multi-agent decompositions with separated local and global quadratic objectives (Bai et al., 2020). This suggests that multitask LQR is best understood as an umbrella term whose exact mathematical meaning depends on where the multiplicity enters: objectives, plants, controllers, or organizational layers.

1. Canonical formulations

A standard single-task LQR problem specifies linear dynamics, quadratic state and control penalties, and a stabilizing state-feedback law obtained from a Riccati equation. Multitask variants retain the linear-quadratic core but alter the optimization target. In one line of work, a single system is paired with multiple quadratic objectives $(Q_i,R_i)$, and the problem is to characterize the Pareto front over stabilizing gains (Jadbabaie et al., 2024). In another, there are multiple tasks $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$, and the goal is to find one controller that is stabilizing and has satisfactory performance on every task, typically by minimizing an average cost over the intersection of taskwise stabilizing sets (Stamouli et al., 23 Sep 2025). A third formulation assumes that tasks are related through either a good initialization for fast adaptation or a shared low-dimensional dynamics representation (Toso et al., 2024, Lee et al., 2024). A fourth decomposes a large control objective into multiple quadratic tasks, such as local group-wise regulation and centroid-level coordination in heterogeneous multi-agent systems (Bai et al., 2020).

Formulation	Core object	Representative paper
Multi-objective LQR	Pareto front over $(Q_i,R_i)$ on one plant	(Jadbabaie et al., 2024)
Heterogeneous multitask LQR	One common controller across tasks	(Stamouli et al., 23 Sep 2025)
Meta / representation learning for LQR	Shared initialization or shared basis	(Toso et al., 2024, Lee et al., 2024)
Hierarchical multi-agent LQR	Separated local and global quadratic tasks	(Bai et al., 2020)

These formulations are not interchangeable. Multi-objective LQR studies tradeoffs among several costs for one system, whereas heterogeneous multitask LQR studies transfer or compromise across several systems. Hierarchical multi-agent variants, by contrast, split a large objective into structured subobjectives. The common element is that the Riccati-based structure of LQR is preserved while the optimization target is broadened beyond a single plant–single cost pair.

2. Multi-objective LQR and the Pareto front

In "Multi-Objective LQR with Linear Scalarization" (Jadbabaie et al., 2024), the system is the infinite-horizon discrete-time LQR

$x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$

with $x_0\sim \mathcal N(0,I_n)$. For objective $i$, the long-run average cost is $L_i(K)=L(K,Q_i,R_i)$, and the feasible set is the set of stabilizing gains

$S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$

The multi-objective problem is to minimize $\{L_i(K)\}_{i\in[m]}$ subject to $K\in S$, under the assumptions that $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$0 is stabilizable and each $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$1 is positive definite (Jadbabaie et al., 2024).

The central theorem is that Pareto optimality is exactly characterized by linear scalarization. For any weight vector $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$2,

$\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$3

where $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$4 and $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$5. Hence each weighted multi-objective problem is exactly a standard single-objective LQR, with optimizer

$\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$6

The paper proves

$\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$7

equivalently,

$\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$8

This is notable because the stable-gain set $\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})$9 is not convex, yet linear scalarization still recovers all Pareto points (Jadbabaie et al., 2024).

The proof proceeds through a lifting argument. The original gain-space problem is mapped to an equivalent convex semidefinite program in variables $(Q_i,R_i)$0, with a surjective map

$(Q_i,R_i)$1

Classical Pareto theory then applies in the lifted convex space, and the result transfers back to the original nonconvex stabilizing-controller space. The same paper also establishes a smoothness result: an $(Q_i,R_i)$2-perturbation of the scalarization parameter yields an $(Q_i,R_i)$3 perturbation in objective space. This leads to a grid-search approximation scheme over the simplex. If $(Q_i,R_i)$4 is an $(Q_i,R_i)$5-net of $(Q_i,R_i)$6, then

$(Q_i,R_i)$7

and the number of Riccati solves is $(Q_i,R_i)$8 (Jadbabaie et al., 2024).

The analysis further extends to certainty equivalence. If the true dynamics $(Q_i,R_i)$9 are unknown and replaced by estimates $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$0 with sufficiently small error, then the same scalarization-grid procedure still stabilizes the true system and approximates the true Pareto front with the same type of $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$1 bound (Jadbabaie et al., 2024). In this formulation, multitask LQR is a Pareto-analysis problem rather than a transfer-learning problem.

3. One common controller across heterogeneous tasks

A distinct formulation appears in "Policy Gradient Bounds in Multitask LQR" (Stamouli et al., 23 Sep 2025). There are $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$2 tasks with dynamics

$x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$3

and quadratic infinite-horizon cost under $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$4,

$x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$5

The feasible set is the intersection of taskwise stabilizing sets,

$x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$6

and the multitask objective is the average cost

$x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$7

The multitask optimum $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$8 minimizes this average over $x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,$9 (Stamouli et al., 23 Sep 2025).

The algorithm analyzed is plain policy gradient,

$x_0\sim \mathcal N(0,I_n)$0

started from a common stabilizing controller. The paper emphasizes that previous multitask LQR analyses measured heterogeneity by open-loop parameter deviations such as $x_0\sim \mathcal N(0,I_n)$1 and $x_0\sim \mathcal N(0,I_n)$2, and argues that these can be very conservative because they ignore closed-loop similarity under a shared controller. The replacement is a closed-loop gradient-discrepancy quantity,

$x_0\sim \mathcal N(0,I_n)$3

bounded via a bisimulation-inspired construction on coupled covariance dynamics. This yields heterogeneity measures $x_0\sim \mathcal N(0,I_n)$4 and $x_0\sim \mathcal N(0,I_n)$5 that enter task-specific suboptimality bounds for both the multitask optimum and the asymptotic policy-gradient iterate (Stamouli et al., 23 Sep 2025).

The resulting guarantees are explicitly taskwise. For each task $x_0\sim \mathcal N(0,I_n)$6, the paper bounds the multitask-optimality gap $x_0\sim \mathcal N(0,I_n)$7 and the asymptotic policy-gradient gap $x_0\sim \mathcal N(0,I_n)$8 in terms of $x_0\sim \mathcal N(0,I_n)$9. It also provides conditions under which all policy-gradient iterates remain stabilizing for every system. In experiments on inverted-pendulum and unicycle task sets, the bisimulation-based measure improves upon baseline heterogeneity measures dramatically, with about 99.9998% average reduction on random inverted-pendulum task sets and about 99.9996% reduction on random two-unicycle-task collections (Stamouli et al., 23 Sep 2025).

This formulation treats multitask LQR as a compromise problem: one controller is trained against many taskwise costs and dynamics, and the irreducible quantity is the heterogeneity bias induced by forcing a common policy across nonidentical tasks.

4. Meta-learning and shared representations

"Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR" (Toso et al., 2024) studies a meta-learning problem over $i$0 discrete-time tasks

$i$1

where each task is $i$2. The meta-objective is not to find one controller that is final for all tasks, but to learn an initialization $i$3 that performs well after one inner adaptation step: $i$4 The paper distinguishes system heterogeneity, cost heterogeneity, and joint system-and-cost heterogeneity, quantified by bounds on the pairwise deviations of $i$5 (Toso et al., 2024).

The MAML-style update has the standard inner-loop and outer-loop structure. In the model-based setting, gradients and Hessians are computed from closed-form LQR expressions; in the model-free setting, both are estimated by a zeroth-order two-point estimator. The main guarantees are that the algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias, in both model-based and model-free learning scenarios, and that in the model-based setting this controller is achieved with a linear convergence rate, improving upon sub-linear rates from existing work (Toso et al., 2024). Here multitask LQR is explicitly tied to fast adaptation on unseen tasks rather than to a single compromise controller.

A different shared-structure formulation appears in "Regret Analysis of Multi-task Representation Learning for Linear-Quadratic Adaptive Control" (Lee et al., 2024). There are $i$6 related systems,

$i$7

and the key assumption is a common low-dimensional dynamics basis,

$i$8

with shared column-orthonormal $i$9 and task-specific coefficients $L_i(K)=L(K,Q_i,R_i)$0. The algorithm is a doubling-epoch certainty-equivalent scheme with two coupled updates: task-specific least squares conditioned on the current representation, and a shared representation update by de-biased feature whitening (DFW) (Lee et al., 2024).

The theory identifies two exploration regimes. In benign exploration settings, the regret of any agent after $L_i(K)=L(K,Q_i,R_i)$1 timesteps scales as $L_i(K)=L(K,Q_i,R_i)$2. In difficult exploration settings, the regret scales as

$L_i(K)=L(K,Q_i,R_i)$3

and the paper compares both regimes to the minimax single-task regret

$L_i(K)=L(K,Q_i,R_i)$4

The interpretation given is that many agents help because DFW improves the shared basis with more tasks, and once the basis is accurate each task only needs to estimate a lower-dimensional parameter vector $L_i(K)=L(K,Q_i,R_i)$5 (Lee et al., 2024). In this sense, multitask LQR is a representation-learning problem embedded inside adaptive control.

5. Hierarchical and cooperative multi-agent formulations

In "Hierarchical Control of Multi-Agent Systems using Online Reinforcement Learning" (Bai et al., 2020), the multitask structure is explicit and architectural. Heterogeneous agents are partitioned into non-overlapping groups, and the control objective is split into two distinct quadratic tasks. The first minimizes a group-wise block-decentralized LQR function representing the local mission inside each group. The second minimizes an LQR function between the average states, or centroids, of the groups: $L_i(K)=L(K,Q_i,R_i)$6 The state-weighting matrix is decomposed as

$L_i(K)=L(K,Q_i,R_i)$7

where $L_i(K)=L(K,Q_i,R_i)$8 is block diagonal and the second term captures centroid-level coupling (Bai et al., 2020).

The technical device is a redefinition of the weighting matrices that decouples the corresponding algebraic Riccati equations. This reduces the overall design to $L_i(K)=L(K,Q_i,R_i)$9 independent local Riccati equations plus an algebraically recovered global term. The local controller is block-decentralized and can therefore be learned in parallel by off-policy ADP, while the global controller is reduced-dimensional because it depends only on average states. The resulting controller is exact for a modified hierarchical objective but suboptimal with respect to the original centralized cost (Bai et al., 2020). In this formulation, multitask LQR is literally multiobjective and hierarchical.

A related, though not identical, line is the cooperative LQR design for multi-input systems in "Linear Quadratic Regulator Design for Multi-input Systems with A Distributed Cooperative Strategy" (Duan et al., 2021). There, each input channel is generated by an agent in a network, input matrices are local information, and the plant is controlled through a distributed information-fusion recursion. The assumptions require only joint controllability of the aggregated pair $S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$0, not controllability of any single agent or neighborhood collection. The proposed controller requires only one-time information exchange at every control step, guarantees bounded controller gains for time-varying systems and convergence for time-invariant systems, and provides a global LQR-type performance bound (Duan et al., 2021). This is not a multitask formulation in the same sense as separated quadratic objectives, but it shows how LQR structure can be distributed across many decision-makers without reverting to a centralized Riccati computation.

6. Extensions and conceptual boundaries

The multitask viewpoint extends beyond fully observed deterministic LQR. "Multitask LQG Control: Performance and Generalization Bounds" (Toso et al., 17 Apr 2026) considers partially observed stochastic systems

$S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$1

and applies a history-dependent lifting based on stacked input-output histories $S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$2. The filtered estimate satisfies

$S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$3

so a lifted policy $S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$4 recasts multitask LQG as an equivalent high-dimensional multitask LQR problem. The paper then derives bisimulation-based heterogeneity measures, taskwise performance bounds, generalization guarantees with $S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$5 scaling, and model-free policy-gradient variance reduction proportional to the number of training tasks (Toso et al., 17 Apr 2026). The role of LQR here is methodological: it is the lifted fully observed surrogate through which multitask LQG becomes analyzable.

Recent literature also makes clear that not every nearby problem is a formal multitask LQR problem. "A model free approach for continuous-time optimal tracking control with unknown user-define cost and constrained control input via advantage function" (Nguyen et al., 20 Sep 2025) studies continuous-time LQR and LQT with constrained inputs, multiple algorithms, and multiple time intervals, but it does not jointly learn multiple tasks, define multiple independent objectives sharing a common policy, or formulate a multi-objective LQR. Similarly, "Biomolecular LQR under Partial Observation" (Zhang et al., 4 Nov 2025) uses reduced-order observers and biochemical implementations of LQR-like feedback, and argues that the matrix $S=\{K:\lambda_j(A+BK)<1\ \forall j\}.$6 can encode multiple biological pressures implicitly, but it does not develop a formal multitask LQR framework with multiple simultaneous objectives or Pareto tradeoffs. These cases delineate the boundary of the term: multiple intervals, multiple motifs, or multiple implicit pressures are not sufficient by themselves to constitute multitask LQR.

Across these formulations, a recurring theme is the tradeoff between shared structure and heterogeneity. In Pareto LQR, the issue is whether all tradeoff points can be recovered by scalarization (Jadbabaie et al., 2024). In shared-controller and multitask policy-gradient settings, the issue is the bias induced by forcing one controller across dissimilar tasks (Stamouli et al., 23 Sep 2025, Toso et al., 17 Apr 2026). In meta-learning and representation learning, the issue is whether shared structure can be exploited so that fast personalization or lower regret outweighs misspecification and exploration costs (Toso et al., 2024, Lee et al., 2024). In hierarchical and distributed settings, the issue is how to preserve Riccati structure while decomposing a large objective into smaller quadratic subproblems (Bai et al., 2020, Duan et al., 2021). Multitask LQR is therefore less a single theorem than a collection of structurally related programs built around the same linear-quadratic backbone.