---
title: Multitask Linear Quadratic Regulation
url: https://www.emergentmind.com/topics/multitask-linear-quadratic-regulation-lqr
type: topic
---

# Multitask Linear Quadratic Regulation

Searching arXiv for the cited multitask LQR papers to ground the article in current literature.
Search query: 2408.04488 multitask LQR Pareto linear scalarization
Multitask Linear Quadratic Regulation (LQR) denotes a family of control and learning problems in which quadratic regulation is posed across multiple objectives, multiple systems, or multiple levels of organization rather than for a single linear system with a single quadratic cost. In recent arXiv work, the term covers at least four distinct formulations: multi-objective LQR on one plant with Pareto tradeoffs [2408.04488], heterogeneous multitask LQR in which a single controller is evaluated across many task instances [2509.19266], meta-learning and representation-learning schemes that exploit shared structure across LQR tasks [2401.14534, 2407.05781], and hierarchical multi-agent decompositions with separated local and global quadratic objectives [2007.14186]. This suggests that multitask LQR is best understood as an umbrella term whose exact mathematical meaning depends on where the multiplicity enters: objectives, plants, controllers, or organizational layers.

## 1. Canonical formulations

A standard single-task LQR problem specifies linear dynamics, quadratic state and control penalties, and a stabilizing state-feedback law obtained from a Riccati equation. Multitask variants retain the linear-quadratic core but alter the optimization target. In one line of work, a single system is paired with multiple quadratic objectives \((Q_i,R_i)\), and the problem is to characterize the Pareto front over stabilizing gains [2408.04488]. In another, there are multiple tasks \(\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})\), and the goal is to find one controller that is stabilizing and has satisfactory performance on every task, typically by minimizing an average cost over the intersection of taskwise stabilizing sets [2509.19266]. A third formulation assumes that tasks are related through either a good initialization for fast adaptation or a shared low-dimensional dynamics representation [2401.14534, 2407.05781]. A fourth decomposes a large control objective into multiple quadratic tasks, such as local group-wise regulation and centroid-level coordination in heterogeneous multi-agent systems [2007.14186].

| Formulation | Core object | Representative paper |
|---|---|---|
| Multi-objective LQR | Pareto front over \((Q_i,R_i)\) on one plant | [2408.04488] |
| Heterogeneous multitask LQR | One common controller across tasks | [2509.19266] |
| Meta / representation learning for LQR | Shared initialization or shared basis | [2401.14534], [2407.05781] |
| Hierarchical multi-agent LQR | Separated local and global quadratic tasks | [2007.14186] |

These formulations are not interchangeable. Multi-objective LQR studies tradeoffs among several costs for one system, whereas heterogeneous multitask LQR studies transfer or compromise across several systems. Hierarchical multi-agent variants, by contrast, split a large objective into structured subobjectives. The common element is that the Riccati-based structure of LQR is preserved while the optimization target is broadened beyond a single plant–single cost pair.

## 2. Multi-objective LQR and the Pareto front

In "Multi-Objective LQR with Linear Scalarization" [2408.04488], the system is the infinite-horizon discrete-time LQR
\[
x_{t+1} = Ax_t + Bu_t,\qquad u_t = Kx_t,
\]
with \(x_0\sim \mathcal N(0,I_n)\). For objective \(i\), the long-run average cost is \(L_i(K)=L(K,Q_i,R_i)\), and the feasible set is the set of stabilizing gains
\[
S=\{K:\lambda_j(A+BK)<1\ \forall j\}.
\]
The multi-objective problem is to minimize \(\{L_i(K)\}_{i\in[m]}\) subject to \(K\in S\), under the assumptions that \((A,B)\) is stabilizable and each \((Q_i,R_i)\) is positive definite [2408.04488].

The central theorem is that Pareto optimality is exactly characterized by linear scalarization. For any weight vector \(w\in\Delta([m])\),
\[
L_w(K)=\sum_i w_iL_i(K)=L(K,Q_w,R_w),
\]
where \(Q_w=\sum_i w_iQ_i\) and \(R_w=\sum_i w_iR_i\). Hence each weighted multi-objective problem is exactly a standard single-objective LQR, with optimizer
\[
K_w = -(R_w+B^\top P_w B)^{-1}B^\top P_w A,\qquad P_w=\mathrm{dare}(A,B,Q_w,R_w).
\]
The paper proves
\[
PF(S)=CCS(S),
\]
equivalently,
\[
K\in PF(S)\iff \exists w\in\Delta([m])\text{ such that }K\in\arg\min_{K'\in S}L_w(K').
\]
This is notable because the stable-gain set \(S\) is not convex, yet linear scalarization still recovers all Pareto points [2408.04488].

The proof proceeds through a lifting argument. The original gain-space problem is mapped to an equivalent convex semidefinite program in variables \((L,P,G)\), with a surjective map
\[
P=dlyape(A+BK,I),\qquad L=KP,\qquad G=P-I.
\]
Classical Pareto theory then applies in the lifted convex space, and the result transfers back to the original nonconvex stabilizing-controller space. The same paper also establishes a smoothness result: an \(\epsilon\)-perturbation of the scalarization parameter yields an \(O(\epsilon)\) perturbation in objective space. This leads to a grid-search approximation scheme over the simplex. If \(N_\epsilon\) is an \(\epsilon\)-net of \(\Delta([m])\), then
\[
PF_\epsilon(S)=\{K_{w_\epsilon}:w_\epsilon\in N_\epsilon\},
\]
and the number of Riccati solves is \(O(\epsilon^{-m})\) [2408.04488].

The analysis further extends to certainty equivalence. If the true dynamics \((A,B)\) are unknown and replaced by estimates \((\hat A,\hat B)\) with sufficiently small error, then the same scalarization-grid procedure still stabilizes the true system and approximates the true Pareto front with the same type of \(O(\epsilon)\) bound [2408.04488]. In this formulation, multitask LQR is a Pareto-analysis problem rather than a transfer-learning problem.

## 3. One common controller across heterogeneous tasks

A distinct formulation appears in "Policy Gradient Bounds in Multitask LQR" [2509.19266]. There are \(N\) tasks with dynamics
\[
x_{t+1}^{(i)} = A^{(i)}x_t^{(i)} + B^{(i)}u_t^{(i)},
\]
and quadratic infinite-horizon cost under \(u_t^{(i)}=-Kx_t^{(i)}\),
\[
J^{(i)}(K) = \mathbb{E}\!\left[\sum_{t=0}^{\infty} x_t^{(i)\top}\big(Q^{(i)} + K^\top R^{(i)}K\big)x_t^{(i)}\right].
\]
The feasible set is the intersection of taskwise stabilizing sets,
\[
\mathcal K_{\text{stab}}=\bigcap_{i=1}^N \mathcal K_{\text{stab}}^{(i)},
\]
and the multitask objective is the average cost
\[
J_{\text{avg}}(K)=\frac1N\sum_{i=1}^N J^{(i)}(K).
\]
The multitask optimum \(K_\star\) minimizes this average over \(\mathcal K_{\text{stab}}\) [2509.19266].

The algorithm analyzed is plain policy gradient,
\[
K_{n+1}=K_n-\alpha \nabla J_{\text{avg}}(K_n),
\]
started from a common stabilizing controller. The paper emphasizes that previous multitask LQR analyses measured heterogeneity by open-loop parameter deviations such as \(\|A^{(i)}-A^{(j)}\|\) and \(\|Q^{(i)}-Q^{(j)}\|\), and argues that these can be very conservative because they ignore closed-loop similarity under a shared controller. The replacement is a closed-loop gradient-discrepancy quantity,
\[
g_{ij}(K)=\|\nabla J^{(i)}(K)-\nabla J^{(j)}(K)\|_F,
\]
bounded via a bisimulation-inspired construction on coupled covariance dynamics. This yields heterogeneity measures \(b_{ij}(K)\) and \(b_i(K)\) that enter task-specific suboptimality bounds for both the multitask optimum and the asymptotic policy-gradient iterate [2509.19266].

The resulting guarantees are explicitly taskwise. For each task \(i\), the paper bounds the multitask-optimality gap \(J^{(i)}(K_\star)-J^{(i)}(K_\star^{(i)})\) and the asymptotic policy-gradient gap \(\limsup_{n\to\infty}(J^{(i)}(K_n)-J^{(i)}(K_\star^{(i)}))\) in terms of \(b_i(K)\). It also provides conditions under which all policy-gradient iterates remain stabilizing for every system. In experiments on inverted-pendulum and unicycle task sets, the bisimulation-based measure improves upon baseline heterogeneity measures dramatically, with about **99.9998%** average reduction on random inverted-pendulum task sets and about **99.9996%** reduction on random two-unicycle-task collections [2509.19266].

This formulation treats multitask LQR as a compromise problem: one controller is trained against many taskwise costs and dynamics, and the irreducible quantity is the heterogeneity bias induced by forcing a common policy across nonidentical tasks.

## 4. Meta-learning and shared representations

"Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR" [2401.14534] studies a meta-learning problem over \(M\) discrete-time tasks
\[
x_{t+1}^{(i)}=A^{(i)}x_t^{(i)}+B^{(i)}u_t^{(i)},
\]
where each task is \(\mathcal T^{(i)}=(A^{(i)},B^{(i)},Q^{(i)},R^{(i)})\). The meta-objective is not to find one controller that is final for all tasks, but to learn an initialization \(K^\star_{\mathrm{ML}}\) that performs well after one inner adaptation step:
\[
J_{\mathrm{ML}}(K)=\frac1M\sum_{i=1}^M J^{(i)}\!\left(K-\eta_l\nabla J^{(i)}(K)\right).
\]
The paper distinguishes system heterogeneity, cost heterogeneity, and joint system-and-cost heterogeneity, quantified by bounds on the pairwise deviations of \(A^{(i)},B^{(i)},Q^{(i)},R^{(i)}\) [2401.14534].

The MAML-style update has the standard inner-loop and outer-loop structure. In the model-based setting, gradients and Hessians are computed from closed-form LQR expressions; in the model-free setting, both are estimated by a zeroth-order two-point estimator. The main guarantees are that the algorithm produces a stabilizing controller close to each task-specific optimal controller up to a task-heterogeneity bias, in both model-based and model-free learning scenarios, and that in the model-based setting this controller is achieved with a linear convergence rate, improving upon sub-linear rates from existing work [2401.14534]. Here multitask LQR is explicitly tied to fast adaptation on unseen tasks rather than to a single compromise controller.

A different shared-structure formulation appears in "Regret Analysis of Multi-task Representation Learning for Linear-Quadratic Adaptive Control" [2407.05781]. There are \(H\) related systems,
\[
x_{t+1}^{(h)} = A_\star^{(h)} x_t^{(h)} + B_\star^{(h)} u_t^{(h)} + w_t^{(h)},
\]
and the key assumption is a common low-dimensional dynamics basis,
\[
\begin{bmatrix} A_\star^{(h)} & B_\star^{(h)} \end{bmatrix}
=
\mathrm{VEC}^{-1}\!\left(\Phi_\star \theta_\star^{(h)}\right),
\]
with shared column-orthonormal \(\Phi_\star\) and task-specific coefficients \(\theta_\star^{(h)}\). The algorithm is a doubling-epoch certainty-equivalent scheme with two coupled updates: task-specific least squares conditioned on the current representation, and a shared representation update by de-biased feature whitening (DFW) [2407.05781].

The theory identifies two exploration regimes. In benign exploration settings, the regret of any agent after \(T\) timesteps scales as \(\tilde O(\sqrt{T/H})\). In difficult exploration settings, the regret scales as
\[
\tilde O(\sqrt{d_u d_\theta}\sqrt{T} + T^{3/4}/H^{1/5}),
\]
and the paper compares both regimes to the minimax single-task regret
\[
O(\sqrt{d_x d_u^2}\sqrt{T}).
\]
The interpretation given is that many agents help because DFW improves the shared basis with more tasks, and once the basis is accurate each task only needs to estimate a lower-dimensional parameter vector \(\theta_\star^{(h)}\) [2407.05781]. In this sense, multitask LQR is a representation-learning problem embedded inside adaptive control.

## 5. Hierarchical and cooperative multi-agent formulations

In "Hierarchical Control of Multi-Agent Systems using Online Reinforcement Learning" [2007.14186], the multitask structure is explicit and architectural. Heterogeneous agents are partitioned into non-overlapping groups, and the control objective is split into two distinct quadratic tasks. The first minimizes a group-wise block-decentralized LQR function representing the local mission inside each group. The second minimizes an LQR function between the average states, or centroids, of the groups:
\[
\mathbf{x}_{av,j} = \frac{1}{p_j}\sum_{i=1}^{p_j} x_i.
\]
The state-weighting matrix is decomposed as
\[
Q=\bar Q + L_w \odot \tilde Q,
\]
where \(\bar Q\) is block diagonal and the second term captures centroid-level coupling [2007.14186].

The technical device is a redefinition of the weighting matrices that decouples the corresponding algebraic Riccati equations. This reduces the overall design to \(N\) independent local Riccati equations plus an algebraically recovered global term. The local controller is block-decentralized and can therefore be learned in parallel by off-policy ADP, while the global controller is reduced-dimensional because it depends only on average states. The resulting controller is exact for a modified hierarchical objective but suboptimal with respect to the original centralized cost [2007.14186]. In this formulation, multitask LQR is literally multiobjective and hierarchical.

A related, though not identical, line is the cooperative LQR design for multi-input systems in "Linear Quadratic Regulator Design for Multi-input Systems with A Distributed Cooperative Strategy" [2107.12596]. There, each input channel is generated by an agent in a network, input matrices are local information, and the plant is controlled through a distributed information-fusion recursion. The assumptions require only joint controllability of the aggregated pair \((A_k,B_k)\), not controllability of any single agent or neighborhood collection. The proposed controller requires only one-time information exchange at every control step, guarantees bounded controller gains for time-varying systems and convergence for time-invariant systems, and provides a global LQR-type performance bound [2107.12596]. This is not a multitask formulation in the same sense as separated quadratic objectives, but it shows how LQR structure can be distributed across many decision-makers without reverting to a centralized Riccati computation.

## 6. Extensions and conceptual boundaries

The multitask viewpoint extends beyond fully observed deterministic LQR. "Multitask LQG Control: Performance and Generalization Bounds" [2604.16730] considers partially observed stochastic systems
\[
x_{t+1}^{(i)} = A^{(i)}x_t^{(i)} + B^{(i)}u_t^{(i)} + w_t^{(i)},\qquad
y_t^{(i)} = C^{(i)}x_t^{(i)} + v_t^{(i)},
\]
and applies a history-dependent lifting based on stacked input-output histories \(z_{t,p}^{(i)}\). The filtered estimate satisfies
\[
\widehat x_t^{(i)} = S_\star^{(i)} z_{t,p}^{(i)},
\]
so a lifted policy \(u_t^{(i)}=\widetilde K z_{t,p}^{(i)}\) recasts multitask LQG as an equivalent high-dimensional multitask LQR problem. The paper then derives bisimulation-based heterogeneity measures, taskwise performance bounds, generalization guarantees with \(O(N^{-1/2})\) scaling, and model-free policy-gradient variance reduction proportional to the number of training tasks [2604.16730]. The role of LQR here is methodological: it is the lifted fully observed surrogate through which multitask LQG becomes analyzable.

Recent literature also makes clear that not every nearby problem is a formal multitask LQR problem. "A model free approach for continuous-time optimal tracking control with unknown user-define cost and constrained control input via advantage function" [2509.16821] studies continuous-time LQR and LQT with constrained inputs, multiple algorithms, and multiple time intervals, but it does **not** jointly learn multiple tasks, define multiple independent objectives sharing a common policy, or formulate a multi-objective LQR. Similarly, "Biomolecular LQR under Partial Observation" [2511.02418] uses reduced-order observers and biochemical implementations of LQR-like feedback, and argues that the matrix \(Q\) can encode multiple biological pressures implicitly, but it does **not** develop a formal multitask LQR framework with multiple simultaneous objectives or Pareto tradeoffs. These cases delineate the boundary of the term: multiple intervals, multiple motifs, or multiple implicit pressures are not sufficient by themselves to constitute multitask LQR.

Across these formulations, a recurring theme is the tradeoff between shared structure and heterogeneity. In Pareto LQR, the issue is whether all tradeoff points can be recovered by scalarization [2408.04488]. In shared-controller and multitask policy-gradient settings, the issue is the bias induced by forcing one controller across dissimilar tasks [2509.19266, 2604.16730]. In meta-learning and representation learning, the issue is whether shared structure can be exploited so that fast personalization or lower regret outweighs misspecification and exploration costs [2401.14534, 2407.05781]. In hierarchical and distributed settings, the issue is how to preserve Riccati structure while decomposing a large objective into smaller quadratic subproblems [2007.14186, 2107.12596]. Multitask LQR is therefore less a single theorem than a collection of structurally related programs built around the same linear-quadratic backbone.

Source: https://www.emergentmind.com/topics/multitask-linear-quadratic-regulation-lqr