Optimal Functional Incentives for Control: The Linear-Quadratic Case with Bilinear Incentives

Abstract: We study the design of functional incentive mechanisms for dynamical systems, in which a leader designs a fixed incentive function to motivate a self-interested follower to actuate the system beneficially over an extended horizon, without real-time revision of the incentive. This stands in contrast to the adaptive paradigm, in which the incentive is itself a continuously updated control variable. We formalize the problem as a discrete-time bi-level optimal control problem and derive analytical results for the linear-quadratic case with bilinear incentives and a myopic follower. Specifically, we establish a necessary and sufficient stability condition for the induced closed-loop system, derive a closed-form expression for the gradient of the expected leader cost with respect to the incentive parameter matrix, and obtain a fully closed-form cost expression in the scalar setting. Based on the latter, explicit characterizations of the optimal incentive parameter are provided in two asymptotic regimes: the infinite-horizon limit and the limit of high follower cost. For long horizons, the optimal incentive is shown to become independent of the follower's private cost parameter, with direct implications for robust mechanism design under private information.

• The paper's main contribution is introducing a bi-level optimal control framework that designs fixed incentive schemes for leader-follower systems.
• It employs closed-form gradients and explicit cost derivations for both multidimensional and scalar cases, highlighting stability and performance trade-offs.
• The work demonstrates robust incentive design for decentralized control in power grids and similar systems, questioning the need for private cost information.

Optimal Functional Incentives for Control: Analytical Advances in the Linear-Quadratic-Bilinear Regime

Introduction

The study addresses incentive design in leader-follower Stackelberg settings for dynamic systems, with primary motivation from applications where the operator (leader) has limited bandwidth or commitment flexibility to update incentives in real time. It makes a rigorous distinction between adaptive incentives—where the leader continually adjusts the payment or price signal in response to observed system trajectories—and functional incentives, where the remuneration rule is fixed for an entire horizon, and the follower acts in closed-loop using this rule in real time. This control allocation is relevant for modern infrastructure—such as power grid operation—where actuation comes from self-interested agents and real-time coordination is infeasible.

Figure 1: Adaptive (top) vs. functional (bottom) incentive schemes—only the latter allows direct closure of the system-follower loop for fast-scale control.

The principal contribution is a bi-level optimal control framework where the leader parameterizes a state-and-action dependent incentive function to rationalize desirable follower behavior for an entire time horizon, as opposed to updating signals at each step. This motivates the analysis of such mechanisms under tractable conditions, specifically the linear-quadratic (LQ) regime with bilinear incentives and a myopic follower.

Figure 2: Schematic of functional-incentive-based control—highlighting the leader's design of a parameterized incentive function and the follower’s closed-loop best response.

Problem Formalization

The discrete-time game unfolds as follows:

• The system: $x_{k+1} = f(x_k, u_k)$.
• The leader: minimizes total cost over horizon $N$, incurring stage cost $\ell_L(x_k)$ and payment $p(x_k, u_k; \Theta)$, by selecting $\Theta$ to parameterize the incentive function.
• The follower: myopically minimizes net cost $\ell_F(u_k) - p(x_k, u_k; \Theta)$ at each stage, effectively acting as a real-time, rational controller.

This problem induces a nontrivial design challenge: the leader must select a fixed $\Theta$ in advance such that, when the follower best-responds for all encountered $x_k$, the resulting closed-loop system achieves acceptable performance over stochastic initializations.

A defining feature is that, by encoding feedback structure into $p(x, u; \Theta)$ rather than pursuing direct intervention, the leader outsources closed-loop actuation to the follower—an essential property for scalable, decentralized system management.

Linear-Quadratic-Bilinear Analysis

In the examined LQ-bilinear instantiation:

• System dynamics: $x_{k+1} = A x_k + B u_k$.
• Cost functions: leader $N$0, follower $N$1.
• Incentive form: $N$2 (bilinear in state deviation and control input).

The leader's optimization becomes: $N$3 subject to the system evolving with $N$4 as the follower’s best response.

Under myopic response, the follower applies a proportional control $N$5. This yields closed-loop dynamics $N$6, introducing new stability constraints: the matrix $N$7 must be Schur stable.

Key results:

• Closed-form gradient of expected leader cost with respect to $N$8 for general dimensions (enabling efficient offline optimization).
• Explicit closed-form cost in the scalar ($N$9) case, facilitating asymptotic analyses and revealing fundamental trade-offs.

Figure 3: Convergence of gradient descent for leader cost (top row) and resulting closed-loop trajectories for state, input, and incentive payment at optimal $\ell_L(x_k)$0 (bottom row).

Critically, though the total system cost collapses to a standard LQR cost when summing leader and follower, the game-induced solution is generally suboptimal due to a gap between coordinated optimization and incentive-aligned, selfish behavior—a quantifiable price of anarchy.

Scalar Regime: Design Trade-offs and Asymptotics

Specializing to scalars facilitates detailed inspection of mechanism properties and boundaries. Explicit cost expressions grant visibility into how horizon $\ell_L(x_k)$1, system dynamics $\ell_L(x_k)$2, tracking penalty $\ell_L(x_k)$3, and follower effort penalty $\ell_L(x_k)$4 shape $\ell_L(x_k)$5.

Infinite-horizon regime $\ell_L(x_k)$6:

• If $\ell_L(x_k)$7 (reference not at open-loop equilibrium), the cost is dominated by steady-state error. The optimal $\ell_L(x_k)$8 becomes

$\ell_L(x_k)$9

Notably, optimal $p(x_k, u_k; \Theta)$0 is independent of the follower's $p(x_k, u_k; \Theta)$1 (cost of effort), except where $p(x_k, u_k; \Theta)$2 influences stability. This robustifies mechanism design to information asymmetry about follower’s private costs.

• If $p(x_k, u_k; \Theta)$3, no steady-state error arises, and $p(x_k, u_k; \Theta)$4 depends on $p(x_k, u_k; \Theta)$5 via a root of a quadratic, again with stability constraints.

High follower cost regime $p(x_k, u_k; \Theta)$6:

• The limiting optimal $p(x_k, u_k; \Theta)$7 is finite and computable in terms of geometric sums of trajectory moments. Thus, incentivizing extremely unwilling (high-$p(x_k, u_k; \Theta)$8) followers is generally not cost-effective above a calculable threshold.

Figure 4: Scalar-case analyses illustrate closed-loop trajectories for optimal $p(x_k, u_k; \Theta)$9 across initializations (a), leader cost as a function of $\Theta$0, and asymptotic parameter behavior as horizon $\Theta$1 and follower weight $\Theta$2 are varied (b-e).

These scalar insights provide theoretical guardrails for static incentive design and highlight cases where further flexibility (e.g., integral or nonlinear incentives) would be required to eliminate steady-state bias.

Practical and Theoretical Implications

The rigorous derivation that, in the infinite-horizon, non-equilibrium regime, optimal functional incentives are independent of the follower’s cost parameter (as long as stability is retained), calls into question the role of information gathering on private costs—a fundamentally strong claim for mechanism robustness. This is analytically tractable only under myopic response; extensions to strategic or multi-follower settings could break this property.

Additionally, closed-form expressions and gradients equip practitioners to conduct efficient mechanism design for high-dimensional LQ systems when adaptive or online incentives are not feasible.

There are clear application avenues in electricity markets, ancillary service provision, and other decentralized systems where contract-based, non-adaptive incentives must ensure regulation at finer timescales than contract renegotiation allows.

Future Directions

Several research extensions are immediate:

• Multi-follower or multi-leader settings: introducing strategic interactions, potential collusion, and equilibrium multiplicity.
• Strategic followers: allowing forward-looking, dynamic programming-based best responses would generalize the analytic framework, but render optimization more involved, possibly leading to non-affine or history-dependent best-response functions.
• Incentive class generalization: exploring static incentive schemes rationalizing integral or nonlinear closed-loop responses, to remove steady-state tracking error, is an appealing route for improved performance while maintaining structural simplicity.

Conclusion

This work establishes a rigorous framework for offline, static incentive design in leader-follower dynamical systems, with analytical tractability in the linear-quadratic-bilinear regime. Main contributions include tight stability and optimality characterizations, tractable gradients for parameter optimization, and significant insights into asymptotic robustness. These results both inform practical contract design in decentralized infrastructure and invite theoretical exploration into richer game-theoretic and dynamic response settings.