Stackelberg Equilibrium in Sequential Games

• Stackelberg equilibrium is a hierarchical, sequential solution concept where a leader commits to a strategy first and the follower best-responds.
• The framework employs advanced methods, including BSDEs, HJB equations, and stochastic target formulations to handle continuous, discrete, and dynamic settings.
• Its applications in economics, engineering, and control illustrate distinct strategic behaviors compared to simultaneous-play Nash equilibria.

A Stackelberg equilibrium is a hierarchical solution concept for sequential games with asymmetric roles, most classically involving a “leader” who commits to a strategy first, followed by a “follower” who best-responds. This paradigm is central in economics, engineering, and control, where one agent with commitment power strategically anticipates how another agent will optimally respond. The concept generalizes Nash equilibrium by introducing hierarchy, leading to distinct mathematical structures and equilibrium selection properties. Recent advances have made Stackelberg equilibrium tractable in continuous, discrete, stochastic, and dynamic settings through techniques ranging from Hamilton–Jacobi–Bellman equations to bi-level programming, variational inequalities, and stochastic control with target constraints.

1. Foundational Formulation and Hierarchical Structure

Stackelberg equilibrium formalizes the intuition of commitment in sequential games. Suppose two agents interact over a finite time horizon $[0,T]$, jointly controlling a stochastic output process $X$, with the leader’s control denoted $\alpha$ and the follower’s $\beta$. The leader selects $\alpha$ first; the follower then observes $\alpha$ and chooses her best response $\beta^\star(\alpha)$. Formally, payoffs are

$$J_{\mathrm{L}}(\alpha, \beta) = \mathbb{E}^{\bar{\mathbb{P}}}\bigg[ \int_0^T C_s(X_{\cdot\wedge s}, \alpha_s, \beta_s) \,\mathrm{d}s + G(X_{\cdot\wedge T}) \bigg]$$

$$J_{\mathrm{F}}(\alpha, \beta) = \mathbb{E}^{\bar{\mathbb{P}}}\bigg[ \int_0^T c_s(X_{\cdot\wedge s}, \alpha_s, \beta_s) \,\mathrm{d}s + g(X_{\cdot\wedge T}) \bigg]$$

The Stackelberg equilibrium $(\alpha^\star, \beta^\star)$ satisfies:

• $X$0 (follower best-responds)
• $X$1 (leader anticipates optimal follower reaction)

This sequential setting can be contrasted with Nash equilibria, which require simultaneity and mutual best-responses among all agents, leading to key differences in solution structure and equilibrium payoffs (Liu et al., 2024).

2. Stackelberg Equilibrium in Continuous-Time Stochastic Differential Games

The transition from static to dynamic settings with stochasticity introduces new mathematical challenges. In continuous-time games, $X$2 is typically modeled via a controlled stochastic differential equation under Brownian motion $X$3: $X$4 Stackelberg equilibrium is characterized by optimizing over progressively measurable closed-loop strategies—functions of trajectory and time—requiring sophisticated information structures to account for historical (and potentially partial) observation (Hernández et al., 2024).

Closed-loop Stackelberg games are fundamentally bi-level stochastic optimal control problems. The follower’s value for any fixed leader control $X$5 is given by: $X$6 and the leader’s value anticipating this response is: $X$7 These problems are generally solved via backward stochastic differential equations (BSDEs), with the leader needing to solve a nonstandard stochastic control problem with target-type constraints (Hernández et al., 2024).

3. Reduction to Stochastic Target Formulation and Second-Order BSDEs

A central methodological insight is the recasting of the bi-level Stackelberg problem into a single-level stochastic target problem. For a fixed $X$8, the follower’s value process $X$9 over $\alpha$0 satisfies a second-order BSDE (2BSDE): $\alpha$1 with a minimality constraint on the nondecreasing process $\alpha$2. The driver $\alpha$3 encodes the follower's control optimization at each time, and the maximizer $\alpha$4 is recovered pointwise from the $\alpha$5 process.

By introducing a forward system involving the state $\alpha$6 (where $\alpha$7 encodes the follower’s running value if both agents play optimally from time $\alpha$8 onward), the Stackelberg equilibrium is recast as a stochastic target problem: for given $\alpha$9, find $\beta$0 almost surely, ensuring the follower's value is achieved at the terminal state, or, equivalently, enforce

$\beta$1

with the follower control at $\beta$2 being $\beta$3. The leader then optimizes her own payoff subject to this target constraint, which encodes the anticipatory best response behavior of the follower in the system dynamics (Hernández et al., 2024).

4. Hamilton–Jacobi–Bellman Characterization of Stackelberg Equilibria

Once reformulated, the Stackelberg equilibrium problem for closed-loop strategies yields a complex system of Hamilton–Jacobi–Bellman (HJB) partial differential equations, with boundary conditions described by the stochastic target (the constraint $\beta$4). The value function $\beta$5—meaning the leader's optimal expected payoff when at time $\beta$6 the state is $\beta$7, follower’s value is $\beta$8—obeys: $\beta$9 with complementary HJBs on the lower and upper boundaries $\alpha$0 of the attainable set, enforcing the stochastic target constraint. Here, $\alpha$1 maximizes over leader control $\alpha$2, candidate feedback for the follower, and certain Lagrange multipliers enforcing the value-target.

Auxiliary boundary functions $\alpha$3 describing the reachable set for the $\alpha$4 dynamics are also characterized as viscosity solutions of their own PDEs, enforcing constraint qualifications at the edges. The system altogether enables explicit characterization—and sometimes computation—of the closed-loop Stackelberg equilibrium (Hernández et al., 2024).

5. Illustrative Example: Linear-Quadratic Stackelberg Game

A concrete realization is provided by a linear-quadratic model: $\alpha$5 with quadratic costs for the leader and follower: $\alpha$6 The 2BSDE for the follower and its embedding into the target system are solved explicitly. The reachable boundaries $\alpha$7 admit closed-form solutions, and the HJB system—in this case, a single PDE in reduced state variables—is suitable for efficient numerical treatment.

Numerical studies show strict ordering of the leader and follower values depending on the information structure (open-loop, feedback, closed-loop) and demonstrate the impact of embedding the follower's value process as a stochastic state in the leader's optimization trajectory (Hernández et al., 2024).

6. Implications: Information Structure, Computation, and Comparative Statics

The reduction to stochastic target problems illustrates that Stackelberg equilibrium with closed-loop strategies is fundamentally an optimal control with state constraints defined by the best-response map of the follower. This contrasts with open-loop or feedback formulations—where the leader’s strategy is optimized without explicit constraint on the follower’s continuation value—leading to different value functions and equilibrium outcomes. The existence and regularity of solutions depend intricately on the reachability of the target and regularity of the system coefficients.

The approach facilitates theoretical and numerical comparison with alternative information structures (open-loop, feedback, closed-loop memoryless), yielding a full ordering of achievable payoffs for each agent and confirming that richer information under closed-loop schemes always benefits the leader (and possibly the follower) (Hernández et al., 2024).

7. Extensions and Generalizations

The stochastic target reformulation and HJB system admit extensions to more general classes of Stackelberg games, including higher-dimensional state and control, more general payoff functionals, and models where control impacts both drift and volatility. The framework aligns with methodologies developed by Soner, Touzi, and coauthors for target problems in stochastic control, and positions the Stackelberg equilibrium as a “maximal solution” to a composite HJB system with state constraints determined by follower continuation values.

Further research directions include refining regularity conditions for the boundary functions, extending to multi-agent hierarchies with multiple followers, and exploiting these methods in practical applications such as principal-agent problems under model uncertainty, energy market design, and dynamic resource allocation (Hernández et al., 2024).