Induced Stackelberg Equilibrium Seeking via Iterative Tikhonov Regularization

Abstract: Existing methods for learning Stackelberg equilibria typically assume that the followers' (variational, generalized) Nash equilibrium is unique. However, in the presence of multiple equilibria, without a selection convention, the problem may become ill-posed, thus leading standard algorithms to potentially fail to converge. This paper addresses this issue by introducing an optimal selection at the lower-level game, hereby defining a Stackelberg game with induced equilibrium selection. To this end, we enable the leader to augment the followers' game with an additional vanishing term that acts as an incentive. We then propose a follower-agnostic zeroth-order method, whereby the leader converges to a solution of the resulting problem by iteratively probing the followers and jointly updating its decision variable and the incentive term.

• The paper introduces an iterative Tikhonov regularization framework to resolve ambiguity by enforcing a unique follower equilibrium in Stackelberg games.
• It leverages a follower-agnostic zeroth-order algorithm that approximates the upper-level gradient without explicit lower-level model knowledge.
• Numerical experiments in P2P trading demonstrate robust convergence and improved system performance, underlining the method's practical implications.

Induced Stackelberg Equilibrium Selection via Iterative Tikhonov Regularization

Introduction and Problem Setting

This paper, "Induced Stackelberg Equilibrium Seeking via Iterative Tikhonov Regularization" (2604.26746), addresses a longstanding issue in hierarchical multi-agent decision-making: ambiguity induced by non-unique lower-level equilibria in Stackelberg games. Standard Stackelberg methods typically assume the uniqueness of lower-level generalized Nash equilibria (GNE). When monotonicity conditions are present and the lower-level’s pseudogradient is monotone but not strongly monotone, the equilibrium set becomes convex and potentially high-dimensional, leading to set-valued follower responses and rendering the upper-level problem ill-posed without explicit equilibrium selection. Traditional "optimistic" or "pessimistic" selection approaches lack operational interpretability in online or decentralized schemes and may not generalize to practical iterative algorithms deployed under incomplete information.

The authors propose explicit and operational equilibrium selection using a strongly convex function $\phi(x)$ to regularize the lower-level game, making the follower mapping single-valued and yielding a well-posed, interpretable induced Stackelberg equilibrium (SE).

Motivating Example and Formalization

The ill-posedness of the conventional bilevel SE formulation is highlighted through a $1$-leader/$2$-follower quadratic Stackelberg game. When the lower-level admits an affine manifold of equilibria (i.e., infinitely many equilibria indexed by one component $x_1$), naively applying gradient-based updates at the upper level can cause oscillations or convergence to non-stationary points, depending entirely on the mechanism of follower equilibrium "selection."

Figure 1: If the sequence $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$ violates the conditions required for stability, leader iterates do not converge.

Introducing a selection function $\phi(x)$ that is strongly convex (e.g., $\phi(x) = (x_1 - 1)^2 + 100 x_2^2$) induces a unique equilibrium $x_\phi^*(y)$ for each leader action $y$. This approach resolves ambiguity and aligns the follower response mapping with the leader’s optimization.

Figure 2: If a selection $\phi(x)$ is introduced, the follower reaction, $1$0, is uniquely determined by $1$1. Without accounting for this dependence, the $1$2-induced SE cannot be achieved.

This example demonstrates that unless the leader is aware of, and reasons about, the actual induced follower selection (and its dependence on $1$3), iterative schemes fail to find a stationary point of the compositional objective $1$4.

Tikhonov Regularization Framework

The core technical contribution is the operationalization of selection via Tikhonov regularization. For each leader action $1$5, the lower-level GNE is regularized by adding the vanishing strongly convex term $1$6 to follower objective functions, yielding a unique solution $1$7. As $1$8, the solution sequence converges to the minimizer of $1$9 over the set of $2$0-GNEs for the given $2$1. This approach draws on variational inequality (VI) theory, where Tikhonov regularization is known to enforce uniqueness over monotone solution sets.

The leader’s problem becomes solving the induced (well-posed) bilevel program

$2$2

where $2$3, and $2$4 is the (convex) set of $2$5-GNEs.

Follower-Agnostic Zeroth-Order Algorithm

A major practical challenge in dynamic games is that the leader generally lacks knowledge of the lower-level game, especially the model and structure required to compute gradients of $2$6 with respect to $2$7. The authors propose a two-timescale, follower-agnostic, zeroth-order algorithm for the upper-level updates:

1. At each iteration $2$8, the leader perturbs $2$9 by a random vector $x_1$0 of radius $x_1$1.
2. The induced lower-level games (with incentive $x_1$2) are played at $x_1$3 and $x_1$4, returning $x_1$5 and $x_1$6.
3. A finite-difference estimator approximates the gradient of $x_1$7 with respect to $x_1$8.
4. Leader updates $x_1$9 and decays $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$0 on a slower timescale.

Theoretical analysis (Theorem 1) shows that under Lipschitz, smoothness, and stepsize/decay separation conditions ($\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$1), the iterates converge in expectation to stationary points of the bilevel objective as $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$2. Bounds are provided with explicit rates in terms of regularization, smoothing, and gradient bias errors.

Numerical Experiments: Energy Community P2P Trading

Application to an energy community peer-to-peer (P2P) trading setup with networked agents is provided. The lower-level agents decide on generation, storage, and trading subject to grid constraints; the leader (community manager) sets trading prices to minimize social cost plus deviation from a reference contract price. The lower-level’s monotone structure results, for each price profile, in a nontrivial set of equilibria.

The selected regularization (incentive) term $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$3 is designed to incentivize reduced grid loading, increased renewables, and improved fairness by penalizing deviation from desirable generation and power flow profiles. Simulation on the IEEE 13-bus test feeder demonstrates that Algorithm 1—with appropriate joint decay of incentive and learning rates—drives the global cost toward an optimum of the induced Stackelberg problem.

Figure 3: Evolution of $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$4 throughout the iterations of Algorithm 1 (Zeroth-order method), demonstrating robust decrease and convergence.

Theoretical and Practical Implications

Formulating Stackelberg games with an explicit, leader-enforced equilibrium selection presents a compelling paradigm for both the theoretical understanding and algorithm design for hierarchical learning in the presence of lower-level multiplicity and ambiguity. Unlike approaches reliant on model knowledge or (potentially infeasible) gradient tracing, the proposed framework supports black-box, query-based interaction, compatible with legacy systems and practical multi-agent deployments.

The regularized bilevel structure admits formal convergence analysis and offers interpretable steering of system-wide metrics through the design of $\left(x_{1 \mid k}\right)_{k\in \mathbb{N}}$5. This methodology is extensible to dynamic, networked, and stochastic games, provided monotonicity and convexity of follower dynamics.

Conclusion

This work formalizes the Stackelberg equilibrium selection problem under follower multiplicity and introduces an iterative Tikhonov regularization scheme with a follower-agnostic zeroth-order upper-level optimizer. The result is an operational and interpretable mechanism for inducing well-posed equilibria, offering convergence guarantees under mild regularity, and providing a design handle for incentivizing favorable system outcomes. Future work may extend this framework to non-monotone games, robust selection under partial compliance, and reinforcement learning-based Stackelberg dynamics.