P-Q Dynamic Game Framework

• P-Q dynamic game framework is a descriptor for various models that embed sequential state evolution with extra structures such as partial observability, potential functions, or Q-learning.
• It leverages methodologies like potential and approximate potential formulations and quadratic surrogates to reduce equilibrium computation to tractable optimization problems.
• Applications span domains like vehicle-pedestrian safety, cyber-defense, and network load balancing, demonstrating enhanced prediction, efficiency, and adaptive learning in dynamic interactions.

Within recent dynamic-game literature, the expression “P-Q dynamic game framework” does not denote a single canonical model. Rather, it appears across several technically distinct research programs that combine time-evolving strategic interaction with a structured secondary ingredient such as partial observability, potential structure, parametrization, quantal response, quadratic cost structure, queueing state, or Q-learning. Representative instances include a POMDP integrated with dynamic-belief-induced quantal cognitive hierarchy for vehicle-pedestrian interaction (Dang et al., 2024), dynamic potential and $\alpha$-potential games (Guo et al., 2023, Guo et al., 2024), parameterized affine-quadratic configuration games (Milzman et al., 22 Jul 2025), and generalized individual Q-learning with partial observations in polymatrix games (Donmez et al., 2024). This suggests that the label functions primarily as a cross-cutting descriptor for structured dynamic-game formulations rather than as a universally standardized formalism.

1. Formal scope and recurring mathematical structure

Across these formulations, the common substrate is a dynamic game with explicit state evolution, agent-specific objectives, and a solution concept adapted to sequential interaction. The state may be represented as a partially observable stochastic process, as in the tuple

$\langle N, S, A, T, O, J, B \rangle,$

where $N$ is the set of agents, $S$ the state space, $A$ the joint action space, $T$ the transition map, $O$ the observation sets, $J$ the reward functions, and $B$ the agents’ beliefs about opponents’ cognitive state and rationality (Dang et al., 2024). In affine-quadratic differential games, the state evolves according to

$$\dot{x} = A(t)x + \sum_{i=1}^N B^i(t;\theta^i)u^i(t) + c(t),$$

with player-specific configuration parameters $\langle N, S, A, T, O, J, B \rangle,$0 chosen before the dynamic interaction (Milzman et al., 22 Jul 2025). In open-loop general-sum dynamic games with constraints, the tuple

$\langle N, S, A, T, O, J, B \rangle,$1

encodes nonlinear dynamics, player objectives, and coupled feasibility constraints (Zhu et al., 2022). In strategic cyber-defense models, the macro layer is a Markov game

$\langle N, S, A, T, O, J, B \rangle,$2

while the tactical layer is an extensive-form game (Yang et al., 2 Jul 2025).

A recurring distinction is between frameworks that seek direct equilibrium computation and frameworks that reduce equilibrium search to a more tractable surrogate problem. In dynamic potential games, unilateral deviations are aligned with a potential function, so equilibrium analysis can be converted into optimization over that potential (Guo et al., 2023). In $\langle N, S, A, T, O, J, B \rangle,$3-potential games, the same reduction holds approximately, with the deviation error bounded by $\langle N, S, A, T, O, J, B \rangle,$4 (Guo et al., 2024). In learning-based models, partial observations determine whether agents use belief-based, payoff-based, or hybrid updates on Q-values (Donmez et al., 2024). In bounded-rationality traffic interaction, beliefs over reasoning level and rationality are updated online, making equilibrium reasoning itself dynamic (Dang et al., 2024).

Framework family	Core mathematical object	Solution notion
AV-pedestrian bounded-rationality game	POMDP with DB-QCH	Adaptive strategy via belief updates
Dynamic potential / $\langle N, S, A, T, O, J, B \rangle,$5-potential game	Potential function over policies	Exact or approximate Nash equilibrium
Game of configuration	Two-stage AQ differential game	SPNE
Open-loop constrained dynamic game	Nonlinear GNE system	Local GNE
Partial-observation polymatrix learning	Generalized individual Q-learning	QRE
Multi-resolution cyber game	Extensive-form + Markov game	SPNE and SPE

These variants are unified less by a single notation than by a shared design pattern: strategic interaction is embedded in an evolving state process, and additional structure is imposed to make prediction, learning, or equilibrium computation analytically manageable (Dang et al., 2024, Milzman et al., 22 Jul 2025, Zhu et al., 2022, Yang et al., 2 Jul 2025).

2. POMDP-behavioral formulations and bounded rationality

A prominent instance of the framework is the vehicle-pedestrian model at an unsignalized intersection, where the interaction is formulated as a POMDP coupled with behavioral game theory (Dang et al., 2024). The autonomous vehicle and the pedestrian are both modeled as dynamic-belief-induced quantal cognitive hierarchy (DB-QCH) agents. The POMDP component captures uncertainty and partial observability, especially the fact that reasoning level and rationality are not directly observed. The behavioral-game component introduces bounded rationality through cognitive levels and quantal response.

In the DB-QCH model, agents are assigned cognitive levels $\langle N, S, A, T, O, J, B \rangle,$6, where level-$\langle N, S, A, T, O, J, B \rangle,$7 is naïve and higher levels simulate and best-respond to lower ones. Quantal response makes action selection probabilistic rather than perfectly optimal, controlled by a rationality parameter $\langle N, S, A, T, O, J, B \rangle,$8. Large $\langle N, S, A, T, O, J, B \rangle,$9 produces nearly optimal behavior; small $N$0 produces more random behavior. The quantal response and level-based policy updates are defined through exponential weighting of expected payoff terms. A dynamic belief updating mechanism then allows the autonomous vehicle to update its estimate of the pedestrian’s reasoning level and rationality in real time via Bayesian updates on $N$1 and conjugate-prior Bayesian updates on $N$2.

This formulation differs sharply from traditional game-theoretic traffic models that assume full rationality, perfect anticipation, static reasoning levels, and fixed rationality. Here, the latent cognitive state is itself part of the dynamic state estimate. The action spaces are also asymmetric: the autonomous vehicle uses continuous acceleration, approximated by sampling from a Gaussian distribution whose mean comes from a trained LSTM, and planning is performed with MCTS; the pedestrian uses a discrete set of speed values. This coupling of learned motion priors with game-theoretic planning is central to the framework’s attempt to approximate real, non-equilibrium, and possibly inconsistent human behavior (Dang et al., 2024).

The reported performance indicators are organized around safety, efficiency, and smoothness.

Indicator	Model / simulation	VR comparison
Collision rate	0.15%	—
Speed	9.60 m/s	9.47 m/s
Avg. jerk	0.843	0.897
Max accel	1.867 m/s²	2.228 m/s²

The paper states that the proposed decision-making approach performs well in safety, efficiency, and smoothness, closely resembles real-world driving behavior, and achieves more comfortable driving navigation than previous virtual reality experimental data (Dang et al., 2024). A plausible implication is that explainability in such settings is being pursued not by eliminating behavioral complexity, but by representing that complexity with explicit latent variables—reasoning level, rationality, and belief state—rather than with a purely black-box predictor.

3. Potential, approximate potential, and queue-based dynamic games

Another major branch of the literature centers on dynamic potential structure. A dynamic game is a potential game if there exists a potential function $N$3 such that the change in an agent’s value function under unilateral deviation is exactly the change in the potential: $N$4 An equivalent characterization states that the game is a potential game if and only if each player’s value function decomposes as

$N$5

where the residual term depends only on the policies of the other players (Guo et al., 2023). The same work develops a symmetric Jacobian criterion using functional derivatives with respect to policies, extending the familiar mixed-partials symmetry condition from static games to dynamic policy spaces.

The $N$6-potential framework relaxes exact potentiality by allowing a bounded deviation error: $N$7 This generalization preserves the key reduction: every minimizer of $N$8 is an $N$9-Nash equilibrium, where $S$0 is the optimization accuracy (Guo et al., 2024). The framework also derives an analytical bound on $S$1 in terms of asymmetry in second-order derivatives, and in stochastic differential games characterizes $S$2 using the number of players, strategy spaces, interaction intensity, and heterogeneity. Two special cases are especially notable: distributed games with decoupled dynamics but potential-derived costs yield $S$3, whereas mean field interaction games satisfy $S$4 (Guo et al., 2024).

Dynamic load balancing provides a concrete queue-based realization of this line of work. In that model, there are $S$5 servers with service rates $S$6 and $S$7 selfish players with job sizes $S$8. At each time step, an arriving player observes the state vector of queued loads $S$9, chooses a fractional allocation $A$0, incurs delay cost

$A$1

and updates each server load according to

$A$2

The paper shows that both the static and dynamic formulations are potential games, that best-response dynamics in the static game converge to a pure Nash equilibrium after $A$3 iterations, and that in the dynamic setting the state converges to zero and the strategies converge to the pure equilibrium of the corresponding static game in polynomial time (Fardno et al., 25 Jan 2025).

Taken together, these results make precise a central thesis of the potential-game branch of the P-Q literature: dynamic strategic interaction can sometimes be recast as global or approximate global optimization, with exact or near-exact guarantees determined by derivative symmetry, interaction sparsity, or mean-field scaling (Guo et al., 2023, Guo et al., 2024, Fardno et al., 25 Jan 2025).

4. Quadratic and parameterized-quadratic formulations

Quadratic structure is another major source of tractability. In the “game of configuration,” each player chooses a private configuration parameter $A$4 in a first stage and then participates in a finite-horizon affine-quadratic differential game in the second stage (Milzman et al., 22 Jul 2025). The state dynamics are

$A$5

and player $A$6 minimizes a quadratic cost functional whose state weighting $A$7 depends on the configuration vector. The equilibrium concept is SPNE: configuration choices must form a Nash equilibrium in Stage 1, and the Stage 2 feedback strategies must be Nash for every realized configuration. The paper derives gradients of the downstream cost with respect to configuration parameters by differentiating through the coupled Riccati equations, and uses those gradients in an iterated best response method for local equilibrium search (Milzman et al., 22 Jul 2025).

A different quadratic route appears in numerical methods for nonlinear dynamic games. Differential dynamic programming for non-zero-sum full-information games builds quadratic approximations to Bellman recursions around a nominal trajectory, thereby producing static quadratic stage games that are solved recursively (Di et al., 2018). The DDP step is shown to remain sufficiently close to Newton’s method to inherit local quadratic convergence under a strict local equilibrium and invertibility of the Jacobian of the stacked equilibrium conditions. This places quadratic surrogate games at the center of computation even when the original dynamics and costs are nonlinear.

For constrained open-loop dynamic games, a sequential quadratic programming approach computes local generalized Nash equilibria by solving only a single convex quadratic program at each iteration (Zhu et al., 2022). The method uses a projected positive-definite Hessian approximation, a novel merit function,

$A$8

and a non-monotonic watchdog line search. Under standard regularity assumptions, the algorithm converges linearly to a local GNE. In car-racing experiments, the paper reports up to $A$9 improvement of success rate relative to a state-of-the-art dynamic-game solver (Zhu et al., 2022).

Quadratic potential structure also supports online dynamic games with evolving information. In a two-player finite-horizon linear-quadratic feedback potential game with sequentially revealed costs, players know only the current and previewed cost matrices up to time $T$0 and predict future costs by holding the revealed values constant (Chen et al., 26 Mar 2025). The resulting predict-and-track algorithm is evaluated through dynamic social regret, termed the “price of uncertainty,” and Theorem 1 bounds that regret by horizon-invariant constants: $T$1 The first term decays exponentially with preview length, and the remaining term depends on the magnitude of differences between the players’ cost matrices (Chen et al., 26 Mar 2025).

These quadratic and parameterized-quadratic frameworks reveal a common methodological preference: difficult dynamic interactions are made analyzable by embedding them in Riccati systems, quadratic surrogates, or differentiable parameter spaces that preserve enough structure for equilibrium computation.

5. Information structures, observability, and adaptive learning

Information architecture is often decisive in whether a dynamic game is tractable. One recent framework models deterministic dynamic games with arbitrary interleaved information structures as Mathematical Program Networks (MPNs), in which each node corresponds to an agent-time optimization problem and the network edges encode informational dependencies (K et al., 19 Mar 2026). In the linear-quadratic case, the MPN representation yields Riccati-like equations for Nash equilibria under information patterns that are neither classical feedback nor classical open-loop. A three-agent cyclic example—Agent 1 observes Agent 2, Agent 2 observes Agent 3, and Agent 3 observes Agent 1—illustrates how the KKT conditions assemble into a generalized backward recursion (K et al., 19 Mar 2026).

A closely related line studies decentralized two-team stochastic dynamic games through mutual quadratic invariance (MQI) (Colombino et al., 2016). For the discrete-time linear system

$T$2

each team must choose a controller subject to a prescribed decentralized information structure. MQI is the condition that the closed-loop interaction of the two structured controllers with the plant preserves those structures. Under MQI, zero-sum two-team games admit a tractable equivalence between structured state-feedback saddle points and structured disturbance-feedforward saddle points. The nonzero-sum case is different: the paper provides a counterexample showing that the same equivalence fails to hold (Colombino et al., 2016). This establishes that information-structure tractability is substantially more fragile in nonzero-sum settings.

Partial observation also shapes learning dynamics. In generalized individual Q-learning for polymatrix games, each agent combines belief-based updates for observed opponents with payoff-based Q-learning for unobserved opponents (Donmez et al., 2024). The observability pattern is described by a graph, and the smoothed best response

$T$3

governs stochastic action selection. The paper proves almost sure convergence of empirical action distributions to quantal response equilibrium in zero-sum and potential polymatrix games under any partial observation structure, and simulations show monotonic improvement in convergence rate as observation becomes richer (Donmez et al., 2024).

At a larger scale, cross-echelon cyber-defense models use information and abstraction jointly. A multi-resolution framework represents tactical interactions as high-resolution extensive-form micro base games (MBGs) and strategic planning as a lower-resolution Markov game over states that abstract those MBGs (Yang et al., 2 Jul 2025). Zoom-in and zoom-out operations selectively change resolution, updating tactical utilities with strategic continuation values or updating macro strategies with MBG outcome probabilities. The case study reports lower game values for the attacker as more tactical detail is introduced, especially when attacker capability is high (Yang et al., 2 Jul 2025). This suggests that selective refinement of the information state can itself be a strategic resource.

6. Broader antecedents, neighboring usages, and common confusions

Several earlier frameworks provide conceptual antecedents even when they do not use the same terminology. The “dynamic system of games” models an event as a graph of interconnected static games, with transitions determined by dominant strategies or rational action pairs (Askari et al., 2018). At each node, potentially all players move simultaneously; node preferences govern local outcomes, while systemic preferences govern longer-run trajectories. The model distinguishes “game-maker” and “strategy-maker” games, introduces explicit transition maps $T$4 and $T$5, and uses system history $T$6 to represent path dependence. Its application to US-Soviet relations from World War II to October 1962 culminates in “Rostam’s Dilemma,” where weak trust and hyper-rationality shape the transition from conflict toward cooperation (Askari et al., 2018).

A related systems perspective appears in work linking Game Theory and System Dynamics. There, repeated best-response updates are embedded in time-discrete simulations, so Nash equilibria appear as steady states of multi-decision-maker systems (Rasouli, 2014). The oligopoly example converges to the unique equilibrium $T$7 under iterative best responses, while a pollution-regulation example exhibits multiple Nash equilibria, with System Dynamics used to determine which equilibrium is realized from a given initial condition (Rasouli, 2014). This line emphasizes transient dynamics and convergence diagnostics rather than purely equilibrium characterization.

One frequent source of confusion is the notation $T$8. In positional game theory, $T$9 denotes the number of elements claimed per move by the two players, as in the Maker-Breaker crossing game on the triangular grid (Wallwork, 2022). There, Maker has a winning strategy if $O$0 and $O$1, while Breaker has a winning strategy if $O$2 and $O$3 (Wallwork, 2022). Although this is a dynamic, turn-based, adversarial game, it is formally unrelated to potential-game, quantal-response, queueing, or quadratic-control frameworks. The coincidence of notation should not be read as a shared theory.

Another neighboring usage appears in game-theoretical interpretations of PDEs. For the doubly nonlinear parabolic equation

$O$4

a stochastic two-player zero-sum game is constructed from a new asymptotic mean value formula and a dynamic programming principle

$O$5

with the game value converging to the viscosity solution (Teso et al., 13 Apr 2026). This is again a dynamic game with a state process and value function, but its purpose is analytical representation of a nonlinear PDE rather than equilibrium analysis in economics, control, traffic, or cyber systems.

The broader record therefore supports two clarifications. First, there is no single universally accepted expansion of “P-Q” in dynamic-game research. Second, the most substantive unifier across the cited frameworks is not nomenclature but method: each framework introduces extra structure—belief states, potential functions, queue states, parametric pre-games, quadratic surrogates, information constraints, or dynamic programming operators—to make sequential strategic interaction mathematically analyzable.