Heterogeneous Agent Models with Aggregate Risk

• Heterogeneous agent models with aggregate risk are macroeconomic frameworks that capture individual and common shocks to derive high-dimensional stochastic equilibria.
• These approaches combine dynamic programming, PDEs, and machine learning techniques like PINNs and reinforcement learning for scalable solutions in incomplete markets.
• Equilibrium is achieved through fixed-point methods ensuring consistency between agent distribution evolution and market-clearing conditions.

Heterogeneous agent models with aggregate risk are a cornerstone of modern macroeconomic analysis, capturing the interaction of individually optimizing agents facing both idiosyncratic and common shocks, and leading to high-dimensional stochastic equilibria. In these models, the endogenous distribution of agent-level state variables is a crucial state variable, and aggregate risk couples the cross-sectional law of motion with the evolution of prices and aggregate shocks. The solution techniques for such models combine advanced tools from dynamic programming, partial differential equations, mean field game theory, simulation-based reinforcement learning, and machine learning. The recent literature provides a suite of scalable, global algorithms for these challenging environments, allowing for the computation of equilibria in incomplete market settings with realistic aggregate shocks.

1. Mathematical Foundations of Heterogeneous Agent Models with Aggregate Risk

The rigorous formulation of heterogeneous agent (HA) models with aggregate risk requires specifying the dynamics of both the cross-sectional distribution of agents and the aggregate (common) shocks. Agents are indexed by idiosyncratic state $x$ and face aggregate state variables $z$. The central object is the joint evolution of the value function $V(x, z, g)$—where $g$ denotes the distribution over $x$—satisfying a high-dimensional non-linear Hamilton–Jacobi–Bellman (HJB) equation, and the law of motion for $g$, typically given by a Kolmogorov–Fokker–Planck (KFP) or Chapman–Kolmogorov equation. Equilibrium requires “belief consistency”: the law of motion for $g$ generated by agents’ optimal policies and market clearing coincides with expectations.

The HJB for the value function is

$$0 = \max_{c\in C(x, z, g)} \left\{ -\rho V + u(c, z, Q(z, g)) + \psi(c, x, Q) + \mathcal{L}_x^{c, Q} V + \mathcal{L}_z V + \mathcal{L}_g^{\mu_g} V \right\},$$

where $Q(z, g)$ denotes the (possibly nonlinear) pricing function, and the diffusion, jump, and mean-field operators act on the respective variables (Gu et al., 19 Jun 2024).

The master equation, which unifies the value function and the law of motion for the distribution $g$, is often infinite-dimensional and intractable for direct numerical solution (Gu et al., 19 Jun 2024, Lyasoff, 2023). Aggregate risk enters both directly, as a state variable, and indirectly via its feedback on the endogenous price process.

2. Numerical Methods for Solving Heterogeneous Agent Models with Aggregate Risk

Recent advances have produced a range of tractable, global solution techniques:

• Finite Difference and Semi-Lagrangian Schemes. Grid-based policy iteration and semi-Lagrangian schemes (e.g., Howard’s policy iteration, upwind discretization, and dual schemes for the Fokker–Planck equation) are used to solve the coupled HJB/KFP system in canonical models such as Aiyagari–Bewley–Huggett (Salguero, 28 Sep 2025, Camilli et al., 1 Oct 2025). These methods are robust in low dimensions but scale poorly when extending to higher-dimensional individual or aggregate shocks.
• Physics-Informed Neural Networks (PINNs) and Economic Model-Informed Neural Networks (EMINNs). Mesh-free, global neural network solvers have been developed to approximate the entire value function and the cross-sectional distribution. PINNs embed the residuals of HJB and Kolmogorov–Forward equations directly into a physics-constrained loss, ensuring scalability and smooth approximation properties (Grzeskiewicz, 25 Nov 2025). EMINNs extend this approach to the master equation, using rich basis representations of the cross-sectional distribution and enabling global accuracy in high-dimensional settings (Gu et al., 19 Jun 2024).
• Moment-Reduction and Generalized Moment Representations. Approximating the cross-sectional distribution with generalized moments or projected basis functions allows a drastic reduction in state-space dimensionality. DeepHAM, for instance, replaces the $N$-dimensional distribution of agent states with permutation-invariant moment vectors, which serve as inputs to the deep neural network policy and value approximators (Han et al., 2021).
• Structural Reinforcement Learning (SRL). A simulation-based approach replaces the high-dimensional cross-sectional state with a low-dimensional vector of equilibrium prices and seeks policy functions that map only the agent’s individual state and aggregate prices to actions. Structural knowledge of transition matrices is exploited, and expectations are computed via Monte Carlo simulation of aggregate price and shock paths (Yang et al., 21 Dec 2025).
• Time-Interlaced Master Systems. Direct solution of the interlaced equilibrium system, linking forward and backward-in-time fixed points for policies and cross-sectional transition operators, allows for numerically verifiable solutions even when classical fixed-point iteration fails for models like Aiyagari–Bewley–Huggett with discontinuous stationary laws (Lyasoff, 2023).

3. Approximating and Projecting the Distribution of Agents

A central challenge is the representation and evolution of the endogenous cross-sectional distribution $g$ under aggregate risk:

• Finite Agent and Discrete-State Approximations: The agent distribution is compressed by considering a finite sample (empirical measure) or discretizing the state space. In both cases, the law of motion for the empirical distribution (vector $\phi$) replaces the full KFP equation, reducing the PDE dimension and enabling neural approximation (Gu et al., 19 Jun 2024).
• Projection onto Basis Functions: Projecting the distribution onto orthonormal bases (e.g., eigenfunctions of the state space or principal components) allows the evolution of the weights (moments $\phi$) to code for high-dimensional population heterogeneity using a low-dimensional parameterization (Gu et al., 19 Jun 2024).
• Generalized Moments (Editor's term): DeepHAM parameterizes the effect of the distribution using learned, permutation-invariant features ($m^V, m^C$), which interact with macro variables in the agent decision problem. This allows the solver to sidestep the curse of dimensionality inherent in tracking full micro distributions (Han et al., 2021).

4. Market Clearing, Belief Consistency, and Equilibrium Computation

Satisfying equilibrium in HA models with aggregate risk requires a fixed-point over distributions, policies, and prices.

• Root-Finding/Fixed-Point Methods: Classical approaches use root-finding loops (e.g., on the aggregate interest rate) or bisection to enforce goods or asset market clearing. Grid- and simulation-based algorithms iteratively update aggregate prices until the markets clear given the induced stationary or transition distributions (Salguero, 28 Sep 2025, Camilli et al., 1 Oct 2025, Grzeskiewicz, 25 Nov 2025).
• Self-Consistent Master System: Interlacing backward and forward sweeps, as in the time-interlaced master system, ensures that policy functions and transitions remain consistent across time, sidestepping issues with non-uniqueness and discontinuity in stationary distributions found in Aiyagari–Bewley–Huggett–type models (Lyasoff, 2023).
• Monte Carlo and Simulation-Based Market Clearing: Simulation-based approaches (e.g., SRL) compute aggregate supply schedules in each simulated period and clear the market by interpolation or vectorized procedures, sidestepping the necessity of computing fixed-points over the full distribution space (Yang et al., 21 Dec 2025).
• Physics-Constrained Loss or Penalty: PINN/EMINN frameworks encode equilibrium and consistency directly as penalty functions that enforce, in mean-square or mass-conservation, the correct law of motion and market clearing at each collocation point of the training set (Gu et al., 19 Jun 2024, Grzeskiewicz, 25 Nov 2025).

5. Recent Innovations: Neural and Reinforcement Learning Global Solvers

Advances in machine learning have enabled global solvers for heterogeneous agent models with aggregate risk that are both scalable and accurate:

Machine Learning-Based Solvers

Method	Distribution handling	Value/Policy Approximation	Market-Clearing
DeepHAM (Han et al., 2021)	Generalized moments $m^V, m^C$	Deep NN (input: state, macro vars, moments)	Simulation + fixed-point
EMINN (Gu et al., 19 Jun 2024)	Projected basis, moments $\phi$	Deep NN on $(x, z, \phi)$	Physics-informed loss
PINN (Grzeskiewicz, 25 Nov 2025)	NN for agent density $f(a,z,t)$	NN for value function $V(a,z,t)$	Mass-conservation, boundary loss

Structural reinforcement learning (SRL) (Yang et al., 21 Dec 2025) further simplifies the state space to observed price trajectories, exploiting known transition structure and using a policy gradient approach over simulated market paths. Crucially, these approaches avoid the curse of dimensionality, which otherwise renders classical grid- or simulation-based model solution intractable for high-dimensional problems.

Diagnostic Checks and Benchmarks

• Residual-based Diagnostics: Wasserstein-2 contraction, maximum-norm policy error, and market-clearing residuals are used to validate the convergence and correctness of numerical solutions (Salguero, 28 Sep 2025, Han et al., 2021, Grzeskiewicz, 25 Nov 2025).
• Analytical Comparisons: Where feasible, models are validated against known analytical solutions (e.g., the Merton problem in the absence of volatility, Krusell–Smith regression benchmarks) (Camilli et al., 1 Oct 2025, Grzeskiewicz, 25 Nov 2025, Gu et al., 19 Jun 2024).
• Empirical and Simulation Benchmarks: Models are tested for replication of aggregate time-series properties and wealth distribution dynamics.

6. Model Classes and Applications

The computational machinery described supports a range of canonical and applied heterogeneous agent models with aggregate risk:

• Canonical Incomplete Market Models: Aiyagari–Bewley–Huggett models with or without capital, featuring borrowing constraints and incomplete insurance against idiosyncratic shocks (Salguero, 28 Sep 2025, Camilli et al., 1 Oct 2025, Grzeskiewicz, 25 Nov 2025).
• Production Economies with Aggregate Shocks: Krusell–Smith–type models, incorporating a production function, endogenous capital, and policy-function regression on aggregate moments (Gu et al., 19 Jun 2024, Yang et al., 21 Dec 2025, Lyasoff, 2023).
• Hybrid Models with Macroeconomic and Financial Frictions: Heterogeneous Agent New Keynesian (HANK) models with forward-looking nominal rigidities (Yang et al., 21 Dec 2025).
• Spatial and Firm Models: Models extending HA frameworks to spatial migration, sectoral, or firm heterogeneity (Gu et al., 19 Jun 2024).
• Reinforcement Learning for General Spatial or Epidemiological Macroeconomic Models: Deep RL methods flexibly accommodate models with nonstandard or multi-layered agent interactions and observables (Hill et al., 2021).

7. Limitations and Future Directions

Common limitations include the handling of highly non-Markovian or path-dependent pricing kernels, the curse of dimensionality for extremely large or richly featured agent spaces, and the challenge of encoding forward-looking learning or expectation formation. Recent work suggests potential extensions through online RL, biased sampling, joint learning of policy and aggregate law of motion (e.g., model-based SRL), and adaptation to new problem classes, such as multi-asset HANK, financial frictions, or interacting multi-sector economies (Yang et al., 21 Dec 2025, Gu et al., 19 Jun 2024). Analytical convergence theorems—beyond classical stochastic approximation results—remain rare, with most approaches validated by empirical and numerical benchmarks.

---

The rapid progress in algorithms for heterogeneous agent models with aggregate risk—spanning grid-based policy iteration, neural PDE solvers, and simulation-based reinforcement learning—has enabled researchers to compute equilibria in models with realistic dimensions and aggregate uncertainty. These advances facilitate the analysis of policy, distributional dynamics, and macroeconomic risk in environments where heterogeneity and aggregate shocks interact in essential ways (Yang et al., 21 Dec 2025, Gu et al., 19 Jun 2024, Han et al., 2021, Grzeskiewicz, 25 Nov 2025, Camilli et al., 1 Oct 2025, Salguero, 28 Sep 2025, Lyasoff, 2023).