ADAGE: A generic two-layer framework for adaptive agent based modelling (2501.09429v1)

Abstract: Agent-based models (ABMs) are valuable for modelling complex, potentially out-of-equilibria scenarios. However, ABMs have long suffered from the Lucas critique, stating that agent behaviour should adapt to environmental changes. Furthermore, the environment itself often adapts to these behavioural changes, creating a complex bi-level adaptation problem. Recent progress integrating multi-agent reinforcement learning into ABMs introduces adaptive agent behaviour, beginning to address the first part of this critique, however, the approaches are still relatively ad hoc, lacking a general formulation, and furthermore, do not tackle the second aspect of simultaneously adapting environmental level characteristics in addition to the agent behaviours. In this work, we develop a generic two-layer framework for ADaptive AGEnt based modelling (ADAGE) for addressing these problems. This framework formalises the bi-level problem as a Stackelberg game with conditional behavioural policies, providing a consolidated framework for adaptive agent-based modelling based on solving a coupled set of non-linear equations. We demonstrate how this generic approach encapsulates several common (previously viewed as distinct) ABM tasks, such as policy design, calibration, scenario generation, and robust behavioural learning under one unified framework. We provide example simulations on multiple complex economic and financial environments, showing the strength of the novel framework under these canonical settings, addressing long-standing critiques of traditional ABMs.

• The paper introduces ADAGE, a two-layer framework using Stackelberg games to model adaptive agent-based systems where agent behavior and environments dynamically co-evolve.
• ADAGE is shown to be versatile across key agent-based modeling tasks, including adaptive policy design, model calibration, scenario generation, and robust behavioral learning.
• The framework enhances adaptive modeling by seamlessly integrating reinforcement learning and other optimization methods for complex agent-environment interactions across various domains.

A Comprehensive Overview of the ADAGE Framework for Adaptive Agent-Based Modeling

The paper introduces a novel adaptive framework for agent-based modeling (ABM) known as ADAGE, which addresses long-standing critiques of ABMs, particularly the Lucas critique related to fixed agent behavior. ADAGE stands as a significant methodological advancement, providing a rigorous formalization for the adaptation of agents and environments, encapsulated within a Stackelberg game framework. Its objectives are twofold: to furnish a generic architecture that encapsulates diverse ABM tasks and to introduce an adaptable system where both agent behaviors and environmental states co-evolve in response to each other.

ABMs have been critiqued for their static agent behaviors, which fail to adjust systematically to environmental changes, challenging their applicability in dynamic real-world scenarios. The Lucas critique brings to light the limitations of using static models in situations where policy changes can alter agent decision rules. ADAGE directly addresses these limitations by establishing a two-layer system where agents adaptively respond to continuous environmental shifts.

The framework capitalizes on Stackelberg games, which allow modeling of hierarchical interactions between an outer layer (environmental characteristics or policy maker) and an inner layer (variable agent behaviors). Specifically, the leader in the framework optimizes environmental policies or characteristics, such as tax rates or resource distributions, while the followers (agents) learn behavioral policies conditioned on these environmental states.

ADAGE is proven to be versatile across several essential ABM tasks, including:

1. Policy Design: As demonstrated in a taxation simulation, ADAGE successfully learned adaptive tax policies that enhance social welfare, outperforming non-adaptive baselines. The tax policies were optimized in real-time, allowing for streamlined policy design informed by simulated household behaviors.
2. Model Calibration: Utilizing economic data from a cobweb market experiment, ADAGE effectively calibrated agent decision-making parameters to match empirical market conditions, yielding a closer alignment between simulated and real-world data compared to non-calibrated benchmarks.
3. Scenario Generation: By fostering scenarios that minimize market volatility through adaptive settings of Tobins tax rates, ADAGE demonstrated its capability to autonomously generate environments leading to desired systemic outcomes, such as stabilized market dynamics.
4. Robust Behavioral Learning: ADAGE facilitated the training of a market maker agent capable of adapting its spread determination based on varying priorities of market share and profit, showcasing its effectiveness in generating robust policies across diverse agent preferences.

The paper rigorously evaluates ADAGE on multiple simulators, highlighting its utility across a broad spectrum of domains, from financial markets to economic policy making. Its ability to seamlessly integrate reinforcement learning—as well as alternative optimization strategies such as Bayesian methods—underscores its adaptability and potential for widespread application.

In conclusion, the ADAGE framework sets a new precedent for adaptive modeling in agent-based simulations, offering a coherent structure that accommodates complex interactions between agents and their environments. It not only enriches the methodological repertoire for computational economists and multi-agent system researchers but also lays a foundation for future developments in AI-powered adaptive modeling frameworks.