AddictSim: Simulating Addiction Dynamics

• AddictSim is a family of simulation frameworks that integrates reinforcement learning, neural dynamics, economic theory, and adaptive networks to study addictive behaviors.
• It quantitatively models addiction from individual neural changes to population-level dynamics using versatile mathematical and algorithmic approaches.
• The frameworks reveal critical thresholds and phase transitions, offering actionable insights into effective intervention and risk mitigation strategies.

AddictSim encompasses a family of simulation frameworks and mathematical models for analyzing the dynamics, mechanisms, and interventions related to addictive behaviors in both biological and artificial agents. These models span individual cognition, neural population dynamics, behavioral reinforcement learning, economic decision-making, population-level epidemiology, and social contagion processes, offering a quantitative foundation for understanding addiction and for evaluating risk mitigation strategies.

1. Theoretical Foundations and Model Scope

AddictSim frameworks integrate diverse disciplinary perspectives, including reinforcement learning theory, neurobiological circuit modeling, economic addiction theory, population dynamics, and adaptive network theory.

• Reinforcement Learning–Based AddictSim: Models emergence of addictive policies in agents as arising from unbalanced reward signals, exemplified by agents in environments where certain actions produce disproportionately large, immediate rewards (“drug” rewards) relative to long-term healthy behaviors (Behzadan et al., 2018).
• Neural Population Dynamics AddictSim: Implements dopamine-modulated plastic attractor networks to study addiction-induced transformations in brain information processing, entropy, and stability (Chary, 2012).
• Economic and Recommendation System AddictSim: Couples rational addiction theory with user consumption data and deep model simulation to analyze addiction in digital media (e.g., short-video platforms) (Xu et al., 22 Jan 2026).
• Population Dynamic and ODE-Based AddictSim: Encodes competition and switching dynamics among multiple substances or behaviors in a coupled-compartment epidemiological structure (Colegate et al., 2022).
• Social Contagion and Adaptive Network AddictSim: Simulates co-evolution of drug use, addiction status, and social network topology under endogenous rewiring and utility-weighted behavioral transitions (Lee et al., 26 Jun 2025).

2. Mathematical and Algorithmic Structure

AddictSim models employ a range of algorithmic and mathematical frameworks:

Reinforcement Learning Models

• Environment Setup: Grid-worlds (e.g., modified Snake) with enhanced (“drug”) and standard (“healthy”) reward objects. The agent’s Q-learning update is:

$$Q_{new}(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s',a') - Q(s,a) \right]$$

with $r$ being $k \cdot r_c$ for drug actions ($k > 1$) and $r_c$ for healthy actions (Behzadan et al., 2018).

• Addictive Policies: Sufficient conditions for addiction emergence formalized as $(k-1) > (n^2 - L_0) \cdot \gamma$ and feasibility constraint $k/u < 1$.
• Exploration: $\epsilon$-greedy action selection.

Neural Network and Hebbian Plasticity

• Dopamine-Modulated Plastic Networks: Core Hopfield dynamics with three-factor synaptic update:

$$\Delta W_{ij}(t) = \eta(t)\,[x_i-\langle x_i\rangle][x_j-\langle x_j\rangle]\,g(\delta(t)) - \gamma W_{ij}(t)$$

with $\delta(t)$ as the phasic TD error and $r$0 scaled by tonic dopamine (Chary, 2012).

• Network State Regimes: Naive (baseline), Acute (post-drug), and Chronic (repeated drug) regimes with distinctive plasticity and homeostatic parameters.

Economic and Policy-Optimization Models

• Rational Addiction Utility: Cumulative utility maximization with addiction stock $r$1 and per-period utility $r$2. Training proceeds in two stages (mean-to-adapted), followed by group relative policy optimization with RL fine-tuning on real behavioral trajectories (Xu et al., 22 Jan 2026).

Population ODE and Network Models

• Compartmental Model: Five-class ODE system tracking susceptibles, users for each drug, and corresponding recovered populations. Reduced Lotka–Volterra competition equations determine phase boundaries for extinction, dominance, and coexistence (Colegate et al., 2022).
• Network Contagion Model: Four states per agent (user/addicted, non-user/potential), Fermi-type transitions modulated by peer fraction, adaptive rewiring, and stochastic physiological transitions (Lee et al., 26 Jun 2025).

3. Key Dynamical Results and Empirical Findings

Individual and Neural Level

• Wireheading and Short-Sightedness: RL-based AddictSim demonstrates that agents may fixate on high-instant reward actions, producing suboptimal long-term returns and persistent maladaptive behaviors when reward signals are improperly scaled (Behzadan et al., 2018).
• Pattern Storage and Stability: Dopamine-driven synaptic adaptation leads to severe reductions in attractor capacity, lowered pattern separability, and flattened energy landscapes in the chronically addicted network state, mirroring experimentally observed neural pathologies (Chary, 2012).

Digital Behavior and User Simulation

• Reproduction of Addiction Curves: Economic-theory-based AddictSim can replicate canonical “rise→peak→decline” utility patterns from real short-video usage, outperforming baseline RL and supervised strategies in both accuracy and behavioral realism (Xu et al., 22 Jan 2026).
• Impact of Recommendation Interventions: Diversity-aware ranking (e.g., CP-Fair, P-MMF) substantially decreases peak session duration and the addiction-reinforcement metric $r$3, suggesting effective mitigation of platform-induced addiction.

Population and Network Level

• Outcome Bifurcations: ODE AddictSim can produce extinction, winner-take-all, or coexistence based on analytically derived parameter regimes for peer influence, switching, recovery, and relapse (Colegate et al., 2022).
• Critical Transitions and Rewiring: Adaptive network AddictSim reveals phase transitions in addiction prevalence, with rewiring probability $r$4 governing a sharp shift from endemic to addiction-free states. Rewiring is more potent for prevalence reduction than direct modification of biological rates (Lee et al., 26 Jun 2025).

4. Implementation Protocols and Algorithmic Pseudocode

AddictSim implementations feature modular algorithms explicitly described in the literature:

• Tabular Q-learning (RL AddictSim):

$r$9 (Behzadan et al., 2018)

• Plastic Attractor Simulation (Hopfield AddictSim): Iterative network update and Hebbian weight adjustment using synchronous/asynchronous updates and three-factor plasticity, with periodic recall tests and entropy/discriminability/flattening metrics (Chary, 2012).
• Mean-to-Adapted RL with Group Relative Policy Optimization (Digital AddictSim): Two-stage model fitting (global, then personalized) followed by batch RL optimization with within-group advantage normalization and KL-penalty regularization (Xu et al., 22 Jan 2026).
• ODE Solution (Compartmental AddictSim): Direct integration via RK4 or adaptive ODE solver across mass-action transition terms, with parameter sweeps on peer influence, switching, and relapse rates (Colegate et al., 2022).
• Monte Carlo Network Simulation (Network AddictSim): Batch-synchronous behavioral, physiological, and rewiring updates over large random networks, tracking four-state distributions and cluster properties (Lee et al., 26 Jun 2025).

5. Parameter Sensitivity, Control, and Phase Transitions

AddictSim models admit rich parameter exploration:

• Thresholds for Addictive Dynamics: In RL, $r$5 leads to strong addiction; in ODEs, critical boundaries exist for extinction versus dominance, determined by system coefficients and their interactions (Behzadan et al., 2018, Colegate et al., 2022).
• Phase Transitions: Network AddictSim demonstrates that the rewiring parameter $r$6 acts as a control knob precipitating critical transitions; heatmaps in $r$7 or $r$8 space exhibit sharp phase boundaries and universal critical scaling exponents (Lee et al., 26 Jun 2025).
• Mitigation Strategies: Diversity-aware interventions, adaptive reward correction, and network-level social rewiring show disproportionate efficacy for reducing addiction prevalence and duration (Lee et al., 26 Jun 2025, Xu et al., 22 Jan 2026).

6. Broader Significance, Limitations, and Extensions

AddictSim models have catalyzed new approaches in AI safety, computational neuroscience, behavioral economics, public health, and digital policy:

• AI Safety and Reward Pathology: The demonstration that “wireheading” (reward hijacking) arises from purely parametric reward modifications in RL calls for reward-signal audit and robustification (Behzadan et al., 2018).
• Neurocognitive Parallels: Modeling pathologies as emergent phenomena in distributed neural systems supports trans-diagnostic analysis across psychiatric domains (Chary, 2012).
• Testing of Policy Interventions: Policy-level simulation, including replay of direct interventions (diversity, scheduling, content throttling, recovery support), is facilitated by scalable AddictSim methodology (Xu et al., 22 Jan 2026, Lee et al., 26 Jun 2025).
• Extensions: Emerging directions involve richer user heterogeneity, real-time intervention modeling, integration with causal inference, and cross-domain application to non-addiction-related behavioral contagion (Xu et al., 22 Jan 2026, Lee et al., 26 Jun 2025).

Significant findings establish that seemingly minor perturbations—such as reward scale shifts or modest empowerment of social rewiring—can dramatically polarize long-term addiction dynamics, suggesting avenues for targeted, high-leverage intervention across both artificial and real-world systems.