Agent-Based Baselines Overview

• Agent-based baselines are explicitly defined reference methods using interactive agents to quantify and calibrate progress across domains such as reinforcement learning, social simulation, and generative modeling.
• They implement canonical models like Sugarscape, Minority/Majority Games, and traffic cellular automata to benchmark new algorithms and reveal limitations of existing approaches.
• They extend into multi-agent RL, language-based architectures, and generative population modeling, providing systematic performance analyses and variance reduction techniques.

Agent-based baselines are rigorously defined reference methods that use explicitly instantiated, environment-interacting agents to quantify or calibrate progress in domains spanning multi-agent reinforcement learning, language modeling, social simulation, offline RL, and generative population modeling. These baselines are foundational to experimental methodology in agent-based science: for both benchmarking new algorithms and for revealing the limitations of existing approaches. Distinct from model-free statistical or random baselines, agent-based baselines are constructed to embody canonical rules, network structures, or modular decision architectures that capture (but intentionally limit) essential agent functionality. Their formulation, analysis, and extension are central themes in recent literature across multi-agent systems, embodied AI, and agent-based simulation frameworks.

1. Canonical Formalisms and Representative Models

Agent-based baselines are rooted in explicit agent-environment interaction protocols. In social physics and simulation, six classical formalisms dominate:

1. Sugarscape (Epstein & Axtell): Resource-accumulating, spatially mobile agents with local vision, metabolic consumption, and emergent wealth distribution dynamics. Used as a baseline for inequality and behavioral policy benchmarking.
2. Artificial Markets: Fundamentalist–chartist heterogeneous trading models (Santa Fe, Lux–Marchesi), capturing market stylized facts under mechanistic agent behavior.
3. Minority/Majority Games: Adaptive agents with finite strategy pools, iteratively optimizing virtual scores, serve as benchmarks for decentralized coordination and unpredictability.
4. Nagel–Schreckenberg Traffic CA: Agents as discrete-speed vehicles with headway-dependent local update rules, forming the standard reference for traffic-flow and jam phenomena.
5. Helbing Social Force Model: Pedestrian agents subject to Newtonian dynamics with social repulsion and self-propulsion, establishing baseline for evacuation and crowd models.
6. Voter Model: Binary-opinion update on arbitrary graph topologies, foundational for consensus and polarization studies.

Implementation in NetLogo, Repast, MASON, Swarm, or Mesa is standard for cross-comparable baselining (Quang et al., 2018). Each model is parameterized according to field-wide conventions (e.g., vision, metabolism, grid size in Sugarscape; agent composition and leverage in markets).

2. Agent Baselines in Multi-Agent Reinforcement Learning

In multi-agent RL, agent-based baselines delineate the minimum functional and statistical capabilities required for progress assessment:

• Behavior Cloning (BC): Supervised imitation of behavior policy; performance lower bound in offline regimes (Formanek et al., 2023).
• QMIX family: Centralised Training Decentralised Execution (CTDE) with monotonic mixing of per-agent Q-functions; QMIX [Rashid et al.], QMIX+BCQ (action masking by behavior density), and QMIX+CQL (conservative penalty for OOD actions), all serving as standard references for the value-decomposition approach (Formanek et al., 2023).
• MAICQ: Implicit constraint Q-learning that advantage-weights TD errors for in-distribution actions, representing the current robust baseline for discrete-action multi-agent offline RL.
• Independent/Decentralised Actor–Critic Variants: Independent TD3 and its offline-regularized forms (TD3+BC, TD3+CQL, OMAR) for continuous-action, multi-agent regimes.

These baselines are evaluated on structured offline datasets (Good/Medium/Poor/Replay), with empirical outcomes tabulated for environments such as SMAC (StarCraft), Multi-Agent MuJoCo, PettingZoo, Flatland, and others (Formanek et al., 2023). Comparative performance consistently highlights the inability of vanilla Q-learning (QMIX, ITD3) to generalize in offline/poor data settings, with conservative (CQL, BCQ) or implicit constraint methods (MAICQ) providing strong robustness and policy improvement.

3. Baselines for Variance Reduction in Multi-Agent Policy Gradients

Variance reduction for multi-agent policy gradient (MAPG) estimators is a central consideration. The main agent-based baselines include:

• Vanilla MAPG: Uses no baseline ($b=0$); excess variance scales with agent-advantage noise and agent count.
• COMA Baseline: Employs a counterfactual baseline ($b=Q^{−i}(s,a^{−i})$), reducing variance to the per-agent noise (Kuba et al., 2021).
• Optimal Baseline (OB): Closed-form, minimal variance baseline for the CTDE estimator. For agent $i$, the OB is:

$$b^{\mathrm{optimal}}(s,a^{−i}) = \frac{E_{a^i\sim\pi^i}[Q(s,a^{−i},a^i)\|\nabla_{\theta^i}\log \pi^i(a^i|s)\|^2]}{E_{a^i\sim\pi^i}[\|\nabla_{\theta^i}\log \pi^i(a^i|s)\|^2]}.$$

Surrogate OBs are defined for deep networks using gradients w.r.t. the “pre-softmax” vector.

• Empirical Impact: Integrating the OB into multi-agent PPO and COMA results in faster convergence, higher win rates, and reduced gradient variance on SMAC and Multi-Agent MuJoCo benchmarks.

Quantitative analysis of excess variance and demonstrated improvements through incorporation of the OB or its surrogates set the technical baseline for variance-controlled MAPG in MARL (Kuba et al., 2021).

4. Baseline Agent Methods in Language-based and Embodied Contexts

• Blindfold (question-only) Baselines in EmbodiedQA: Ignoring visual inputs and navigation, BoW or LSTM-only agents are trained to answer solely from textual questions. Despite being degenerate with respect to embodiment, these baselines achieve or surpass state-of-the-art Navigation+VQA agents on all but the shortest spawn distances (Anand et al., 2018). Their performance reveals severe dataset bias; consequently, comparison to blindfold baselines is now a mandatory experimental control.
• Minimal Text-Game Baselines: The SSAQN (Siamese State-Action Q-Network) maps both game states and possible actions into embeddings, then selects via cosine similarity. Despite architectural simplicity, this baseline achieves optimal or near-optimal mastery in deterministic interactive fiction games under single-game and transfer learning regimes, but fails to generalize zero-shot (Zelinka, 2018).

In language-agent architectures for complex planning, canonical baselines are:

• ReAct: Single-agent interleaved reason+act loop. No explicit decomposition or agent spawning.
• Plan-and-Solve (PnS): Single agent, plan generation followed by stepwise execution.
• Plan-and-Execute (PnE): Main agent decomposes, executor agent sequentially solves fixed subtasks.
• ADaPT: Recursive task decomposition with self-invocation but static memory. Each of these is systematically evaluated on multi-stage benchmarks such as ItineraryBench, with TDAG (dynamic decomposition plus agent generation) outperforming all (Wang et al., 2024).

5. Generative Population Baselines: Static Retrieval and LLM-Generation

In high-fidelity synthetic agent initialization, baselines fall into two classes (Chen et al., 9 Jan 2026):

• Static Data-Based Retrieval: Uniform random sampling or topic-retrieval from a real-user persona pool (World Values Survey), with performance evaluated in terms of macro-level distributional alignment (JSD, KL, diversity) and micro-level coherence (judge-based scales).
• LLM-Based Generation Baselines: Prompting large models to generate personas conditional on topic, typically assuming factorized marginals; HAG-Flat ablates joint path dependencies. While LLM generation is flexible and produces plausible individuals, both these baselines are substantially misaligned with macro joint distributions and present high diversity error. Empirical quantification across domains (Bluesky, Amazon, IMDB) demonstrates that macro-micro consistency is not attainable by these baselines, motivating hierarchical approaches that enforce conditional dependencies.

6. Modular and Ensemble Agent-Based Baselines in Software Engineering

In repository-level software issue resolution, agent-based baselines have evolved to modular, compositional agent ensembles:

• Trae Agent: Pipeline of generation, pruning, and selection agents, each integrating specialized tool APIs for code editing, test execution, and repository-level reasoning. Ensemble size and LLM mixture tuning serve as test-time scaling axes; majority voting and hierarchical pruning stabilize outputs.
• Four main baseline classes compared: Augment, DeiBase (with/without pruning), Adversary, and Average; Trae Agent establishes state-of-the-art Pass@1 in SWE-bench, outperforming all previous ensemble prompting baselines (Team et al., 31 Jul 2025).

Reusable architectural modularity—distinct generation, pruning, and selection phases—characterizes current best-practice agent-based baselines in automated reasoning and program synthesis.

7. Best Practices, Limitations, and Experimental Design Considerations

The selection and construction of agent-based baselines is subject to several operational guidelines:

• Orient toward the simplest agentified process that is both nontrivial and interpretable in the given domain. For MARL this often means the minimal policy-gradient or Q-learning agent; for social simulations, a well-characterized dynamical rule set; for LLM-based planning, a generic loop agent.
• Explicitly characterize data or environment biases that allow trivial baselines (e.g., question-answer overlap, distributional skew) to succeed; always report agent-based and non-agent statistical baselines together (Anand et al., 2018).
• When modeling joint attribute spaces, distinguish between macro (distributional) and micro (individual) consistency—neither static retrieval nor LLM-based generation is sufficient for both (Chen et al., 9 Jan 2026).
• Architecture, memory structure, and decomposition heuristics should be reported for all agent-based baselines, including ablations (e.g., agent generation off, static vs. dynamic decomposition).
• Baseline performance should be referenced empirically against standard datasets and parameterizations; failure modes must be analyzed with respect to variance, generalization, and policy realism.

Agent-based baselines remain essential for scientific rigor, enabling reproducible, interpretable, and systematically improvisable reference points for new agent architectures, learning rules, and simulation methods.