Ensemble Planning Agent Overview

• Ensemble Planning Agent is a compound system that combines multiple decision modules, leveraging LLM rankings, voting, and uncertainty-aware utilities for robust outputs.
• It systematically aggregates diverse candidate strategies in applications like automated ML pipelines, real-time game AI, and reinforcement learning to optimize performance.
• Empirical evaluations show that ensemble methods can yield substantial gains, such as a 76% improvement in game score and measurable accuracy boosts in data analysis tasks.

An Ensemble Planning Agent ($\mathcal{A}_\text{ens\_planner}$) is a compound agent that integrates multiple planning or decision modules, typically leveraging complementary capabilities or diverse candidate strategies, and synthesizes their recommendations or outputs via a principled arbitration or ensembling mechanism. The architectural instantiations of $\mathcal{A}_\text{ens\_planner}$ span LLM-driven automated data science, game AI, and uncertainty-aware reinforcement learning. Core to all instances is the systematic combination of outputs from individual modules—whether these are pipelines, value-function components, or role-specific agents—to robustly optimize for predictive or decision performance under uncertainty and complex task decompositions.

1. Formal Definitions and Core Formulations

Fundamental to the ensemble planning paradigm is a structured aggregation of diverse candidate plans or valuations. In LLM-based multi-agent data science, $\mathcal{A}_\text{ens\_planner}$ is defined as a function mapping candidate full-pipeline plans, data and task descriptions to a set of top $k$ selected pipelines:

$$\mathcal{A}_\text{ens\_planner} : (\mathcal{P}, D, T) \to \{P^*_1, \dotsc, P^*_k\}$$

where each $P^*_i$ is a tuple $$(p_\text{pre}, p_\text{feat}, p_\text{model}, p_\text{hp})$$ across the key stages of the ML pipeline and scored via an LLM-based ranking function $s(P_j; D, T)$ (Seo et al., 30 Mar 2025).

In real-time decision-making agents (e.g., Ms. Pac-Man), the agent is a function:

$\mathcal{A}_\text{ens\_planner}: S \to A$

with $S$ the state-space, $\mathcal{A}_\text{ens\_planner}$0 the action-space. Multiple components ("voices") each provide a real-valued rating $\mathcal{A}_\text{ens\_planner}$1 for each feasible action $\mathcal{A}_\text{ens\_planner}$2, aggregated into a composite score $\mathcal{A}_\text{ens\_planner}$3. The final action is selected as $\mathcal{A}_\text{ens\_planner}$4 (Rodgers et al., 2017).

For uncertainty-sensitive RL, $\mathcal{A}_\text{ens\_planner}$5 couples an ensemble of $\mathcal{A}_\text{ens\_planner}$6 model-free value functions $\mathcal{A}_\text{ens\_planner}$7 with a planning module (MCTS), integrating the ensemble’s uncertainty via risk-sensitive action selection rules (such as UCB or plurality voting) (Miłoś et al., 2019).

2. Principal Architectures and Module Interactions

In LLM-based data science automation, the architecture decomposes into four modular agents for data preprocessing, feature engineering, model selection, and hyperparameter tuning. Each module generates multiple candidates, whose Cartesian product forms the set $\mathcal{A}_\text{ens\_planner}$8 of full pipelines. The ensemble planner uses an LLM prompt (“SPIO-E Optimal Method Agent”) to rank these, returning the top $\mathcal{A}_\text{ens\_planner}$9 pipelines in a strict JSONL schema. For each selected $\mathcal{A}_\text{ens\_planner}$0, an independent code-generation agent $\mathcal{A}_\text{ens\_planner}$1 materializes executable solutions; predictions from all $\mathcal{A}_\text{ens\_planner}$2 models are ensembled via soft-voting (classification) or averaging (regression), assuming equal weights across all models (Seo et al., 30 Mar 2025).

In real-time agents, component voices reflecting different behavioral drives (e.g., short-term survival, pill collection, bonus item pursuit) compute action preferences in isolation. The Arbiter mechanism applies a weighted aggregation (Eqns. 1-2):

$\mathcal{A}_\text{ens\_planner}$3

The actions are then selected via $\mathcal{A}_\text{ens\_planner}$4, with tie-breaking as necessary. Modularity in feature observation and time-bounded deliberation maintain real-time tractability (Rodgers et al., 2017).

Ensemble RL planners utilize an ensemble of $\mathcal{A}_\text{ens\_planner}$5 value networks parameterized by $\mathcal{A}_\text{ens\_planner}$6. For each planning step, Q-value estimates $\mathcal{A}_\text{ens\_planner}$7 guide an MCTS-style planner. Uncertainty (ensemble variance) is mapped to exploration bonuses or risk-sensitive utilities $\mathcal{A}_\text{ens\_planner}$8, which bias both tree traversal and action selection (Miłoś et al., 2019).

3. Plan and Decision Aggregation Schemes

Aggregation in $\mathcal{A}_\text{ens\_planner}$9 is uniformly handled through either explicit ensemble scoring or voting mechanisms:

• LLM-based pipeline ranking (SPIO-E): The LLM implicitly orders complete pipelines in response to a prompt, returning a structurally-parseable list with top-k pipelines selected for ensembling; no explicit scoring is required, as relative ranking suffices (Seo et al., 30 Mar 2025).
• Action arbitration in real-time games: Each component agent (voice) produces normalized preferences; the composite rating is a weighted function emphasizing survival (Ghost Dodger) with other goal-driven voices multiplicatively modulating the rating. No single reactive voice can outright veto, but the survival voice can nullify actions that guarantee failure (Rodgers et al., 2017).
• Ensemble epistemics in RL: Each member of the value function ensemble provides Q-value estimates, whose distribution is used to compute mean and variance. The final planning policy uses uncertainty-aware utilities, e.g.

$k$0

and selects actions maximizing the ensemble-averaged $k$1 (Miłoś et al., 2019).

4. Algorithmic Workflow and Implementation Constraints

The LLM-based planner follows a structured workflow: candidate pool generation by module agents, enumeration and ranking (by LLM), code generation (per pipeline), parallel model execution, and final ensemble prediction. The default ensemble size is $k$2, with candidate pool size per module restricted to $k$3 ($k$4); experiments confirm $k$5 as optimal for the majority of tasks (Seo et al., 30 Mar 2025). LLM temperature is set at 0.5 to balance output coherence and creativity. Strict adherence to JSONL output is necessary for automated parsing.

In real-time agents, modular decomposition ensures agents operate on tractable input "slices" (Pill Muncher ignores ghosts, for instance), preserving sub-millisecond action computation. The deliberative component (Ghost Dodger) is strictly time-limited (e.g., 10 ms per move), with reactive components contributing via precomputed metrics. The Arbiter executes the selection rule for every feasible action, maintaining real-time feasibility (Rodgers et al., 2017).

In ensemble RL planners, the number of value-network ensemble members $k$6 is typically 3–20; masking during training allows each network to learn from a random subset of transitions. The MCTS planner traverses and expands states with values bootstrapped from the ensemble, using risk-sensitive action selection. Loop-avoidance penalties and prioritized replay buffers are incorporated (Miłoś et al., 2019).

5. Empirical Evaluation and Performance Characteristics

Domain	Agent Variant	Baseline Metric(s)	Ensemble Metric(s)	Improvement
Kaggle Classification (Seo et al., 30 Mar 2025)	SPIO-S	ACC=0.7927	SPIO-E top2 ACC=0.8062	+1.35%
Kaggle Regression (Seo et al., 30 Mar 2025)	SPIO-S	RMSE=0.1268	SPIO-E top2 RMSE=0.1219	–0.0049
OpenML Boston (Seo et al., 30 Mar 2025)	SPIO-S	MSE=9.1884	SPIO-E MSE=8.5220	Lower error
Ms. Pac-Man (Rodgers et al., 2017)	MCTS	Mean=58,058	Ensemble Mean=102,238	+76% mean score
Deep-sea RL (Miłoś et al., 2019)	No Ensemble	Failed (N>20)	Ensemble+UCB Solves N=30 grid	Speed-up, success
Montezuma’s Revenge (Miłoś et al., 2019)	No Ensemble	0/43 seeds solved	Ensemble+σ-bonus 30/37 seeds	+73%

Ensemble planning yields consistent, often substantial, improvements over single-path, single-model, or mean-only baselines. SPIO-E achieves up to ∼11% average gain in classification accuracy, with only $k$7 ensemble size. RL ensembles using uncertainty bonuses solve previously intractable environments and markedly speed up exploration. In real-time game AI, modular ensemble planners outperform both purely reactive and pure-planning agents in both survival and scoring benchmarks.

6. Theoretical and Practical Limitations

Key limitations are noted:

• Ranking fidelity: In LLM-driven planning, performance hinges on the quality of the LLM’s scoring/ranking. Mis-ranked pipelines degrade ensemble quality (Seo et al., 30 Mar 2025).
• Computational scaling: Larger ensemble sizes $k$8 increase both inference and code-execution costs. For real-world use, $k$9–4 is a practical upper limit (Seo et al., 30 Mar 2025).
• Equal weighting assumptions: Both across selected pipelines and within individual pipeline model-ensembles, uniform aggregation is assumed. Optimal weight learning or stacking is deferred to future research.
• Modular myopia: Real-time ensemble agents rely on feature-isolated modules, which may fail in cases where strong interdependencies exist between goals or input features (Rodgers et al., 2017).
• Uncertainty quantification: Ensemble RL methods approximate posterior uncertainty only empirically, and risk-sensitivity is tuned by hyperparameters (e.g., $$\mathcal{A}_\text{ens\_planner} : (\mathcal{P}, D, T) \to \{P^*_1, \dotsc, P^*_k\}$$0). Suboptimal tuning or small ensemble sizes may attenuate benefits (Miłoś et al., 2019).

7. Connections and Applications Across Domains

Ensemble Planning Agents encapsulate a broad family of AI architectures:

• In data science automation, $$\mathcal{A}_\text{ens\_planner} : (\mathcal{P}, D, T) \to \{P^*_1, \dotsc, P^*_k\}$$1 orchestrates entire ML production pipelines, integrating LLMs as scoring/ranking arbiters bridging otherwise combinatorial search spaces (Seo et al., 30 Mar 2025).
• In game AI, modularity enables expert behavior decomposition and time-bounded action selection, translating abstract goals (e.g., survival, scoring maximization) into composite ratings, thus exploiting both reactive and deliberative methods (Rodgers et al., 2017).
• In RL, ensemble planning fuses epistemic uncertainty from deep neural networks with local search planners, resulting in more efficient strategic exploration and robust value estimation in sparse-reward or complex environments (Miłoś et al., 2019).

A plausible implication is that ensemble planning agents provide a unifying abstraction for heterogeneous decision systems where parallel candidate generation, uncertainty aggregation, and arbitration are essential to task performance, robustness, or adaptability.