DotaBench AI in Dota 2: Architecture & Applications

• DotaBench is a benchmark framework that enables evaluation of AI systems for Dota 2 by integrating context-aware decision support modules.
• It combines Transformer-based encoders, retrieval-oriented modules, and real-time inference pipelines to deliver interpretable recommendations for in-game items and hero drafts.
• Leveraging insights from Dota 2 Bot Competitions and personalized draft models, DotaBench demonstrates enhanced performance in both item recommendation accuracy and draft optimization.

DotaGPT is an umbrella term for AI systems designed to provide context-aware, interpretable decision support in Dota 2, a leading Multiplayer Online Battle Arena (MOBA) game. It includes GPT-style LLMs enhanced with MOBA-specific modules for in-game item recommendation, draft advisory, and agent control. State-of-the-art implementations integrate Transformer-based context encoders, retrieval-augmented modules, and real-time inference pipelines. DotaGPT systems have been prototyped as both interactive teammate assistants and autonomous agents, leveraging research from the Dota 2 Bot Competition, interpretable contextual item recommendation, and personalized draft optimization (Font et al., 2021, Villa et al., 2020, Lee et al., 2022).

1. Architectural Foundations and Communication Protocols

The Dota 2 AI framework—critical infrastructure for DotaGPT experiments—operates as a custom LUA script that hooks into the standard Dota 2 game client and exposes bot-relevant game state over HTTP/JSON. At each event (game tick, hero selection, chat event), entity-structured state vectors are serialized and sent to an external controller, which replies synchronously with action commands. The high-level communication protocol is as follows (Font et al., 2021):

Component	Role in Protocol	Data format/example
Dota 2 Engine (LUA mod)	Serializes events/state, sends HTTP POST	Entities [{id,class,...}], hero select
External Controller (bot/DotaGPT)	Parses state, runs policy/LLM, returns command	{ "command":"ATTACK", "target":760 }
HTTP Endpoints	/select, /update, /chat	As above

Observation space is defined as $o_t = \phi(s_t) \in \mathbb{R}^d$, representing local, featurized game state for every tracked entity. The action space $A$ includes discrete and continuous primitives: NOOP, MOVE(𝑥,𝑦,𝑧), ATTACK(target_id), CAST(ability_id, target_id), and various item operations.

A practical Python pseudocode sketch is provided in (Font et al., 2021) (see dataset for full code).

2. Contextual Item Recommendation: TTIR as DotaGPT Module

The Team-aware Transformer Item Recommendation (TTIR) model forms the canonical architecture for DotaGPT’s item recommendation (Villa et al., 2020). Each Dota match with $N=10$ participants (5 per team) is encoded as a set of context vectors. Embedding composition per participant $i$:

$$x_i = E_{champ}(h_i) + E_{role}(r_i) + E_{team}(t_i) + E_{pos}(i)$$

These are stacked into $X \in \mathbb{R}^{N \times d_{model}}$ and processed through $L$ Transformer encoder layers, producing contextualized embeddings $C = H^{(L)}$. Each player’s output vector yields item logits via:

$$o_i = W_{rec} c_i + b_{rec}, \quad s_{i,j} = \sigma(o_{i,j})$$

where $M$ is the number of Dota items ($A$0). Recommendation for each hero is given by top-K scores (typically $A$1).

Training objective is summed binary cross-entropy over batch, heroes, and items:

$A$2

Attention heads deliver interpretability by exposing an $A$3 matrix $A$4 highlighting which heroes influence each recommendation. This supports player- and item-centric heatmaps for post-hoc recommendation explanation.

TTIR, ported to DotaGPT, outperforms D-Tree, logistic regression, and CNN/ANN baselines in LoL (see Table 2 in (Villa et al., 2020)). A plausible implication is that TTIR’s context-aware attention mechanisms generalize to Dota’s more complex role/item system, especially when transfer learning is applied.

3. Draft Optimization and Personalized Hero Suggestions: DraftRec Integration

DraftRec extends DotaGPT’s scope to draft phase hero optimization by modeling both player preferences and team context (Lee et al., 2022). The architecture parses each draft as a sequence of $A$5 turns, incorporating:

• Player histories $A$6, per-player and per-role embeddings, in-game feature encodings.
• Hierarchical two-level Transformer: Player network (history encoder), Match network (team/pickboard context).

For a given draft state, DraftRec predicts pick distributions $A$7 and match outcome logits $A$8 via transformer heads. Objective is joint cross-entropy over all turns:

$A$9

DraftRec’s attention matrix quantifies synergy/counter-synergy and allows recommendations to be visualized (e.g., Jungle–Mid, ADC–Support). For DotaGPT, this module can be coupled to a GPT backbone, yielding explanations such as "Pangolier synergy with your Shadow Demon is high," with top-K picks filtered by pick distribution thresholds.

Key Dota2-specific performance figures (Lee et al., 2022):

Metric	DraftRec	Best Baseline (SASRec/LR)
HR@5 (champ rec.)	0.1492	0.1477
NG@10	0.1496	0.1296
ACC (match outcome)	0.5755	0.5751/0.6126
MAE	0.4782	0.4842/0.4642

Removing player history degrades HR@10 by ≈36% on LoL, illustrating the net benefit of personalized modeling.

4. Real-Time Inference and System Integration

DotaGPT item and draft modules are implemented for efficient live inference. TTIR’s architecture processes sequences $N=10$0 through a 1- or 2-layer Transformer, giving per-hero recommendations with latency <10 ms (GPU) or <50 ms (CPU) (Villa et al., 2020). Larger batch serving and online web APIs are supported.

Practical design notes for DotaGPT variants:

• Replace LoL hero/item vocabularies with Dota-specific sets ($N=10$1, $N=10$2).
• Embed additional match-state features (gold, kill stats) directly into token streams via small feed-forward layers.
• Incorporate ban-phase and patch-version embeddings for draft-aware models.
• Use τ-threshold on pick distributions to avoid out-of-distribution hero recommendations.
• Continuous patch retraining ensures adaptation to evolving meta/game balance.
• Offline evaluation via HR@k, NG@k, match outcome prediction accuracy; A/B testing in live games for real-world impact.

5. Interpretable Decision Rationales via Attention Visualization

DotaGPT’s interpretability is grounded in the attention weights of its Transformer blocks. For item recommendation, attention matrices $N=10$3 aggregate influence scores between participants, allowing heatmap visualizations highlighting which heroes most impacted a particular build (Villa et al., 2020). In DraftRec, attention analysis reveals established synergy patterns (e.g., Jungle–Mid) and can be reported in natural language alongside recommendations.

User interface schemes include portrait rows, attention maps, item builds, and tooltips with quantifiable influence scores (“Hero₁ paid 24% attention to Hero₇ when picking his boots”). Preliminary user surveys on similar systems indicated >80% approval for plausibility and win likelihood, especially among high-rank players.

6. Limitations and Future Directions

Several practical and research challenges constrain current DotaGPT systems:

• The Dota 2 Bot Competition framework allows only a single external controller per match; multi-agent (5v5) support and multi-controller synchronization remain open tasks (Font et al., 2021).
• No visual input (raw pixel data) is currently available for state featurization; all input is structured.
• Engineering barrier to mod setup can restrict participation; public documentation and sample code partially address this (Font et al., 2021).
• No per-tick time limit (v1) allows bots to stall the game.
• Extending to true stateful, dialogue-centric draft assistants requires jointly fine-tuned generative LLMs and supervised attention-grounded draft modules (Lee et al., 2022).

Future improvements proposed in primary sources include:

• GPU-accelerated visual pipelines for vision-based controllers.
• Leaderboards and automated matchmaking for benchmarking.
• Ban-round modeling, patch-version and MMR conditioning, and sequential item/draft recommendation.
• Safety via offline RL techniques (e.g., conservative Q-learning) and filtering to avoid recommending atypical or detrimental picks.

A plausible implication is that DotaGPT’s modular architecture—combining structured retrieval, context-encoded attention, and interactive language modeling—positions it as a general-purpose template for interpretable and adaptive AI teammates in real-time, multi-agent strategic environments.