ALFWorld & Tau2-Bench: Evaluating LLM Agents

• ALFWorld and Tau2-Bench are benchmark suites that rigorously evaluate language-driven agents’ interactive reasoning, planning, and coordination in complex environments.
• They combine text-based simulation with embodied feedback and dual-control frameworks to assess performance using metrics like success and pass¹ rates.
• Innovative training paradigms, including imitation learning and reinforcement world model learning, drive policy optimization and enable effective zero-shot transfer.

ALFWorld and Tau2-Bench ($\tau^2$-Bench) are benchmark suites designed to rigorously evaluate the interactive reasoning, planning, and coordination capabilities of language-driven agents in complex environments. ALFWorld focuses on aligning linguistic abstraction with embodied task execution in simulated households, while $\tau^2$-Bench targets dual-control, tool-mediated conversational domains that require agent-user collaboration. Both environments are central to the assessment of LLM agents, particularly in their capacity to simulate, predict, and act under uncertainty, and have recently been leveraged in the development of world-model learning for LLM-based policies.

1. Architectural Foundations

ALFWorld integrates two core simulation engines: TextWorld, an interactive, text-only environment based on PDDL, and ALFRED, an embodied THOR-based simulation providing visual and physics-based feedback (Shridhar et al., 2020). Both environments share a PDDL-structured internal symbolic state, enabling seamless mapping between high-level abstract plans and embodied actions. TextWorld produces template-based textual observations and executes high-level actions (e.g., goto, take, put, etc.), whereas ALFRED provides RGB-D frames and supports low-level motor commands (e.g., MoveAhead, Pickup). The system at each step uses a state estimator (e.g., Mask R-CNN for object detection from images) to bridge the modalities, followed by a modular controller for navigation (A* search on a grid) and manipulation (invoking simulator APIs).

In contrast, $\tau^2$-Bench is formalized as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). Each interaction is characterized by a tuple: $$\mathcal{M} = (I, S, \{A_i\}_{i \in I}, \{O_i\}_{i \in I}, T, \Omega, R, \rho_0)$$ where $I = \{\text{Agent}, \text{User}\}$, $S$ is the global state, $A_i$ and $O_i$ denote each player's actions and observations, $T$ and $\Omega$ represent transitions and observation kernels, $R$ the reward, and $\rho_0$ the initial state. Both agent and user possess complementary toolkits and interact via tool calls (API-style function calls) and chat-based communication. The environment supports rich, dynamic control flow, designed to emulate domains such as technical support in Telecom, Retail, and Airline settings (Barres et al., 9 Jun 2025).

2. Task Structure, Observation/Action Spaces, and Reward Models

ALFWorld tasks are composed of household instructions (e.g., "put a knife in sidetable") with success determined by satisfaction of all goal predicates after a sequence of up to 30 actions. The observation space in TextWorld consists of token sequences ($o_t \in V^*$), whereas in ALFRED the agent receives high-dimensional image tensors. Actions are either high-level (text) or low-level (embodied), with rewards structured as binary (goal completion) or fractional (number of goal conditions satisfied over total goals).

$\tau^2$-Bench tasks are built compositionally from atomic subtasks, each defined as a set of function calls for initialization, solution, and assertion, grouped for balanced coverage. Both agent and user act in tandem, using a diverse set of read and write tools (see summary table below), with state represented as a combination of database world state and interaction history. The user simulator adheres to a finite-state machine policy, ensuring deterministic, on-plan transitions and significantly reduced off-policy deviations. Rewards are typically based on terminal task resolution; intermediate metrics (e.g., pass$^1$ rates over multiple domains, error rates in user simulation) are reported.

Player	Read Tools	Write Tools
Agent	get_customer_by_id, get_network_status, ...	enable_roaming, change_plan, ...
User	get_network_status, get_signal_strength, ...	toggle_airplane_mode, reseat_sim_card, ...

3. Model Architectures and Training Paradigms

ALFWorld's pipeline centers on the BUTLER agent—comprising a DA-Seq2Seq text agent with transformer-based encoder-decoder, BERT-initialized embeddings, GRU memory aggregator, and pointer-softmax decoder. Training is executed in two phases. First, imitation learning (IL) via DAgger is employed in TextWorld with expert rollouts from a PDDL-based planner. The cross-entropy loss over ground-truth high-level actions ensures robust policy acquisition. Second, zero-shot transfer is evaluated in ALFRED using a frozen policy; optionally, reinforcement learning (RL) can refine policies directly on embodied data.

$\tau^2$-Bench benchmarks LLM-based agents capable of both tool invocation and natural language interaction, with evaluations in both zero/few-shot and fine-tuned settings. The environment directly assesses an agent's capacity for coordinated problem-solving, context-sensitive communication, and precise tool usage in dialog.

4. Evaluation Protocols and Results

ALFWorld employs multiple train/test splits (e.g., red$_1$, green$_1$, deepmagenta$_1$ for seen/unseen room configurations), with performance measured in terms of task success rate, partial goal completion, and throughput (episodes/sec). Empirical findings show that TextWorld-trained BUTLER agents achieve approximately 40% overall success on green$_1$ (seen) and 35% on deepmagenta$_1$ (unseen), in contrast to 6–15% success reported for behavioral cloning baselines (Shridhar et al., 2020). TW-only training is observed to be 7× faster than embodied or hybrid strategies and generalizes more effectively.

$\tau^2$-Bench reports pass$^1$ rates (fraction of tasks solved per run) across domains. In the Telecom domain, for example, GPT-4 achieves 34% pass$^1$ in "Base" (dual-control) mode, while "Solo" mode (agent controls all tools) yields 54%. The ablation from "Solo" to "Base" quantifies the significant performance cost imposed by communication and coordination requirements. User simulator audits further demonstrate substantial reductions in critical error rates (e.g., Telecom: 6%) compared to prior domains (Barres et al., 9 Jun 2025).

5. Application to World-Model Learning and Policy Optimization

Both ALFWorld and $\tau^2$-Bench are adopted as primary benchmarks for evaluating Reinforcement World Model Learning (RWML) in LLM-based agents (Yu et al., 5 Feb 2026). RWML introduces a self-supervised mid-training phase that conditions policies to predict simulated next states in an embedding space ($\phi$), with sim-to-real gap rewards computed as

$$r_{\text{gap}}(s_t, a_t) = -\Vert \phi(s_{t+1}) - \phi(\hat{s}_{t+1}) \Vert_2^2$$

This mechanism fosters consistent, semantically aligned world-modeling in the agent's behavior, in contrast to token-level prediction approaches. On ALFWorld, RWML alone increases success rates from 13.0% (base LLM) to 32.6%, with further gains to 87.9% when combined with direct task-reward RL. For $\tau^2$-Bench, RWML elevates base performance from 31.9% to 38.8% (+6.9), and to 43.7% when combined with RL, matching or exceeding imitation learning on expert data.

Ablations confirm that embedding-based gap rewards are substantially more robust than LLM-as-judge approaches (which are susceptible to reward hacking), and subsampling "hard" transitions is critical to model fidelity. RWML also demonstrates superior retention against catastrophic forgetting and induces efficient parameter adaptation, integrating seamlessly with downstream RL.

6. Comparative Scope, Limitations, and Future Directions

ALFWorld is characterized by single-agent, embodied spatial planning tasks wherein language plans must be concretely executed amid visual feedback and physical constraints. Agents are evaluated predominantly for policy abstraction, symbolic reasoning, and task efficiency. $\tau^2$-Bench, by contrast, introduces a collaborative paradigm with both agent and user empowered to manipulate shared stateful environments, thereby isolating reasoning, communication, and coordination abilities. Tasks require negotiation of tool use, error handling, and goal-oriented dialog.

Key limitations in ALFWorld include domain transfer gaps due to size and detection errors and reliance on template-based state estimation. In $\tau^2$-Bench, the primary challenge is agent guidance of users within the dual-control interaction protocol, with significant degradation observed in coordinated settings.

Anticipated directions—grounded in ALFWorld roadmaps and the RWML agenda—include learned, end-to-end state estimators, perceptual navigators, generative world models in language space, synthetic-to-real transfer, and more sophisticated evaluation in dual-agent environments. Both suites serve as critical testbeds for iterative advancements in embodied AI and conversational agent architectures.