ReSpAct: LLM Agent Framework

• ReSpAct is a large language model-based agent framework that integrates reasoning, dialogue, and environment actions for improved task performance.
• The framework uses a unified LLM policy to alternate between internal thought, proactive user interaction, and direct action execution.
• Empirical evaluations reveal that dynamic user engagement significantly boosts success rates in environments like ALFWorld, WebShop, and MultiWOZ.

ReSpAct is a LLM-based agent framework that harmonizes reasoning, conversational dialogue, and environment interaction for task-solving agents in fully conversational, user-in-the-loop scenarios. It extends prior reasoning-first approaches by allowing an LLM to fluidly alternate between internal thought, external actions, and free-form dialogue, dynamically deciding when and how to engage the user for clarification, confirmation, or updates. This schema-free, alternation-centric structure aims to boost both user alignment and downstream task performance across diverse real and simulated environments (Dongre et al., 2024).

1. System Architecture and Action Loop

ReSpAct partitions the action space at each timestep into three disjoint subspaces:

• Environment actions $\mathcal{A}$: Concrete step executions within the environment, e.g., “go to cabinet,” “take pan.”
• Internal reasoning actions $\mathcal{L}$: LLM-generated “Thought:” traces not directly observable by the environment or user.
• Dialogue actions $\mathcal{U} \subset \mathcal{L}$: “Speak:” outputs directed to the user, encompassing clarifications, confirmations, and status updates.

The recurrent context $c_t$ at step $t$ is a tuple comprising all prior environment observations $o_1...o_t$, actions $a_1...a_{t-1}$, reasoning traces $l_1...l_{t-1}$, and user replies $r_1...r_{k-1}$. A single LLM-based policy $$\pi: \mathcal{C} \rightarrow \mathcal{A} \cup \mathcal{L}$$ maps each context to its next atomic maneuver—selecting among “Think,” “Speak,” or “Act.”

This yields the following loop:

$\mathcal{L}$9

Termination occurs either via a “Done” act from the LLM or domain-specific goal completion (Dongre et al., 2024).

2. Formalization and Decision Process

In ReSpAct, for a set of observations $\mathcal{L}$0, action set $\mathcal{L}$1, language actions $\mathcal{L}$2, and dialogue actions $\mathcal{L}$3, the context at step $\mathcal{L}$4 is formally defined as

$\mathcal{L}$5

The policy chooses

$\mathcal{L}$6

where $\mathcal{L}$7 is parameterized by the LLM's weights. Integration of user replies dynamically shapes future context, allowing the model to course-correct or solicit missing information as needed. The selection mechanism enables a non-Markovian, context-rich decision loop, avoiding fixed dialogue schemas and hand-crafted flow charts (Dongre et al., 2024).

3. Relationship to ReAct and Conceptual Advances

ReAct enables step-wise alternation between “Think” and “Act” for LLM agents by expanding the action space to include explicit reasoning traces, offering interpretable “chain-of-thought” style outputs without agent-user dialogue by default. ReSpAct introduces a further subdivision: dialogue actions, empowering the agent to proactively engage the user using unconstrained, contextually decided utterances.

Unlike schema-driven dialogue policies or template-based clarifications, ReSpAct’s key contribution is its policy’s in-context discretion over:

• When to ask clarifying questions (e.g., when object locations are ambiguous)
• When to confirm inferences or choices before proceeding (e.g., confirming booking details)
• When to provide proactive status updates or explain subtask failures

Dialogic involvement is triggered strictly through learned prompting and in-context exemplars, not via pre-defined rule sets (Dongre et al., 2024). This supports both granular user alignment and robust failure recovery, especially in tasks where information is distributed between environment and user.

4. Evaluation Metrics, Benchmarks, and Ablation Studies

Empirical evaluation spans three distinct environments:

• ALFWorld: Text-based household tasks requiring sequential tool use and navigation.
• WebShop: Online shopping agent tasked with attribute-driven product selection.
• MultiWOZ: Multi-domain task-oriented dialogue benchmark.

Primary metrics include:

• ALFWorld: Success rate (tasks completed)
• WebShop: Attribute-coverage, success rate (completed goals/products)
• MultiWOZ: Inform and Success scores (valid entity return, attribute fulfillment)

Environment	Baseline	ReSpAct Absolute Gain
ALFWorld	79.4% → 85.3%	+6.0%
WebShop	8% → 12% SR	+4%
MultiWOZ	Inform 66.7→72.2	+5.5

Ablations reveal that replacing dialogue with dense inner monologue (“thought-only”) degrades performance (to ∼48.5% in ALFWorld). Schema-guidance for dialogue yields marginally increased turn counts but not success rate improvement, indicating the sufficiency of schema-free alternation. User-simulator fidelity is paramount: performance drops to 32.1% with unhelpful users versus 85.3% for oracle-like ones (Dongre et al., 2024).

5. Mechanisms for Dynamic User–Agent Collaboration

ReSpAct operationalizes mixed-initiative dialogue, alternately THINK→SPEAK→(USER REPLY)→THINK→ACT as required by context. This enables:

• Real-time clarifications on ambiguous subgoals (“Where should I look for the pan first?”)
• Slot-filling interactions for API or database tasks (“How many people for this restaurant booking?”)
• Confirmation of inferred or defaulted values (“Just to confirm, you want a taxi at 00:30 AM, correct?”)
• Fine-grained disambiguation in multi-step manipulation tasks (“Which two credit cards to put in the dresser?”)
• Adaptive plan revision in response to changing goals or environment state (“Couldn’t find MN4 shade under $40. Prefer color match or price?”)

This approach attains a human-like, steady convergence to correct solutions, reducing spurious failures and unnecessary actions (Dongre et al., 2024).

6. Implementation Guidelines and Domain Adaptation

Key implementation stages include:

• Prompt Engineering: Define “Think:”, “Speak:”, and “Act:” tokens. Few-shot exemplars must intermingle all action types, including both error and recovery cases, to elucidate when user input is valuable.
• User Simulation: Construct rule-based or LLM-based user agents for dialogue, allowing controlled experimentation with varying oracle and adversarial user behaviors.
• Domain Generalization: Extend the action set $\mathcal{L}$8 to domain-specific primitives; tailor user interaction style from search clarifications to rich slot-filling as needed. Optionally, layer a state-tracking interface for multi-step transactions.

Table: Summary of Implementation Recommendations

Component	Guideline	Comment
Prompt Design	Use explicit action tokens, show error/failure	Aids model generalization across domains
User Simulation	Oracle, perturbed, random users	Stress-tests robustness; ablation essential
Speak Frequency	Tune with in-context examples	Sparse, targeted dialogue yields best tradeoff
Safety	Human-in-the-loop for critical actions	Ensures reliable, auditable deployment

Ablation insights advise restraint in dialogue invocation—overly frequent schemas result in verbosity without commensurate performance gains. Human-in-the-loop settings are recommended for high-stakes decisions, with stringent logging of all internal thought and dialogue states for transparency and audit.

7. Implications, Limitations, and Prospective Directions

ReSpAct demonstrates that integrating schema-neutral, LLM-governed dialogue in agent policy loops yields measurable improvements in multi-modal task-solving success, transparency of decision process, and robustness to real-time user participation. Key limitations emerge from dependence on user reply informativeness, the open-endedness of in-context prompting, and the risk of verbosity or confirmation redundancy.

A plausible implication is that, as LLMs grow more capable and contextually aware, policy spaces unifying reasoning, communication, and environment actions may become the prevailing paradigm for interactive task agents. Further research could explore curriculum-driven exemplar selection, automatic detection of confidence thresholds for dialogue invocation, and application in safety-critical or high-latency domains (Dongre et al., 2024).