PreAct Framework: Multi-Step LLM Agents

• PreAct framework is a multi-step planning system for LLM agents that decouples planning from acting to ensure global coherence over long-horizon tasks.
• It employs an iterative process where an initial detailed plan is dynamically refined using tool feedback and predictive reasoning for enhanced adaptability.
• Its modular design—integrating planner, executor, refiner, and finalizer—yields significant performance gains in action recall and goal completion compared to ReAct.

The Pre-Act (PreAct) framework designates a family of LLM-centric agent architectures that augment typical reasoning–action cycles with explicit, plan-first multi-step reasoning and dynamic re-planning. Pre-Act enables LLM agents to maintain global coherence over long-horizon tasks by first drafting an explicit, stepwise execution plan—each step paired with detailed reasoning—then iteratively refining this plan as observations from tool calls or environment feedback accumulate. The paradigm originated as a response to limitations in the ReAct (Reasoning + Action) approach, where the agent interleaves a single Thought and Action per turn, which can fail on complex, multi-step or out-of-domain scenarios. Subsequent Pre-Act variants add prediction and strategic handling, supporting more robust and adaptive agentic behavior (Rawat et al., 15 May 2025, Fu et al., 2024).

1. Motivation and Distinguishing Features

Pre-Act extends ReAct by separating planning and acting: rather than reasoning only about the immediate next step, the agent first generates a structured multi-step execution plan based on the input request, with each step ($s_i$) comprising both "thought" ($\phi_i$) and action ($a_i$). This plan is not static; after each step is executed and tool outputs or feedback ($o_i$) are received, the agent revisits and refines the remainder of the plan in light of new evidence, updating both future actions and their supporting rationale.

Key features:

• Global multi-step coherence: The upfront plan encourages a high-level perspective, enabling the agent to preserve long-horizon goals, rather than devolving into myopic local reasoning.
• Incremental refinement: Each tool interaction result becomes part of an accumulating context, supporting error correction and adaptive strategies.
• Data-efficient learning: Fine-grained, stepwise plan-annotation enables even smaller LLMs to learn explicit structured reasoning from trajectories.
• Prediction-enriched reasoning: In some variants, agents explicitly predict possible feedback or world responses at each step, using these predictions to anticipate contingencies and enrich follow-up planning (Fu et al., 2024).

2. Formal Multi-Step Planning and Updating Loop

Formally, given a user request $x$, the Pre-Act agent generates a plan

$$S = \{s_1, s_2, \dots, s_n, s_{fa}\}, \quad\text{where} \quad s_i = (\phi_i, a_i)$$

and $s_{fa}$ denotes the final answer step. The agent maintains an internal context

$C_0 = \{\}$

and updates

$C_i = C_{i-1} \cup \{(a_i, o_i)\}$

after execution and observation. At each iteration, the agent computes the next state: $$\phi_i, a_i = \text{LLM}(x, C_{i-1}, P^{\text{draft}}_{i,..,n+1})$$ and upon obtaining $o_i$, refines remaining steps: $$P^{\text{draft}}_{i+1,..,n+1} = \text{LLM}(x, C_i, P^{\text{draft}}_{i+1,..,n+1})$$ The high-level control loop can be summarized as:

def PreActAgent(request x): C = [] P = LLM_Plan(x) # draft S = [s_1,...,s_n,s_fa] for i in range(1, n + 1): phi_i, a_i = P[i] if is_tool_call(a_i): o_i = CALL_TOOL(a_i) else: o_i = None C.append((a_i, o_i)) P[i+1:] = LLM_Refine(x, C, P[i+1:]) phi_fa, a_fa = P[fa] return a_fa

This architecture supports agents that can iteratively correct for errors, prune invalid branches, and update strategies on-the-fly as real-world feedback deviates from prior assumptions.

3. Modular Architecture and Data Flow

Pre-Act decomposes into four principal modules:

1. Planner: Produces an initial draft plan (steps $s_1$...$s_{fa}$) from the input request.
2. Executor: Executes steps, triggering tool APIs or environment actions as specified, capturing observations.
3. Reasoning Refiner: Incorporates the updated context at each step, invoking re-planning routines such that subsequent steps adapt to actual outcomes (e.g., tool errors, environment changes).
4. Finalizer: Aggregates reasoning chains and tool outputs to produce the agent's final answer.

Data Flow Table:

Step	Input	Output
Planner	Request $x$	Draft $\{s_1…s_n,s_{fa}\}$
Executor	Steps $s_i$, accumulated context $C_{i-1}$	Action execution, observation $o_i$
Refiner	Context $C_i$, remaining plan	Refined draft steps $\{s_{i+1}…s_{fa}\}$
Finalizer	Complete context, thoughts, tool outputs	Final answer $a_{fa}$

The data flow sequence is: input request → Planner (drafts plan) → Executor (steps, feedback) → Refiner (plan updates) → Finalizer (answer).

4. Evaluation Framework: Turn-Level and End-to-End

To comprehensively quantify agent performance, Pre-Act introduces a two-level evaluation framework:

Level 1 – Turn-Level: Measures stepwise competence.

• Action Recall: Proportion of predicted tool calls/final answers matching ground truth:

$$\mathrm{ActionRecall} = \frac{1}{N} \sum_{i=1}^N \mathbb{1}\{a_i^{\text{pred}} = a_i^{\text{gt}}\}$$

• Tool-Call F1: Precision/recall/F1 for tool-call parameter matching (where applicable).
• Final Answer Similarity: F1 and semantic similarity (via sentence embeddings) on natural language answers.

Level 2 – End-to-End: Assesses holistic scenario success.

• Tasks are factored into milestones $\{m_1,…,m_M\}$ with directed dependencies.
• Goal Completion Rate (GC): Fraction of milestones completed in valid order,

$\mathrm{GC} = \frac{k}{M}$

• Progress Rate (PR): Length of the longest valid prefix of milestones, normalized by $M$.

This dual-level evaluation allows disentanglement of local missteps from global task planning failures (Rawat et al., 15 May 2025).

5. Comparative Empirical Results

Pre-Act demonstrates substantial improvements over ReAct and, when paired with finely tuned smaller LLMs, can outperform even flagship models such as GPT-4 on benchmark datasets:

Model	Action Recall (Almita)	Goal Completion (GC)	Progress Rate (PR)
ReAct (avg; 5 LLMs)	≈ 0.35	--	--
Pre-Act (avg; 5 LLMs)	≈ 0.60	--	--
Llama 70B (Pre-Act, fine-tuned)	0.8861	0.82	0.89
GPT-4 (Pre-Act)	0.8201	0.64	0.76
GPT-4 (ReAct)	--	0.32	0.50

• Turn-level: Pre-Act outperforms ReAct by ≈ 70% in Action Recall (avg. across five models; Almita dataset).
• Llama 70B (fine-tuned with Pre-Act): Provides an absolute 69.5% recall improvement over GPT-4 on Almita single-turn action prediction and surpasses GPT-4 (Pre-Act) by +28% in end-to-end Goal Completion.
• Smaller models (Llama 8B, fine-tuned): Approach SOTA Action Recall (≈ 0.827), at ~2× speed and ~5× cost reduction compared to Llama 70B.

6. Prediction-Augmented Pre-Act Variants

Another PreAct formulation (Fu et al., 2024) augments the classic planning-act loop with a prediction step. At each reasoning turn, the agent anticipates possible environmental or tool feedbacks before committing to an action (“PREDICTED FEEDBACK”). This prediction is not directly executed but serves to expand reasoning alternatives:

• Supports agent self-reflection by explicitly enumerating possible contingencies.
• Enables downstream handling plans to be conditioned on each prediction, improving diversity and strategic orientation. Empirical analysis reveals that providing LLM agents with historical predictions consistently augments planning scope and action effectiveness.

Illustrative block:

THOUGHT: The fridge has no lettuce, so my assumption about its storage was incorrect. … ACTION: open pantry door PREDICTED FEEDBACK: 1. “You see a head of lettuce…” — Handling: pick up … 2. “There is no lettuce here…” — Handling: reflect, search elsewhere 3. “The pantry door is stuck…” — Handling: grab tool, retry

This approach enhances ReAct by injecting deliberate counterfactual reasoning, supporting richer agent capability on intricate tasks.

7. Practical Tuning, Latency, and Deployment

Pre-Act’s increased up-front and ongoing token generation for explicit multi-step plans impacts inference cost and latency, particularly for large models. Deployment optimization strategies:

• Model selection: Smaller models (Llama 8B fine-tuned for Pre-Act) significantly reduce latency (2×) and cost (5×), with only minor performance degradation.
• Fine-tuning protocol: Use curriculum learning: pretrain on ReAct-formatted corpora, then incrementally introduce high-quality Pre-Act trajectories via parameter-efficient LoRA, minimizing catastrophic forgetting.
• Prompt engineering: Structure inputs to require enumerated “Thought:” and “Action:” pairs, mandate explicit plan refinement, and maintain context accumulation.
• Plan sampling: For robust execution, generate several candidate plans and select via a self-consistency criterion (“beam-width”).
• Milestone graphing: Precompute scenario milestone DAGs (with human validation) to ensure deterministic, reproducible end-to-end evaluation (Rawat et al., 15 May 2025).

Pre-Act thus delivers an architecture for agentic LLMs that bridges generalization gaps in out-of-domain tasks, yielding evaluation gains on both stepwise and end-to-end axes, and enabling resource-efficient deployment in practical workflows.