SOP-Agent Framework

• SOP-Agent Framework is a modular, LLM-centric system that processes domain-specific procedures using both shallow (LRS) and deep (HRS) decision modules.
• It categorizes SOP tasks into high-branch (LRS) and deep sequential (HRS) challenges, employing specialized modules like Branch Selector and Step Verifier.
• Evaluation metrics based on JSON schema compliance and branching accuracy rigorously validate the framework's ability to manage procedural complexity and conversational noise.

A Standard Operating Procedure Agent (SOP-Agent) Framework denotes a system that operationalizes precise, context- and constraint-aligned execution of standard operating procedures (SOPs) using modular, LLM-centric architectures. SOP-Agents process domain-specific procedural knowledge, orchestrate long-horizon decision graphs, manage complex user-agent interactions, and enforce schema-level compliance in outputs. The SOP-Agent paradigm, comprehensively evaluated in the SOP-Maze benchmark, provides a blueprint for building agents that robustly follow intricate business SOPs, address conversational noise, and perform accurate procedural reasoning (Wang et al., 10 Oct 2025).

1. SOP Task Taxonomy and Agent Functional Requirements

SOP-Maze defines two canonical classes of SOP tasks, each imposing distinct computational and architectural challenges:

• Lateral Root System (LRS) Tasks:

These tasks consist of shallow decision trees (depth ≤ 3) but a high branching factor (often 10+ alternatives per node). The core challenge is breadth—the agent must accurately select among many parallel options according to the given context. An exemplar is the “complaint disposition code” SOP with 12 candidate codes.

• Heart Root System (HRS) Tasks:

Here the structure is a narrow but deep decision graph (depth ≥ 5), with extended sequential or nested dependencies. The principal challenge is depth—requiring the agent to maintain a long chain of logical inferences and update intermediate state consistently. For instance, “bulk order clarification” SOPs with 7 nested conditionals on dates, product types, and stock levels.

Functional mapping of these categories to the requisite agent modules:

• LRS requires a high-throughput breadth module with option enumeration, similarity scoring, constraint verification, and schema-compliant buffering.
• HRS demands depth modules for chain-of-thought (CoT) persistence, hierarchical subgoal decomposition, and logical consistency verification between steps.

2. SOP Representation: Data Schema and Formal Model

SOP-Maze formalizes each SOP instance as a structured, labeled directed graph: $G = (N, E, s_0, S_f)$, where

• $N = \{n_1, ..., n_k\}$: decision/action nodes,
• $E \subseteq N \times N$: edges labeled with guard predicates $\phi(e)$,
• $s_0 \in N$: start node,
• $S_f \subseteq N$: set of terminal nodes encoding output requirements.

Each agent state $s \in N$ is advanced by the transition function: $$\delta(s, M) = s' \;\text{if}\; (s \to s') \in E \;\text{and}\; \phi(s \to s')(M) = \text{true}$$ where working memory $M$ holds all relevant extracted variables (timestamps, intents, slot values). SOP data are represented as JSON meta-schemas, with explicit segregation of Objective, Procedure graph, User Input, and Output Requirement (as a reference JSON Schema).

A correct execution is a path $s_0 \to s_1 \to \cdots \to s_f \in S_f$ where all transitions respect their guards under the evolving memory context.

3. Evaluation Metrics

SOP-Agent capability is rigorously quantified on SOP-Maze via a multi-tier metrics suite:

• Primary Score (JSON Schema Compliance):

$$S_{\mathrm{instance}} = \begin{cases} 1.0 & \text{exact schema and value match} \ 0.2 & \text{schema-valid but incorrect values} \ 0.0 & \text{schema violation/malformed} \end{cases}$$

• Aggregate Accuracy:

$$\mathrm{Acc} = \frac{1}{N}\sum_{i=1}^N \mathbf{1}\{S_i=1.0\}$$

• Mean Instance Score:

$\overline{S} = \frac{1}{N} \sum_{i=1}^N S_i$

• Branching Coverage (LRS):

$$\mathrm{Coverage}_{\mathrm{LRS}} = \frac{1}{N_{\mathrm{LRS}}} \sum_{i\in\mathrm{LRS}} \frac{C_i}{B_i}$$, assessing breadth-precision.

• Chain Depth Accuracy (HRS):

$$\mathrm{DepthAcc}_{\mathrm{HRS}} = \frac{1}{N_{\mathrm{HRS}}} \sum_{i\in\mathrm{HRS}} \frac{c_i}{d_i}$$, quantifying deep procedural correctness.

These metrics collectively capture both output well-formedness and decision-path optimality.

4. Systematic Error Analysis

SOP-Maze’s breakdown across top SOTA models reveals three dominant error classes:

• Route Blindness:

The agent diverges from the prescribed SOP graph, either by incorrect option selection (LRS) or by skipping/fast-forwarding over dependencies (HRS). Example: 177/397 errors for DeepSeek-V3.1.

• Conversational Fragility:

The model fails under dialogic noise, such as intent reversal, ambiguity, or sarcasm, leading to misinterpretation or missed transitions. Example: 149 conversational errors for Claude-Opus-4.

• Calculation Errors:

Numeric/time reasoning faults, e.g., imprecise latency calculations or order-statistics, most prevalent in business operations involving arithmetic. Example: 63 instances for Doubao-Seed-1.6.

Collectively, these categories ground agent design in error-aware, modular supervision.

5. Modular SOP-Agent Architecture and Dataflow

The recommended production-grade SOP-Agent incorporates the following modules:

1. Input Processor: Utterance normalization, slot/entity extraction (NER/time parsing), and sarcasm/intent detectors. Outputs memory updates $M$.
2. SOP Interpreter: Loads SOP graph $\pi = (N, E, \phi, \cdots)$, manages current node $s$, applies $\delta(s, M)$ for transitions. Features a Branch Selector for LRS (multi-condition scoring and ranking) and a Step Verifier for HRS (post-hoc logical consistency check).
3. Reasoning Core: Maintains a CoT recursive buffer and interfaces with an Arithmetic Engine (delegating all numeric computation to a calculator API or module).
4. Dialogue Manager: Manages dialogue history, resolves anaphora, tracks user intent/state, and dynamically generates clarifying questions when ambiguity is detected.
5. Output Formatter: Fills the output JSON schema from terminal state, performs schema compliance validation and correction.

A canonical pseudocode outline encapsulates the agent's control logic:

def handle_turn(user_utterance): M_updates = InputProcessor.parse(user_utterance) M.merge(M_updates) while True: possible_edges = SOPInterpreter.get_outgoing(s) valid = [e for e in possible_edges if φ(e)(M)] if len(valid) == 1: s = δ(s, M) # deterministic transition elif len(valid) > 1: s = BranchSelector.choose(valid, M) else: clar = DialogueManager.request_clarification(s, M) user_response = get_user_input(clar) continue break if s in S_f: result = OutputFormatter.fill_schema(s, M) return result else: return prompt_for_next_step()

Training protocols should:

• Pre-train numerical reasoning components separately.
• Fine-tune the Dialogue Manager on actual business conversation data with nuanced intent annotation.
• Jointly optimize the Branch Selector and Step Verifier with composite loss: $$\text{Loss} = \text{CE(branch\_choice)} + \lambda \cdot \text{CE(step\_sequence)} + \mu \cdot \text{schema\_violation\_penalty}$$.

6. Implications for Business AI and Future Directions

The SOP-Agent framework, as codified in SOP-Maze, presents a reconciliatory architecture for LLM-powered business agents: it resolves the dichotomy between breadth (option identification/disambiguation) and depth (logical sequentialization), bridges agent interaction with realistic noisy input, and formalizes precise output validation (Wang et al., 10 Oct 2025).

Embedding the SOP formalism at every tier—input processing, procedural graph traversal, dialogue grounding, and output schema enforcement—substantially reduces the dominant error modes observed in current models. The framework provides a path for targeted fine-tuning and module ablation, enabling the systematic elevation of SOP adherence, resilience to dialogic perturbations, and arithmetic reliability in deployed agents.

Continued research should focus on scalable SOP graph generation, dynamic procedure adaptation, conversational robustness, and the integration of formal verification components into the SOP pipeline for mission-critical business automation.