Beyond Generalist LLMs: Specialist Agentic Systems for Structured Code Workflow Execution

Abstract: LLMs have accelerated the adoption of software development agents, now widely available as Integrated Development Environment (IDE) extensions and standalone applications. While these agents are typically general-purpose, it remains unclear whether specialist agents justify their additional development effort. We investigate this question in the context of business process automation, focusing on the transformation of Business Process Model and Notation (BPMN) diagrams into executable agentic workflows. Since BPMN specifies explicit control-flow semantics, we focus on deterministic workflows in which a fixed process model and inputs uniquely determine the executed path. We introduce a specialist workflow for this task and compare it against generalist agents such as Roo and Cline. Our results show that the specialist solution produces agents that outperform generalist baselines by approximately 9-20 percentage points in tool-use exactness, 2-4x in penalty-adjusted latency, and 3x fewer tool-call errors, while reducing generation token cost by over 95% and eliminating repair iterations. We also find that generalist agents generate code inconsistently in both functionality and quality, limiting their suitability for industrial settings where reliability and maintainability are essential.

• The paper presents a specialist system that transforms BPMN diagrams into deterministic, modular agentic workflows.
• It demonstrates significantly reduced token usage and zero repair iterations, achieving higher tool-use exactness and process adherence.
• The system offers improved reliability, cost efficiency, maintainability, and scalability for structured business process automation.

Specialist Agentic Systems for Structured Code Workflow Execution

Motivation and Background

The paper ("Beyond Generalist LLMs: Specialist Agentic Systems for Structured Code Workflow Execution" (2607.14456)) addresses the limitations of generalist LLM-based coding agents in deterministic workflow automation scenarios, particularly when transforming Business Process Model and Notation (BPMN) diagrams into executable agentic code. Generalist agents (e.g., Roo, Cline, FLOW, AutoGen, MetaGPT) are characterized by unconstrained task flexibility but exhibit inconsistent output quality, inefficiency in token utilization, excessive repair iterations, and lack of process adherence. The study posits that specialist systems, wherein agentic workflows are modularly constructed and externally specified, provide superior performance for structured business process automation through explicit control-flow constraints, targeted context management, and deterministic tool orchestration.

System Architecture and Workflow Compilation

The specialist system parses BPMN diagrams, extracts workflow primitives (tasks, decision gateways, branches), and compiles them into ReAct-style agentic workflows. Each BPMN node is mapped to a typed tool contract and a localized policy; the orchestration layer ensures branch/join semantics and context scoping per node. API specifications are transformed into reusable modules and tool wrappers, facilitating modular decomposition and code reuse. Dynamic refinement loops are implemented for agent self-verification, where tool implementations are iteratively adjusted until they pass end-to-end validation.

Figure 1: Proposed System Architecture, depicting modular parsing, tool generation, iterative refinement, and agent assembly for BPMN-driven workflows.

Experimental Setup and Methodology

Evaluation was conducted on ten diverse deterministic BPMN workflows spanning domain-specific automation tasks (e-commerce, costing, deal assessment, classification, news aggregation, risk analysis, social media moderation, customer promotion, tournament orchestration, weather reporting). The workflows vary in structural complexity (9–52 nodes). Baseline systems Roo and Cline, as well as supplementary candidates (AutoGen, MetaGPT, FLOW), received identical BPMN and API specifications; failed executions were allowed iterative repair attempts. Each agentic system generated 10 successful agents per workflow, tested exhaustively on all possible execution paths.

Agent Generation Metrics

Metrics for agent generation include repair iterations and token utilization. The specialist system achieved zero repair iterations per successful agent, outperforming Roo (2.08) and Cline (1.89), both of which required iterative correction cycles.

Figure 2: Agent Generation Metrics highlighting repair iteration counts and token usage statistics for each system.

Token usage analysis demonstrated stark reductions: the specialist system's input token consumption averaged 49.24k and output 5.82k per agent, compared to Roo (1,484.46k in / 17.40k out) and Cline (1,029.27k in / 15.36k out), corresponding to orders-of-magnitude lower cost (95%+ reduction).

Agent Run Evaluation and Process Adherence

The functional correctness and efficiency of generated agents were evaluated using process adherence (binary verdict against workflow specification), Tool-Use Exactness (TUE, fraction of runs with zero tool-call errors), penalty-adjusted latency, and error decomposition (missed, excess, out-of-sequence tool calls).

Figure 3: Per-workflow agent run evaluation metrics for the specialist system, including exactness, errors, latency, and adherence.

Figure 4: Key metrics by approach, comparing specialist and baseline systems across tool-use exactness, errors, latency, and adherence.

Key findings:

• The specialist system achieved TUE 57.69%, compared to Cline (48.62%) and Roo (38.11%).
• Tool-call errors per run were 1.27 for the specialist system versus 3.19 (Cline) and 3.22 (Roo), with the dominant mode being missed calls.
• Process adherence (binary conformance) was 54.68% specialist, surpassing Cline (51.57%) and Roo (42.43%), with adherence dropping significantly in generalist systems as workflow complexity increased.
• Penalty-adjusted latency was 2.49s (specialist), versus 6.59s (Cline) and 9.08s (Roo)—a 2.6–3.6× reduction.

Representative Workflows

The evaluated workflows included practical business logic such as online cart fulfillment, cost estimation, deal evaluation, classification, news processing, risk assessment, social signal aggregation, customer promotion, tournament progression, and weather reporting. These were instantiated as BPMN diagrams to ensure explicit control flow and deterministic execution.

Figure 5: Online retail cart/order fulfillment workflow, illustrating the deterministic branching and validation embedded in BPMN.

Figure 6: Costing/estimation workflow, showcasing sequential data gathering and decision gateways.

Figure 7: News processing workflow, demonstrating multi-stage enrichment and conditional branching.

Practical Implications and Theoretical Impact

The results demonstrate that specialist agentic systems provide significant operational advantages in structured code generation settings:

• Reliability: Deterministic BPMN translation yields consistent, auditable executable code, drastically reducing maintenance burden and technical debt.
• Cost efficiency: Targeted context scoping and modular tool composition result in minimal token exposure and execution costs, critical in enterprise-scale deployments.
• Maintainability: Modular decomposition of code and prompt instructions simplifies debugging, integration, and iterative improvement.
• Scalability: Specialist workflows support repeated and large-scale automation tasks without runtime plan rediscovery or context drift.

Generalist systems are limited by plan discovery at runtime, excessive and unpredictable context exposure, and lack of structural guarantees, making them less suitable for domain-specific automation tasks requiring strict adherence and traceability.

Future Directions in AI Workflow Automation

Future research directions include:

• Extending the evaluation to nondeterministic workflows and hybrid agent architectures.
• Incorporating component-level ablations to identify the marginal impact of control-flow constraint, context management, and modular decomposition.
• Integrating specialist workflow paradigms into generalist multi-agent systems as dynamic modules for improved reliability.
• Benchmarking across open business process datasets to further generalize conclusions.

Conclusion

Specialist agentic systems anchored in BPMN-driven workflows and modular decomposition outperform generalist LLM-driven coding assistants in deterministic process automation by several strong metrics: higher tool-use exactness, lower error rates, improved process adherence, reduced latency, and substantial cost savings. The approach addresses key operational requirements—reliability, maintainability, and scalability—essential for practical deployment in enterprise settings. Theoretical implications underscore the necessity of domain structural constraints and scoped context management in agentic code synthesis, suggesting future hybrid architectures that leverage both specialist and generalist capabilities.