HMASP: Hierarchical Multi-Agent Payment System

• HMASP is an LLM-based multi-agent system for payment workflows that implements a four-level hierarchy to delegate tasks and ensure structured handoffs.
• The architecture employs persistent, role-specific state objects and decoupled message states to maintain modularity and prevent cross-talk in regulated settings.
• Evaluation studies show high task success rates alongside ASR-identified workflow deviations, emphasizing the need for rigorous checkpoint validations and conformance measures.

Searching arXiv for the cited HMASP papers and closely related evaluation work. Hierarchical Multi-Agent System for Payments (HMASP) is an LLM-based multi-agent architecture for end-to-end agentic payment workflows. It was introduced as a modular system that can leverage either open-weight or proprietary LLMs and that organizes payment operations through hierarchical delegation, shared state variables, decoupled message states, and structured handoff protocols (Chua et al., 27 Feb 2026). A later evaluation work placed HMASP in a broader measurement framework for workflow fidelity, arguing that outcome-centric metrics such as Task Success Rate (TSR) and Agent Handoff F1-Score (HF1) are insufficient in regulated payment settings because they can miss trajectory deviations such as skipped confirmation checkpoints (Huang et al., 7 May 2026).

1. Architectural definition and agent hierarchy

The original HMASP specification consists of four hierarchical agent levels. An external natural-language request enters the Conversational Payment Agent (CPA) at Level 1, which delegates the request through Supervisor agents at Level 2, Routing agents at Level 3, and Process Summary agents at Level 4. The CPA is the entry point for all external requests, parses intent, updates PayAgentState, decides whether to hand off to a Supervisor or reject the request as irrelevant, and aggregates the final summary returned from downstream agents before replying to the user (Chua et al., 27 Feb 2026).

Supervisor agents are domain-scoped. The system specifies one Supervisor per domain: CardsSupervisor for card-related tasks and PaymentsSupervisor for payment-related tasks. A Supervisor receives the handoff from the CPA, updates a domain-specific state object such as CardsState or PaymentState, determines the appropriate task-specific workflow, and delegates to the corresponding Routing agent. The Routing layer is task-specific: examples include R₁ for Task 1 (Card Registration), R₂ for Task 2 (Card Retrieval), and R₃ for Task 3 (Payment Processing). Routing agents perform a final relevance check, trigger function modules such as “Luhn, CVV, Expiry Check,” “Request OTP,” or “Authorize 3DS,” and then pass control to the Process Summary agent. Process Summary agents gather module outputs and compose structured outcome summaries including success or failure flags and key data such as last-4 card digits or transaction ID (Chua et al., 27 Feb 2026).

Original hierarchy level	Agent(s)	Primary role
Level 1	Conversational Payment Agent (CPA)	Entry point, intent parsing, top-level coordination
Level 2	CardsSupervisor, PaymentsSupervisor	Domain routing and state updates
Level 3	Task-specific Routing agents (R₁, R₂, R₃)	Workflow triggering and module invocation
Level 4	Process Summary agents	Structured outcome summarization

A later paper describes HMASP differently, as “a two-level agentic architecture built around a top-level orchestrator called the Payment Supervisor and several domain-specialist sub-agents,” including a Card Payment Agent (CPA), a Payment Execution Agent (PEA), and a Rejection/Filtering Agent (Huang et al., 7 May 2026). The terminology therefore differs across the two papers: in the original, CPA denotes the Conversational Payment Agent at the root of a four-level hierarchy, whereas in the later evaluation, CPA denotes a Card Payment Agent beneath a Payment Supervisor. This suggests that the later work abstracts HMASP into a coarser orchestration view for conformance analysis rather than restating the original hierarchy verbatim.

2. State model, message isolation, and handoff semantics

HMASP uses LangGraph’s Graph State abstraction and defines three persistent role-based state objects: PayAgentState $S^p$, used only by the CPA; CardsState $S^c$, used by all card-related agents and modules; and PaymentState $S^{pay}$, used by all payment-related agents and modules (Chua et al., 27 Feb 2026). Only designated shared variables are visible across these states, with examples including userID, cardID, and transactionID; all other fields remain compartmentalized.

The formal state evolution is defined by

$S_t = (S_t^p, S_t^c, S_t^{pay}),$

and for agent $A$ processing message $m_t$ at time $t$,

$$S_{t+1}^A = f_A(S_t^A, m_t), \qquad A \in \{\text{CPA}, \text{Cards}, \text{Payments}\}.$$

The global state then evolves as

$$S_{t+1} = \left(S^{CPA}_{t+1}, S^{Cards}_{t+1}, S^{Payment}_{t+1}\right).$$

This formulation makes explicit that state mutation is local to an agent role while the global workflow remains synchronized through designated shared variables (Chua et al., 27 Feb 2026).

Each agent also maintains a decoupled message state $M^A$, partitioned into upstream messages from parent agents, self-outputs generated within the agent, and downstream-responses from child agents. When agent $S^c$0 sends a message $S^c$1, it is appended to $S^c$2; when it receives a message from a child, it appends the message to $S^c$3. Agents read only their own $S^c$4, which the paper states reduces cross-talk and token bloat (Chua et al., 27 Feb 2026). In architectural terms, HMASP therefore combines persistent role-specific state with communication locality.

Handoffs are implemented via LangGraph calls. At the top level, the CPA updates PayAgentState, classifies the input domain, then hands off to CardsSupervisor, PaymentsSupervisor, or rejects the request as not payment related. The paper also provides a finite-state-machine sketch for the CPA: $S^c$5 is “Waiting for user request,” $S^c$6 is the parse/update state, $S^c$7 and $S^c$8 correspond to handoff to the card or payment Supervisor respectively, and $S^c$9 is rejection followed by a return to $S^{pay}$0 (Chua et al., 27 Feb 2026). The importance of this handoff discipline becomes more pronounced in the later ASR work, where every handoff is treated as an atomic transition for auditability and trajectory evaluation (Huang et al., 7 May 2026).

3. Workflow semantics, interrupts, and validation routines

The HMASP workflow space in the original paper is organized around four task categories: Task 1 for Card Registration triggers, Task 2 for Card Retrieval triggers, Task 3 for Payment Processing triggers, and Task 4 for irrelevant inputs that must be rejected (Chua et al., 27 Feb 2026). Within this structure, Routing agents are responsible for the final workflow-specific relevance check and for triggering the required functional subgraph.

The Task 3 payment-processing pseudocode is representative. If the input is not relevant, the payment Routing agent hands off to the payment Process Summary agent with a rejection outcome. Otherwise, it invokes an interrupt prompting the user to “Please select a card or enter new,” validates the selected card, requests OTP, authorizes 3DS, and hands off the resulting outcome data to the payment Process Summary agent. The key API-like calls identified in the paper are interrupt(prompt), which pauses the LLM flow, obtains human input, and persists it in state; requestOTP(card), a simulated OTP dispatch and retrieval process; and authorize3DS(card, OTP), a simulated 3D Secure call (Chua et al., 27 Feb 2026).

The validation submodule is explicit. validateCard(cardNumber, cvv, expiry) throws InvalidCardNumberError if LuhnCheck(cardNumber) fails, InvalidCVVError if length(cvv) == 3 is false, and CardExpiredError if expiry < today(). On success it returns true (Chua et al., 27 Feb 2026). The paper therefore frames determinism not as fully autonomous free-form generation, but as bounded execution around stored state, validation routines, and human-in-the-loop interruptions.

A later HMASP evaluation paper makes the payment-processing trajectory more explicit by specifying an expected T3 sequence of 11 agent visits and 10 consecutive handoff transitions. Its canonical sequence is: User $S^{pay}$1 Payment Supervisor; Payment Supervisor $S^{pay}$2 CPA; CPA $S^{pay}$3 Payment Supervisor; Payment Supervisor $S^{pay}$4 CPA for the confirmation checkpoint; CPA $S^{pay}$5 Payment Supervisor with the user’s confirmation; Payment Supervisor $S^{pay}$6 PEA for execution; PEA $S^{pay}$7 Payment Supervisor with success or failure; Payment Supervisor $S^{pay}$8 CPA for logging; CPA $S^{pay}$9 Payment Supervisor; Payment Supervisor $S_t = (S_t^p, S_t^c, S_t^{pay}),$0 Output Agent; and Output Agent $S_t = (S_t^p, S_t^c, S_t^{pay}),$1 Payment Supervisor (Huang et al., 7 May 2026). The key checkpoints are the validation handoff, explicit confirmation, and execution. This later formulation sharpens a point that is only implicit in the original architecture: the semantics of a payment workflow depend not only on reaching a correct final result but also on visiting mandatory intermediate checkpoints.

4. Experimental setup and baseline performance

The original HMASP experiments use a dataset of 1,000 natural-language prompts evenly split into four tasks, with 250 prompts each for Card Registration, Card Retrieval, Payment Processing, and Irrelevant inputs. The models evaluated include a proprietary baseline, GPT-4.1 (2025-04-14 release), and open-weight models run locally on Ubuntu 24.04.1 with an NVIDIA H100 with 96 GB VRAM via Ollama v0.9.6: Qwen3:8b, 14b, 32b; Qwen2.5:7b, 14b, 32b; Mistral-small3.2:24b; Magistral:24b; and Llama 3.1:8b, 70b (Chua et al., 27 Feb 2026).

Two baseline metrics are defined. Task Success Rate uses the condition

$S_t = (S_t^p, S_t^c, S_t^{pay}),$2

with success rate measured as the number of successes divided by 250 per task. Agent Handoff F1-Score compares actual versus expected CPA$S_t = (S_t^p, S_t^c, S_t^{pay}),$3Supervisor and Supervisor$S_t = (S_t^p, S_t^c, S_t^{pay}),$4Routing handoffs; for Task 4, “no handoff” is the correct behavior, and true positives count correct rejections (Chua et al., 27 Feb 2026).

The quantitative results reported in the original paper establish strong performance. GPT-4.1 leads at at least $S_t = (S_t^p, S_t^c, S_t^{pay}),$5. Among open-weight models, Qwen2.5:32b achieves at least $S_t = (S_t^p, S_t^c, S_t^{pay}),$6 on all payment tasks. On Agent Handoff Performance, GPT-4.1 attains an average F1 of $S_t = (S_t^p, S_t^c, S_t^{pay}),$7, with Qwen2.5:32b close at $S_t = (S_t^p, S_t^c, S_t^{pay}),$8. The paper also states that Llama 3.1 variants underperform, especially on rejecting irrelevant inputs in Task 4 (Chua et al., 27 Feb 2026).

Model	T1	T2	T3	T4
Qwen2.5:32b	96.4%	98.8%	95.6%	99.6%
GPT-4.1	99.6%	99.6%	99.6%	99.6%

These results support the feasibility claim for end-to-end agentic payment workflows, but they are limited to final-task completion and coarse handoff correctness. The later ASR work treats that limitation as central rather than incidental.

5. Workflow fidelity and the Agentic Success Rate

The later HMASP evaluation paper argues that TSR and HF1 capture only final outcomes or unordered routing decisions and therefore can fail to expose workflow noncompliance in regulated domains (Huang et al., 7 May 2026). It introduces Agentic Success Rate (ASR), a trajectory-fidelity metric based on transition-level agreement between an expected agent-visit sequence $S_t = (S_t^p, S_t^c, S_t^{pay}),$9 and an observed sequence $A$0. With multisets of transitions

$A$1

$A$2

the metric is defined through Transition Recall and Transition Precision:

$A$3

and

$A$4

The ASR study evaluates HMASP across 90,000 runs, computed as 1,000 inputs $A$5 4 scenarios $A$6 18 models $A$7 5 repeats (Huang et al., 7 May 2026). Its principal finding is that 10 of 18 models systematically skip a confirmation checkpoint during payment checkout. In the reported shortcut pattern, the expected number of transitions is 10, while the actual number recorded is 8, yielding $A$8, $A$9, and $m_t$0. The paper states that this deviation is invisible to both TSR and HF1 because the final outcome can still be correct and the coarse routing edges can still appear valid (Huang et al., 7 May 2026).

The comparative findings are especially notable. Eight models—identified as all Gemma4 variants, both GPT-OSS models, Mistral-Small-3.2:24B, and GPT-5.2—achieve perfect ASR of $m_t$1. GPT-4.1 achieves perfect TSR of $m_t$2 and HF1 of $m_t$3 yet has ASR below $m_t$4, indicating hidden workflow shortcuts. GPT-5.2 is reported as the only proprietary model with perfect ASR across all scenarios (Huang et al., 7 May 2026). The resulting conceptual distinction is straightforward: high task completion does not guarantee conformance to a mandated payment procedure.

6. Limitations, deployment constraints, and subsequent engineering guidance

The original HMASP paper lists several limitations. Its dataset of 1,000 prompts is “relatively small,” and future work is said to require a larger, more diverse corpus covering edge-case payment workflows. Security and compliance, including PCI-DSS and fraud controls, are only simulated; real-world deployments must integrate hardened tokenization and industry-grade guardrails. Architectural variants such as heterogeneous LLM assignments per agent and dynamic agent spawning remain unexplored, and comparative studies of alternative multi-agent topologies and protocols are identified as possible sources of improved robustness or efficiency (Chua et al., 27 Feb 2026).

The later ASR paper translates these constraints into deployment guidance. It recommends going beyond final-outcome metrics and unordered-edge metrics, instrumenting every agent handoff in logs, using conformance checking against expected transitions, defining and mandating critical checkpoints such as user confirmation and fraud checks, employing deterministic routing guards for security-critical branches, and re-auditing live logs with ASR or weighted-ASR variants before and after model or prompt changes (Huang et al., 7 May 2026). Its examples include a cart-items guard that blocks any transition toward payment review if cart_items is empty, and a CPA card-view fallback that uses keyword patterns to reroute ambiguous card intents rather than skipping confirmation.

The same paper also reports ASR-guided prompt refinements for the CPA, including: “After extracting card info, you must always solicit explicit user confirmation before any payment execution,” and “Do not hand off to the execution agent unless you have seen the user’s ‘Yes, go ahead’.” Combined with deterministic routing guards in the Payment Supervisor, these changes produce substantial TSR improvements for previously struggling models, with reported average gains over five repeats of $m_t$5 percentage points, $m_t$6 points, and $m_t$7 points for Llama 3.1:8B on Tasks 1–3 respectively; $m_t$8, $m_t$9, and $t$0 points for Magistral:24B; and $t$1, $t$2, and $t$3 points for Llama 3.1:70B (Huang et al., 7 May 2026).

Taken together, the two papers position HMASP as both an architectural proposal and an evaluation target. The original work presents it as the first LLM-based multi-agent system to implement end-to-end agentic payment workflows and attributes its behavior to a hierarchical, state-partitioned, and interrupt-capable design that achieves modularity, determinism via stored state and validation, and high success and handoff accuracy (Chua et al., 27 Feb 2026). The later work shows that such systems must also be assessed at the trajectory level, because a payment workflow can appear correct under TSR and HF1 while still violating mandatory intermediate steps (Huang et al., 7 May 2026).