Auto-Matching Network for Long-Horizon Control

• AMN is a modular long-horizon control framework that uses vision-language models for decomposing complex mobile manipulation tasks into logical, sequential subtasks.
• It dynamically matches each subtask with a pre-trained expert policy, significantly reducing error accumulation compared to monolithic end-to-end approaches.
• Empirical results demonstrate that AMN improves final task completion by over 40 percentage points on tasks such as Clean Trash and Deliver Tool.

Auto-Matching Network (AMN) denotes, in the RoboMatch framework, a long-horizon control architecture for mobile manipulation that decomposes a complex task into a sequence of shorter logical subtasks and dynamically assigns each subtask to a specialized lightweight policy network for execution (Liu et al., 10 Sep 2025). In that formulation, AMN is not a single end-to-end visuomotor policy, but a planner-plus-router architecture in which a vision-LLM interprets a command and scene, produces a sequential task decomposition, and matches each stage to a pre-trained expert. The acronym is not stable across the literature: other arXiv works use AMN to mean Analogical Matching Network (Crouse et al., 2020), Adversarial Multimodal Network (Yuan et al., 2019), Dual Attention Matching Network (Mao et al., 2021), or Associative Memory Net (Dash, 2013). Within RoboMatch, however, AMN specifically refers to auto-matching for long-horizon robotic manipulation (Liu et al., 10 Sep 2025).

1. Terminology and acronym disambiguation

The phrase Auto-Matching Network is explicitly associated with RoboMatch, where it is introduced as an architecture for long-horizon mobile manipulation in dynamic environments (Liu et al., 10 Sep 2025). This usage is distinct from several unrelated AMN acronyms in adjacent literatures. In analogical reasoning, AMN denotes the Analogical Matching Network, a neural model for correspondence selection over symbolic relational DAGs (Crouse et al., 2020). In Movie Question Answering, AMN denotes the Adversarial Multimodal Network (Yuan et al., 2019). In knowledge graph entity alignment, the relevant model is Dual Attention Matching Network (Dual-AMN) (Mao et al., 2021). In handwritten character recognition, AMN refers to Associative Memory Net (Dash, 2013).

This acronym overlap is not merely terminological. The RoboMatch AMN is a robotics control framework whose central operation is task decomposition and policy matching; it is not a symbolic analogy engine, not a multimodal adversarial QA model, not an entity-alignment encoder, and not an auto-associative memory network. A common misconception is therefore to treat “AMN” as a single established architecture family. The published record instead supports a narrower reading: the term is domain-specific, and in RoboMatch it names a modular long-horizon execution system (Liu et al., 10 Sep 2025).

2. Problem formulation in long-horizon mobile manipulation

RoboMatch motivates AMN by a failure mode of monolithic long-horizon imitation policies: in long-horizon robot imitation learning, “inference errors accumulate, while the absence of logical dependencies between action segments hinders a single policy from generalizing to multi-skill, multi-scenario applications” (Liu et al., 10 Sep 2025). The target setting is mobile manipulation in dynamic or open environments, where an episode may require navigation, object-specific grasping, transport, placement, and return within a single trajectory.

The paper contrasts AMN with conventional end-to-end models such as ACT, vanilla Diffusion Policy (DP), and the stronger manipulation policy PVE-DP. These baselines may perform well on shorter or more homogeneous tasks, but their long-horizon performance degrades because they do not explicitly encode task-stage logic or subtask-specific specialization (Liu et al., 10 Sep 2025). AMN addresses this by representing a task not as one uninterrupted control problem, but as an ordered set of subtasks:

$$\mathcal{T} = \{T_i\}_{i=1}^n, \quad T_i \sim \pi_i(s_t,a_t|\theta_i)$$

where each $T_i$ is executed by its own policy $\pi_i$ with parameters $\theta_i$ (Liu et al., 10 Sep 2025).

The experimental tasks make this decomposition concrete. Clean Trash is a four-step task over 3000 time steps, with stages Grasp → Move → Throw → Back. Deliver Tool is a six-step task over 4000 time steps, with stages Move → Grasp(screw) → Grasp(pouch) → Move → Put → Back (Liu et al., 10 Sep 2025). These examples show that AMN is stage-specific and skill-specific rather than modality-specific: the matching routes among policies specialized by subtask semantics.

3. Architecture and decomposition mechanism

RoboMatch consists of three interacting components: a unified teleoperation platform, PVE-DP, and AMN (Liu et al., 10 Sep 2025). AMN sits above the low-level imitation-policy layer and orchestrates long-horizon execution through decomposition and policy matching. Figure descriptions in the paper characterize AMN as combining “the semantic parsing capabilities of VLMs with the execution advantages of specialized small policy networks for long-horizon execution” (Liu et al., 10 Sep 2025).

At the planning layer, the inputs are:

• A command from the operator, given by voice or text
• A scene image observed by the robot

These are processed by the vision-LLM GLM-4.1V, which performs Chain-of-Thought-style reasoning to break the task into sequential steps (Liu et al., 10 Sep 2025). Each resulting subtask is then matched to a corresponding pre-trained expert policy. The execution logic is sequential: after one subtask completes, “the system automatically selects the next network until the entire task is completed” (Liu et al., 10 Sep 2025).

The architecture can therefore be described as a planner-plus-router. Its outputs are not, in the first instance, direct robot torques or actions; AMN first produces subtask-level execution decisions, namely which policy should run at the current stage (Liu et al., 10 Sep 2025). The selected policy then emits low-level action commands. For manipulation stages, the paper states that PVE-DP is matched to perform robotic manipulation, indicating that PVE-DP functions as one of the execution policies within the AMN-controlled stack (Liu et al., 10 Sep 2025).

The method section is concise, and the paper explicitly leaves several components unspecified. It does not provide:

• an explicit subtask completion detector
• a formal matching score
• a confidence threshold
• a state machine diagram with transition rules
• a learned gating loss
• a detailed scheduling policy for distributed inference
• pseudocode for asynchronous module execution

Accordingly, the most faithful characterization is the paper’s own high-level one: AMN is a modular long-horizon control framework with VLM-based decomposition and expert-policy matching, rather than a fully mathematically defined end-to-end trainable mixture-of-experts network (Liu et al., 10 Sep 2025).

4. Training pipeline and subtask policy learning

AMN is trained from long real-world demonstrations, in contrast to PVE-DP, which is trained from short demonstrations (Liu et al., 10 Sep 2025). The paper reports two long-horizon AMN tasks:

Task	Real demonstrations	Length
Clean Trash	50	1.1 h
Deliver Tool	50	1.4 h

During collection, the operator uses a cockpit-style control interface to operate the mobile base and dual arms synchronously, producing coherent whole-body trajectories (Liu et al., 10 Sep 2025). AMN then turns those long trajectories into modular training data by saving state-action snippets at subtask boundaries and training separate networks on the resulting short-horizon segments.

The training-time mechanism is described as a step-wise saving mechanism for tasks spanning more than 3000+ time steps: “upon completion of each subtask $\mathcal{T}_i$, the current state $s_{t_i}$ and action $a_{t_i}$ are saved as a subtask sample” (Liu et al., 10 Sep 2025). Through this temporal decoupling strategy, the original long task is decomposed into $n$ short-horizon subtasks $\{T_1,\ldots,T_n\}$, “each trained with an independent network architecture $\Phi(\cdot|\theta_i)$” (Liu et al., 10 Sep 2025).

The operational training recipe stated in the paper is:

1. collect demonstrations of long-horizon tasks using the RoboMatch teleoperation platform
2. identify logical subtask boundaries within each long demonstration trajectory
3. for each completed subtask $T_i$0, save boundary-aligned samples $T_i$1
4. train an independent policy network $T_i$2 or $T_i$3 for each subtask using the corresponding short-horizon data
5. compose the experts temporally at inference

No standalone AMN loss function is given. The paper therefore frames optimization as supervised imitation learning applied separately to each subtask expert, coupled with VLM-driven task decomposition and policy routing (Liu et al., 10 Sep 2025). This suggests that AMN’s main technical contribution lies in decomposition and composition rather than in a new low-level policy objective.

5. Inference-time execution and representation stack

At deployment, the command and current scene image are sent to GLM-4.1V, whose Chain-of-Thought reasoning generates an ordered subtask plan $T_i$4 (Liu et al., 10 Sep 2025). For the current subtask $T_i$5, AMN matches the corresponding pre-trained expert policy $T_i$6, executes it, then transitions to the next matched policy after completion. This stage-aware sequencing is the core of the method’s “distributed inference”: the paper’s description indicates that inference is distributed across multiple small expert networks rather than concentrated in one large policy (Liu et al., 10 Sep 2025).

The execution stack is multimodal. At the AMN layer, the salient inputs are language instruction and scene image. At the low-level policy layer, the selected expert—especially PVE-DP during manipulation—consumes richer observations including:

• RGB images from robot cameras
• joint positions $T_i$7
• end-effector quaternion from IMUs $T_i$8
• fused visual-proprioceptive internal representations

The paper is careful to separate these levels. The fused visual-proprioceptive features belong to the execution policy, not to AMN’s matching mechanism itself (Liu et al., 10 Sep 2025). Thus, AMN’s internal object of reasoning is best understood as the subtask sequence inferred by the VLM, while PVE-DP and related experts handle local perception-action mappings.

Several practical implications follow directly from the stated design. First, shortening the horizon handled by any one policy reduces error accumulation and the burden of implicit stage inference inside a monolithic model (Liu et al., 10 Sep 2025). Second, explicit stage decomposition makes dependencies such as “grasp before move” or “move before put” part of the execution plan rather than an emergent property of one policy. Third, the paper does not specify replanning during execution, fallback policies, or recovery behaviors when a subtask fails (Liu et al., 10 Sep 2025). A plausible implication is that policy boundaries and handoffs remain a critical engineering locus.

6. Empirical performance, interpretation, and limitations

AMN is evaluated on the real RoboMatch platform in the two long-horizon tasks Clean Trash and Deliver Tool, against ACT, DP U-Net, PVE-DP, and AMN (ours) (Liu et al., 10 Sep 2025). The reported metric is long-horizon inference performance, operationalized as success rates at each stage of the task.

Task	Method	Stagewise success rates
Clean Trash	ACT	54%, 46%, 26%, 10%
Clean Trash	DP U-Net	30%, 14%, 0%, 0%
Clean Trash	PVE-DP	66%, 42%, 20%, 8%
Clean Trash	AMN	74%, 70%, 54%, 50%
Deliver Tool	ACT	100%, 52%, 34%, 24%, 6%, 2%
Deliver Tool	DP U-Net	100%, 26%, 8%, 0%, 0%, 0%
Deliver Tool	PVE-DP	100%, 54%, 28%, 8%, 0%, 0%
Deliver Tool	AMN	100%, 68%, 66%, 62%, 48%, 44%

These results show a characteristic horizon-dependent pattern. On the easiest early stages, some baselines remain competitive; for example, all methods achieve 100% on the first Move stage of Deliver Tool (Liu et al., 10 Sep 2025). As the sequence length grows, however, baseline success rates decay sharply, whereas AMN degrades more gradually. The paper summarizes this as follows: “AMN maintains a more stable success rate during long-horizon inference, outperforming other policies by over 40% in final task completion” (Liu et al., 10 Sep 2025).

The numerical basis for that claim is explicit. In Clean Trash, final-stage success rises from 10% with the best baseline (ACT) to 50% with AMN, a +40 percentage point gain. In Deliver Tool, final-stage success rises from 2% with the best baseline to 44% with AMN, a +42 point gain (Liu et al., 10 Sep 2025). The paper’s abstract also states that AMN enhances long-horizon inference performance by approximately 40% (Liu et al., 10 Sep 2025).

A second result is interpretively important: PVE-DP alone does not solve the long-horizon problem. Although PVE-DP improves fine manipulation in short-horizon settings, its long-sequence behavior still collapses, reaching only 8% final success on Clean Trash and 0% on the final two stages of Deliver Tool (Liu et al., 10 Sep 2025). The paper uses this to argue that improved low-level control is insufficient without explicit long-horizon decomposition and routing.

The study also identifies a limitation directly. Future work will focus on “reducing gaps between subtasks in the AMN framework” (Liu et al., 10 Sep 2025). This indicates that transitions between expert policies remain imperfect. The paper does not report a dedicated ablation isolating the contributions of VLM decomposition versus subtask-specific policy training, nor does it provide AMN-specific inference speed, memory, or communication-overhead measurements (Liu et al., 10 Sep 2025). For reproduction and adaptation, this leaves the policy boundary and handoff mechanism as the principal underspecified components.

In the literature sampled here, the RoboMatch formulation is therefore the relevant reference for Auto-Matching Network proper: a VLM-guided long-horizon control architecture that converts language-plus-scene input into a sequential subtask plan and routes execution across specialized short-horizon policies, replacing brittle monolithic long-sequence inference with modular, stage-aware policy composition (Liu et al., 10 Sep 2025).