Papers
Topics
Authors
Recent
Search
2000 character limit reached

CuArms: GPU-Accelerated Multi-Arm Manipulation

Updated 2 November 2026
  • CuArms is a GPU-accelerated planner for multi-arm manipulation, which coordinates multiple robotic arms sharing a workspace. It uniquely uses per-robot multi-modal roadmaps, Lazy Conflict Avoidance Method (LaCAM), and Priority Inheritance with Backtracking (PIBT) to ensure efficient and collision-free trajectories.
  • CuArms works in a high-dimensional problem space, involving static obstacles, grippers, and defineable relationships between arms and objects. The scheduler and a high-level decision process allow for dynamic replanning and re-evaluating of current plans to adjust schedules for efficiency and feasibility.
  • Key use cases include a schedule implementation for multi-armed welding (like RoboBallet welding) and dexterous tabletop manipulation of many objects. The algorithm improves initial solution quality by 25% to 47% and provides a 40% to 57% reduction in makespan.

cuArms is a GPU-accelerated conflict-based planner for multi-arm manipulation. It coordinates multiple robot arms sharing a workspace by combining reusable per-robot multi-modal roadmaps, lazy conflict resolution based on LaCAM and PIBT, GPU-batched geometric validation, and hierarchical task-and-motion planning. A high-level scheduler proposes task skeletons specifying assignments, action order, manipulation skills, and precedence constraints; cuArms grounds those skeletons into collision-free, timed, and progressively refined multi-arm trajectories (Huang et al., 26 Oct 2026).

1. Problem formulation and planning scope

cuArms addresses multi-arm manipulation in a workspace W\mathcal W containing nn robot arms, static obstacles, and movable rigid objects. Each robot rir_i has a configuration space CiC_i, joint-position and joint-velocity limits, and a known base-to-world transform. Robot links, grippers, and carried objects are represented as unions of collision spheres, enabling batched forward kinematics and collision testing on the GPU.

A complete geometric state includes the configuration of every robot and a discrete manipulation mode:

s=(q1,…,qn,m).s=(q_1,\ldots,q_n,m).

The mode m∈Mm\in\mathcal M specifies the attachment relation of every movable object to either the world or a robot frame. It may represent an object on a table, grasped by one robot, transferred during a handover, or jointly attached to two robots during a bimanual carry. Robot configurations and modes together determine the collision geometry and attachment structure.

The planner supports two principal action classes. A move is a single-robot motion executed in a fixed mode. A manipulation skill is a parameterized, contact-rich action involving one or two arms. Its parameters include entry and exit configurations, a minimum duration, and a bounding volume enclosing the participating arms and manipulated objects:

xj=(qjin,qjout,dj,Bj).x_j=(q_j^{\mathrm{in}},q_j^{\mathrm{out}},d_j,\mathcal B_j).

Each skill induces a mode transition,

mout=Taj(min),m^{\mathrm{out}}=T_{a_j}(m^{\mathrm{in}}),

such as picking, placing, handing an object to another robot, or initiating a bimanual carry. In the roadmap representation, a skill is a special edge connecting its entry and exit configurations while applying the corresponding mode transition.

A schedule skeleton specifies the discrete structure that cuArms must geometrically realize:

Σ=({(a^j,Rj,xj)}j=1M,≺Σ).\Sigma=\left(\{(\hat a_j,\mathcal R_j,x_j)\}_{j=1}^{M},\prec_\Sigma\right).

Here, a^j\hat a_j denotes a move or manipulation skill, nn0 is the set of participating robots, nn1 contains fixed skill parameters, and nn2 is a precedence relation. The relation includes per-robot ordering and cross-robot task constraints. Actions on different robots may execute asynchronously if no precedence relation orders them.

The skeleton determines robot participation, action ordering, grasps, placements, handover poses, inverse-kinematic solutions, and mode-transition semantics. It does not determine collision-free coordination. Given a workspace and skeleton, cuArms produces timed trajectories satisfying duration, precedence, connectivity, and collision-avoidance constraints. Collision avoidance applies to robot–obstacle, robot–robot, robot–object, carried-object–obstacle, and active-skill-volume interactions.

2. Objectives and system architecture

The objective combines temporal completion and motion costs:

nn3

The makespan is

nn4

where nn5 is the completion time of robot nn6. The sum-of-completion-times term is

nn7

The motion-distance term may use joint-space path length, Cartesian or task-space path length, volume-weighted distance traveled by collision spheres, or a weighted combination. The initial LaCAM-style search seeks feasible solutions rapidly rather than global optimality. Subsequent shortcutting and evaluation of alternative schedule skeletons provide anytime improvement.

The overall architecture separates task-level reasoning from continuous multi-arm coordination:

nn8

The scheduler determines task allocation, action ordering, temporal constraints, robot participation, grasp and placement choices, handover poses, and inverse-kinematic solutions. It may be ScheduleStream, a domain-specific optimization program, a fixed-order assembly planner, or a greedy scheduler. cuArms then grounds each proposed skeleton by constructing or retrieving roadmaps, updating mode validity, adding skill edges, running conflict-based search, and refining the resulting trajectories.

A typical grounding sequence is:

  1. Construct or retrieve reusable per-arm base roadmaps.
  2. Update mode-validity masks.
  3. Add skill entry and exit nodes.
  4. Add skill edges and mode transitions.
  5. Compute action heuristics.
  6. Run joint conflict-driven search.
  7. Backtrack the joint path into timed trajectories.
  8. Apply GPU-batched shortcutting.
  9. Evaluate the objective.
  10. Retain the best plan found so far.

If a skeleton is infeasible, the scheduler may generate another. If it is feasible but inefficient, cuArms can refine it while alternative skeletons are evaluated.

3. Reusable multi-modal roadmaps

For each robot nn9, cuArms constructs a probabilistic roadmap

rir_i0

where nodes are robot configurations and edges are local motions. The base roadmap contains sampled configurations, collision-free local connections, and the initial configuration. Base nodes and edges are checked against the static environment and stored in GPU memory.

The planner deliberately avoids constructing a full joint configuration-space roadmap. Geometry associated with an individual robot and the static environment can instead be reused across different task skeletons, object placements, schedules, and multi-arm combinations. This decomposition is particularly relevant when a fixed workcell supports many manipulation instances.

A base roadmap edge may become invalid when the robot enters a different manipulation mode. For example, an attached object may collide with an obstacle or another object even though the bare robot configuration was valid. cuArms handles this through a mode-validity mask rir_i1 for each robot and mode. The mask records which roadmap nodes and edges remain valid under the mode without reconstructing the base roadmap.

Collision checking is decomposed into reusable and mode-dependent components:

  1. Robot forward-kinematic geometry and robot–static-obstacle checks are reused.
  2. Robot–object and attached-object checks are evaluated in batches.
  3. The results are stored as masks over the existing roadmap.

When new modes appear, cuArms updates the masks while preserving the base roadmap.

For each manipulation skill, the planner adds an entry node, an exit node, and a skill edge connecting them. The edge represents contact-rich execution, its bounding volume, and its mode transition. Skill durations are rounded up to whole-number multiples of the maximum roadmap-edge interval, placing skill execution and ordinary roadmap motions on a common discrete time grid.

The planner also computes cost-to-go heuristics rir_i2 for each robot, action, and roadmap node. It temporarily ignores mode-validity masks, runs backward Dijkstra from the target set of each action, assigns unit costs to motion and skill edges, and adds estimates of remaining motion and skill costs. When precedence constraints prevent a robot from progressing toward its nominal action, a separate cost-to-home heuristic can direct it toward a home or retreat configuration for deadlock resolution.

cuArms builds on Priority Inheritance with Backtracking (PIBT) and Lazy Constraint Addition for Multi-Agent Coordination (LaCAM). PIBT generates locally conflict-free joint successors through dynamic priorities, priority inheritance, recursion, and backtracking. LaCAM performs lazy depth-first search, storing constraints and exploring alternative joint successors without enumerating every combination of robot actions.

A joint search state is represented as

rir_i3

where rir_i4 is the current roadmap node of robot rir_i5 and rir_i6 is its symbolic action or stage. The state implicitly determines the current mode, completed actions, available actions, and satisfied precedence constraints. Each state also stores a constraint queue, a parent pointer, and the path required for backtracking.

At a state rir_i7, each robot receives a candidate set rir_i8 containing roadmap motions, waits, manipulation skills, and possibly yield or retreat actions. A skill is available only when all participating robots have reached their entry configurations, the relevant precedence constraints are satisfied, and the mode is valid. Candidates are ranked lexicographically:

rir_i9

Skills are therefore prioritized over ordinary movement, while lower heuristic cost is preferred within each class.

GPU conflict-table construction

The central acceleration mechanism is a GPU-generated conflict table:

CiC_i0

For candidate actions from robots CiC_i1 and CiC_i2, the table records whether their geometries conflict. Ordinary moves use aligned robot trajectories, while manipulation skills use their enclosing bounding volumes CiC_i3. All relevant inter-robot candidate pairs are batch-checked on the GPU before PIBT recursion.

PIBT then performs CPU-side recursive control flow by reading entries in CiC_i4, rather than launching repeated collision kernels. This avoids repeated GPU kernel launches and repeated CPU–GPU synchronization during priority inheritance. The separation of GPU geometric computation from CPU recursive decision-making is a central design feature of cuArms.

Priority inheritance and deadlock resolution

PIBT processes robots according to dynamic priorities. For a robot selecting an action, cuArms considers ranked candidates, rejects candidates conflicting with already committed higher-priority actions, and recursively asks a blocking robot to select an alternative when necessary. If no alternative exists, the search backtracks.

This mechanism supports temporary priority reversals and cannot be reduced to simple retiming. In an object-swapping scenario, one robot may yield so another can pick, after which the second robot retreats while carrying its object and the first robot proceeds. The effective priority order can therefore change during execution.

Deadlocks may arise from blocked approach paths, an arm holding an object, mutually incompatible priorities, or tightly coupled bimanual and handover operations. cuArms addresses them through priority inheritance, backtracking, lazy alternative constraints, waits, candidate ranking, yield motions, cost-to-home heuristics, and symbolic precedence constraints.

Bimanual skills commit the participating robots jointly. Both arms must reach the skill entry configurations, synchronize for execution, apply the shared mode transition, and reach the corresponding exits. This supports coordinated pick–carry–place operations and related coupled manipulation behaviors.

5. Anytime refinement and scheduler interaction

The first feasible solution may be inefficient because of coarse roadmaps, nonoptimal LaCAM search, or a suboptimal schedule skeleton. cuArms therefore performs batched shortcutting.

For each move trajectory, it enumerates a bounded set of straight-line joint-space interpolation candidates. Each candidate is checked against static obstacles, movable objects, and the current mode. Valid single-arm shortcuts are retained. The planner then samples batches of multi-arm shortcuts from these candidates.

A multi-arm shortcut may consist of a single-arm shortcut while other arms retain their trajectories, a prioritized shortcut, or a composite shortcut involving multiple concurrent single-arm shortcuts. Inter-arm conflicts are checked in parallel. Valid shortcuts are sorted by cost improvement, and non-overlapping shortcuts are greedily combined to replace segments of the original plan.

Because shortcut validation respects the current mode and the trajectories of the other arms, refinement does not optimize each robot independently and subsequently introduce new collisions. The full weighted objective may improve even when makespan increases, because cuArms can trade temporal completion against path-distance cost.

The scheduler may generate a batch of skeletons or request additional skeletons after cuArms fails or returns a poor plan:

CiC_i5

The resulting hierarchical process is:

CiC_i6

This separation assigns symbolic allocation, ordering, and manipulation-mode sampling to the scheduler, while assigning tightly coordinated continuous motion to cuArms.

6. Evaluation, applications, and limitations

The reported experiments used an NVIDIA RTX Pro 6000 GPU workstation. Most simulation measurements used 15 random seeds and reported medians and interquartile ranges. The evaluations covered RoboBallet welding, tabletop multi-arm task-and-motion planning, and real-world demonstrations with Flexiv Rizon4 arms.

RoboBallet welding

The RoboBallet environment contains four or eight robot arms, 10 or 40 spot-welding targets, obstacles, and a shared constrained workspace. cuArms used a greedy scheduler that repeatedly assigned targets through Hungarian matching, sampled joint target configurations using GPU-batched inverse kinematics, constructed schedule skeletons, and invoked cuArms. Sixteen schedule skeletons were evaluated.

The workspace remained fixed while weld targets changed, allowing base-roadmap reuse. New targets were connected to the roadmap using RRT-guided steering. Refinement budgets were two seconds per skeleton for 10-target tasks and six seconds per skeleton for 40-target tasks.

Setting Solved Initial solution Initial makespan Final makespan
10 targets, 4 arms 150/150 0.20 s 10.61 s 5.64 s
40 targets, 4 arms 150/150 0.41 s 31.16 s 16.56 s
40 targets, 8 arms 150/150 0.87 s 25.35 s 14.87 s

The paper reports real-time initial solutions, 100% success on the listed cuArms instances, approximately 47% reduction in initial makespan after shortcutting and alternative-schedule search, median makespan improvements over the RL baseline of 20.1% for 10 targets and 27.9% for 40 targets on commonly solved instances, and a 10.2% reduction in median makespan when increasing from four to eight arms in the 40-target setting.

The retrained RoboBallet policy solved 29 of 150 10-target episodes and 50 of 150 40-target episodes. Reported failures involved collisions, deadlocks, or failure to meet RoboBallet’s goal tolerances of 2.5 cm and CiC_i7.

For four arms and 10 targets, collision checking accounted for 18.1% of total runtime. The corresponding shares were 24.7% for four arms and 40 targets and 50.2% for eight arms and 40 targets. Roadmap construction and target connection can become a bottleneck in narrow, obstacle-constrained regions, while conflict-table construction becomes a major runtime component as the number of arms increases.

Tabletop multi-arm task-and-motion planning

The tabletop experiments used two Panda arms for holding, stacking, packing, and unpacking tasks. Principal tasks involved two to four cubes, with scaling experiments extending to six and eight objects. ScheduleStream generated symbolic skeletons and sampled grasps, placements, and inverse-kinematic solutions. The total high-level search budget was 60 seconds.

ScheduleStream planned each arm independently and then retimed trajectories. cuArms jointly coordinated motions and skills, supported asynchronous execution, resolved conflicts during planning, and refined motion quality.

Task cuArms solved Initial time Initial objective Final objective ScheduleStream solved
Hold 45/45 0.48 s 5.46 s 2.37 s 44/45
Stack 45/45 2.26 s 12.84 s 6.67 s 38/45
Pack 45/45 2.96 s 12.02 s 6.96 s 38/45
Unpack 45/45 1.60 s 11.50 s 6.55 s 39/45

cuArms produced initial solutions in 0.48–2.96 seconds, achieved 100% success on the reported instances, had an average initial-solution speed advantage of 46.4% over ScheduleStream, and reduced median makespan by 42%–57% through shortcutting and alternate skeletons. ScheduleStream failed to find solutions for some eight-object packing cases.

Real-world demonstrations

The real-world system used two Flexiv Rizon4 arms and ScheduleStream-based symbolic skeleton generation. Demonstrations included automatically sampled and optimized handovers with object reorientation, as well as a bimanual carry involving collision-free approach around camera mounts, coordinated grasping, and transport of a busbar fixture into a server tray. The paper states that poses and carry trajectories were sampled rather than hard-coded.

The supplied results do not give measured real-robot execution success rates, cycle times, trial counts, controller frequencies, calibration details, hardware safety margins, or perception-system details.

Limitations

cuArms depends on roadmap coverage. A coarse roadmap can miss feasible narrow passages, impose long detours, or make a feasible task appear infeasible. The LaCAM-style search is not claimed to provide global optimality, and anytime shortcutting improves quality without establishing optimality.

The planner coordinates a supplied schedule skeleton but cannot generally repair an incorrect task assignment, unsuitable grasp, impossible placement, inappropriate participating-robot set, or poor skill parameterization. These deficiencies require alternative skeletons from the scheduler.

The GPU conflict table may grow substantially with the number of robots and candidate actions. The eight-arm results show that conflict-table construction can become a dominant runtime component. Collision geometry is approximated by collision spheres and skill bounding volumes; their accuracy and conservativeness affect feasibility and solution quality.

The approach is most suitable when several arms share a constrained workspace, tasks are long-horizon, manipulation modes change, asynchronous execution is useful, temporary yielding is required, the workspace is reused across instances, and GPU resources are available for batched geometric computation. It is less suitable when geometry is highly dynamic or unmodeled, reliable skill samplers and skeleton generators are unavailable, feasible paths require difficult-to-sample narrow passages, exact global optimality is required, or the principal challenge is discovering new manipulation strategies rather than coordinating supplied skills.

cuArms is therefore best characterized as a GPU-accelerated conflict-driven grounding layer for multi-arm task-and-motion planning: a scheduler specifies what should happen, while cuArms determines how multiple arms can execute it collision-free, asynchronously, and with progressively improved motion quality.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (1)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to cuArms.