Human-Multirobot Mobile Manipulation

• Human-multirobot collaborative mobile manipulation is a framework integrating humans and multiple mobile manipulators to perform complex tasks through formal task representation and dynamic allocation.
• It leverages precise sensing, real-time communication, and reactive controllers, exemplified by frameworks like ConcHRC and Omnid mocobot with robust quantitative performance metrics.
• The approach ensures safe, compliant physical interaction via series elastic actuators and decentralized force control, supporting scalable coordination and load-sharing in diverse environments.

Human-multirobot collaborative mobile manipulation refers to systems and methodologies enabling teams of humans and multiple mobile manipulators to jointly perform physical tasks involving complex object manipulation, transportation, or assembly. These systems require tight integration of task representation, dynamic allocation, compliant physical interaction, feedback, and coordination strategies. Recent research has provided formal frameworks and experimental platforms that address these challenges for both structured industrial environments and less structured, human-centric scenarios.

1. Formal Task Representation and Allocation

Task-level collaboration in human-multirobot contexts relies on precise modeling of the space of possible subtask interleavings and agent assignments. The ConcHRC (Concurrent Human-Robot Collaboration) framework extends the FlexHRC paradigm using layered AND/OR graphs to capture task deconstruction and the dynamic interplay of concurrent and sequential dependencies (Karami et al., 2020). Each agent—whether a human or robotic manipulator—is associated with a subgraph, and their coordination is expressed through hyper-arcs (many-to-one transitions) and entangled nodes that link subtask completions across agents.

Nodes in these AND/OR graphs represent cooperation states, while hyper-arcs define ordered sets of actions with explicit precedence relations. Runtime feasibility of nodes and arcs is computed via Boolean flag logic, supporting dynamic selection of the next hyper-arc to minimize a cost metric such as estimated execution time. Human and robot primitives are then allocated to available agents according to their capabilities and ongoing system state.

This multi-layer representation enables hierarchical decomposition (using n-layer graphs) and parallel execution via a c-layer graph structure. Dependencies, such as mandatory handovers or mutual exclusion, are handled by node-entanglement constructs that formally link progress conditions between agent subgraphs.

2. Sensing, Recognition, and Human-Robot Communication

Effective interleaving of human and robot actions depends on accurate, low-latency state estimation and robust communication modalities. In the ConcHRC approach, human primitive actions (e.g., pick-up, put-down, ready) are recognized in real time from inertial data gathered by a wrist-worn smartwatch, using Gaussian Mixture Models and regression. Execution managers for robot agents are responsible for low-level control, enforcing hardware velocity and safety limits (joint velocity: 0.6 rad/s; mobile base: 0.4 m/s linear, 0.3 rad/s angular).

Action allocation to humans is accompanied by explicit prompts (visual/auditory), and humans may override suggested sequences by performing alternative, recognized gestures. Both robots and humans interact via a shared Knowledge Base, with explicit acknowledgment (done(a)) signaling the completion of each assigned primitive. Communication throughout the workflow is managed over ROS topics and services, supporting real-time updates and tight handover synchronization.

3. Compliant Multiagent Physical Interaction

Safe, high-fidelity human-multirobot physical collaboration, particularly for delicate or articulated payloads, requires passively compliant, force-sensitive manipulators and decoupled force/base control. The Omnid mocobot platform exemplifies this strategy, comprising a Mecanum-wheel omnidirectional base and a 3-DOF series-elastic Delta-type parallel manipulator with a passive gimbal wrist (Elwin et al., 2022). Series elastic actuators (SEA) provide well-characterized joint compliance (stiffness $k\approx60.1$ Nm/rad), precise force sensing (resolution 0.01 Nm), and limitation of peak interaction force for human safety.

The manipulator’s Cartesian stiffness is computed as $K(x) = J(\theta)^{-T} K_{\theta} J(\theta)^{-1}$, where $J(\theta)$ is the manipulator Jacobian. For teams of $N$ mocobots interacting with a shared payload, total compliance aggregates in the payload frame according to the spatial transforms and individual manipulator stiffnesses.

End-effector force control in the SEA Delta manipulator is executed at high bandwidth (20–30 Hz) and is structurally decoupled from mobile base odometry inaccuracies. The Omnid controller maps desired Cartesian force into joint torques ($\tau = J(\theta)^T f_{\text{com}}$), with commanded force comprising gravity compensation, payload load-sharing, and workspace boundary repulsion.

4. Multiagent Coordination and Communication Paradigms

Human–multirobot collaborative mobile manipulation exhibits differing coordination models, ranging from explicit, planner-mediated task allocation to fully decentralized, haptic-only approaches. In ConcHRC (Karami et al., 2020), a centralized task planner traverses the AND/OR collaboration graph, computes feasible progressions, and dispatches actions through explicit software communication. By contrast, the Omnid mocobot paradigm operates without wireless or central communication; mocobots and humans interact solely via mechanical couplings through the payload. Each robot locally solves its static equilibrium problem for load-sharing, and global coordination emerges from closed-loop dynamical coupling and the physical structure of the task (Elwin et al., 2022).

In this latter approach, safety and compliant coexistence are guaranteed by physical properties—series elasticity limits force magnitude and passive gimbal wrists filter unwanted torques. No vision, auditory, or external messaging is required for effective high-bandwidth intent exchange.

5. Experimental Results and Quantitative Metrics

Empirical evaluations of both paradigms confirm robust, efficient human-multirobot collaboration. In the ConcHRC product defect inspection scenario using a dual-arm Baxter and a Kuka youBot, five runs involving four parts each yielded average total scenario completion times of 246.75 s (Baxter pipeline) and 310.19 s (youBot pipeline), with task planning overhead below 1% of execution time. Most time was spent in physical manipulation and human gesture execution. Standard deviations under 7 s indicate repeatable, stable performance (Karami et al., 2020).

In the Omnid mocobot experiments, quantitative metrics of single-SEA performance include torque tracking error of ±2% FS (±9 Nm), deadband threshold under 1 N, settling time below 0.1 s for 5 Nm steps, and bandwidth up to 30 Hz. Three-mocobot/one-human teams achieved effortless 6-DOF manipulation of 15.6 kg rigid pipes, with successful 2 mm clearance insertions without prior training. Teams comprising three mocobots and two humans smoothly manipulated 7 DOF articulated payloads, achieving full manipulability rank with precise load-sharing. All human–robot interaction occurred physically via the payload, with no non-physical communication (Elwin et al., 2022).

6. Scalability, Generalization, and Design Constraints

The layered AND/OR graph formalism allows direct scaling to more agents: each new mocobot or human operator introduces additional graph “slots,” with interdependencies represented by entangled nodes. It is theoretically possible to expand ConcHRC-type systems to arbitrary team sizes, provided networked communication and synchronization constraints remain tractable. For mocobot teams, required team size is dictated by the required degrees of freedom of payload control: for an $n_p$-DOF payload, the aggregate system must satisfy $\mathrm{rank}(F_p(q))=n_p$ (Elwin et al., 2022). Compliance bounds must be carefully set: excessive overall stiffness undermines safety, while insufficient stiffness degrades tracking bandwidth.

Transferability to other collaborative mobile manipulation tasks—kitting, sorting, assembly—requires only redefinition of the AND/OR graph’s structure and cost models. A plausible implication is that cross-domain portability is straightforward in task-graph-based approaches. Future development directions include unifying the distributed planners under a centralized scheduler (e.g., with Answer Set Programming or metaheuristics) and extending human activity prediction for further reduction of robot/human idle time (Karami et al., 2020).

7. Significance and Research Directions

Human-multirobot collaborative mobile manipulation advances the automation of complex, physical joint actions—particularly in environments not amenable to full automation or requiring human flexibility. The integration of formal task graphs, compliant actuation, decentralized force control, and scalable coordination paradigms provides a generalizable foundation for both industrial and service robotics. Ongoing research topics include improved human intention prediction, variable impedance adaptation, optimal team formation, and multisensory feedback integration.

Key Papers:

• "A Task Allocation Approach for Human-Robot Collaboration in Product Defects Inspection Scenarios" (Karami et al., 2020)
• "Human-Multirobot Collaborative Mobile Manipulation: the Omnid Mocobots" (Elwin et al., 2022)