AhaRobot Ecosystem

• AhaRobot Ecosystem is an integrated, open-source framework for embodied AI that combines low-cost hardware, robust control methodologies, and advanced vision-language models for failure detection.
• Its innovative dual-arm mobile manipulator and counter-drive PD control strategy deliver precise, oscillation-free performance using affordable, off-the-shelf components.
• The ecosystem features RoboPilot teleoperation and a modular ROS2-based software stack for scalable data collection, imitation learning benchmarks, and reproducible research.

The AhaRobot Ecosystem is an integrated, open-source framework for embodied AI research that combines a low-cost dual-arm mobile manipulator (AhaRobot) with scalable software infrastructure, robust control methodologies, advanced failure-detection via vision-LLMs (VLMs), and seamless teleoperation and data-collection pipelines. Developed to democratize access to state-of-the-art embodied manipulation and failure reasoning, it emphasizes reproducibility, low hardware cost, and extensibility for reinforcement learning and imitation learning benchmarks in real-world environments (Cui et al., 13 Mar 2025, Duan et al., 2024).

1. Hardware Architecture

AhaRobot is a bimanual mobile manipulator constructed from off-the-shelf components, with a base hardware cost of $1,000 (excluding optional compute and power modules, which add up to an additional$1,000). The design features:

• Mobile Base: A differential-drive chassis with two front BLDC wheels (ODrive 3.6, Hall-effect encoders—64 counts/rev) and a rear passive caster, enabling full ground mobility and floor reach. The chassis uses aluminum T-slot framing for modular mounting.
• Lifting Slider: A belt-driven linear slide for Z-axis motion eliminates the need for expensive leadscrews. A photoelectric switch provides reliable homing.
• Dual SCARA-Style Arms: Each arm has four joints, each driven by two Feetech STS3215 motors with a 1:345 gearbox (35 kg·cm max torque), arranged in a counter-drive configuration. Magnetic encoders (4096 counts/rev) provide precise joint feedback.
• End Effectors: Simple parallel-jaw grippers (up to 120 mm aperture) are employed for manipulation tasks.
• Sensing: Three 640×360@30 Hz cameras (one panoramic head camera on a 2-DoF pan-tilt gimbal, one wrist camera per arm) provide dense visual feedback for embodied tasks.
• Computation and Power: A Mini-ITX PC (Intel i5-12700KF, NVIDIA RTX4060) is the core compute node; five ESP32 microcontrollers handle local PID/trajectory profiles. Power is provided by an onboard 24 V/20 Ah LiPo battery (providing 4–5 hr runtime) and an optional 1 kWh external Jackery for the PC. Inter-process communication is managed via ROS 2 Humble.

Compared to commercial platforms, AhaRobot achieves unique trade-offs: dual arms, full mobility, floor reach, and 16 DoF for $1,000–2,000 USD—versus$24,000–200,000 for alternatives—while maintaining open-source accessibility (Cui et al., 13 Mar 2025).

2. Control Methodology

Robust control in the presence of low-cost motors and high-ratio gearboxes necessitates countermeasures to mechanical backlash and static friction. The AhaRobot implements:

• Dual-Motor Counter-Drive Backlash Elimination: Each joint is actuated by two rigidly coupled motors. A constant bias torque $u_b$ is applied to compress gears in opposite directions, eliminating backlash:

$u_1 = u_o + u_b,\quad u_2 = u_o - u_b,$

where $u_o$ is the PID output. Experimental evidence shows oscillation-free direction reversals.

• Static Friction Compensation via Dithering: Friction is modeled as:

$$\tau_f = \begin{cases} \tau_s\,\mathrm{sgn}(\dot q) + \tau_v\,\dot q, & \dot q \neq 0,\ \tau_e, & \dot q = 0 \wedge |\tau_e| < \tau_s,\ \tau_s\,\mathrm{sgn}(\tau_e), & \text{otherwise} \end{cases}$$

A small alternating feed-forward term $u_d(t)$ at the PID update rate enables sub-degree step tracking without stiction-induced stalls:

$u_d(t) = (-1)^{\lfloor t/T\rfloor} u_b$

• PD Control with Friction Compensation: Each joint executes a standard PD loop:

$$\tau = K_p(\theta_{des} - \theta) + K_d(\dot\theta_{des} - \dot\theta) - b\,\mathrm{sign}(\dot\theta),\quad b = f_s + f_v\,\dot\theta$$

This control stack yields reliable fine-grained actuation using cost-efficient actuation components (Cui et al., 13 Mar 2025).

3. Teleoperation and Data Collection (RoboPilot)

RoboPilot is a web-based teleoperation interface designed for low-burden, fully remote human control and large-scale data collection:

• 6-DoF Handle Tracking: Each hand manipulates a 26-faced polyhedral AprilTag marker, eliminating pose ambiguity in PnP estimation. Pose is computed at 30 Hz in-browser (WebAssembly + OpenCV.js) with average errors of 2.1 mm (translation) and 1.1° (rotation).
• Foot-Pedal Module: Four Hall-effect pedals (ESP32+WebSerial) switch between base-drive and arm-operation; map to base velocity, gripper actuation, and lift control.
• Web Interface: Multiple live video streams (panorama and wrist) via WebRTC; all control state exchanges via WebRTC DataChannel. No VR headset required.
• Workflow:

1. Browser-based access to https://aha-robot.github.io/robo-pilot.
2. Marker and pedal data capture and transmission to robot.
3. Robotic inverse kinematics/velocity computation and teleoperation retargeting.
4. Demonstration logging (state, command, images).

Performance: RoboPilot delivers a 30% reduction in task completion time versus two-3D-mouse SpaceMouse and leader-follower methods, with marked improvements in success rate (100% vs 81.5%) across standardized manipulation benchmarks. It further enables seamless execution of long-horizon, multi-step real-world tasks in a single session (Cui et al., 13 Mar 2025).

4. Software Infrastructure and Open-Source Assets

The AhaRobot software stack, released under MIT license at https://aha-robot.github.io, is designed for modularity and reproducibility. Core components include:

• aha_description: URDF/SRDF models for both robot and teleop workstation.
• aha_driver:
  • esp32_firmware (motion profile/PID for arms, lift, and head),
  • odrive_driver (CAN-based mobile base control).
• aha_control:
  • dual_motor_node (counter-drive/dithering PD control implementation),
  • MoveIt2 integration for high-level trajectory planning.
• aha_teleop: Web client (TypeScript + WebAssembly) and ROS2 teleop_bridge node (WebRTC communication).
• aha_imitation: logging_node (18D state + images) and training scripts for ACT imitation learning policies.

The stack is ROS 2 Humble–native, leveraging OpenCV.js, AprilTag library, WebRTC (Janus/GStreamer), MoveIt!2, tf2, and rclcpp. This modular structure supports rapid prototyping, large-scale data collection, and easy extension for policy learning (Cui et al., 13 Mar 2025).

5. Vision-Language Reasoning and Failure Detection (AHA VLM)

AHA is a vision-LLM specifically adapted for free-form failure detection and explanation in robotic manipulation tasks (Duan et al., 2024). Salient features:

• Architecture:
  • Frozen ViT-based image encoder (e.g., CLIP-ViT),
  • Two-layer projector for aligning image features to language token space (Dv=1024 to Dlm=4096 dimensionality),
  • LLaMA-2-13B transformer decoder,
  • Training via LLAVA-style next-token prediction loss:

$$\mathcal{L}_{\text{LM}} = -\sum_{t=1}^{T} \log P_\theta(w_t \mid w_{<t}, I)$$

• Training Data: Co-fine-tuning on the Aha dataset (49,000 image–failure pairs produced with FailGen), 665,000 VQA pairs, and 100,000 LVIS detection pairs.
• Failure Taxonomy: Seven procedural types—No_Grasp, Slip, Translation, Rotation, No_Rotation, Wrong_Action, Wrong_Object—systematically generated using the FailGen framework.
• Evaluation: Benchmarked on Aha-Test, ManiSkill-Fail, and RoboFail datasets. Metrics include ROUGE_L, embedding cosine similarity, LLM-FuzzyMatch, and binary success rate. AHA-13B achieves 70.2% binary success on Aha-Test (vs. 61.1% for GPT-4o), and outperforms six other VLMs by an average of 35.3% across all metrics.

This model supports integration as a dense reward signal editor (Eureka+ Aha), feedback provider for task and motion planning (PRoC3S+Aha), and verifier in zero-shot keyframe generation (Manipulate-Anything+Aha), consistently improving downstream task success rates (up to +36.7%) (Duan et al., 2024).

6. Primary Use Cases and Demonstrations

AhaRobot has been empirically validated in real-world and simulated settings spanning teleoperated and autonomous modes:

• Remote Crowdsourced Teleoperation: Users successfully perform complex tasks (plate-in-rack, tube-in-rack, open-drawer+erase) with RoboPilot achieving 100% success rate and lower task times compared to legacy devices.
• Long-Horizon, Multi-Stage Tasks: Demonstrations include delivering coffee over a 200 m indoor path and executing multi-step kitchen tasks (fridge, microwave, object placement), leveraging vertical lift and coordinated bimanual manipulation.
• Imitation Learning and Policy Execution: ACT policies are trained from 50–80 teleoperated demonstrations, supporting tasks such as Pick Box (100% success), Insert Pen (60% success, cup tipping as failure mode), and Collect Toy (70–80% per substage). Position control proves more effective than velocity control for reliable base learning (Cui et al., 13 Mar 2025).
• Generalization in Vision-Language Failure Detection: AHA VLM generalizes from solely simulated fine-tuning to out-of-distribution real-robot failures (UR5), outperforming large proprietary models.

7. System-Level Impact and Recommendations

The AhaRobot Ecosystem reduces the cost and technical threshold for embodied AI research to under $2,000, enabling rapid, reproducible deployment across global research labs. Its key impacts are:

• Democratization of Embodied AI: Affordable hardware, straightforward build (~weekend replication), and cross-institutional benchmarking capability facilitate community-scale data collection and algorithm evaluation.
• Open-Ended Learning Benchmarking: Unified hardware and vision-language infrastructure standardizes real-world testing, breaking dependence on simulation and proprietary robotics.
• Failure-Aware Research Pipelines: Integration of scalable, taxonomy-driven failure data (via FailGen), advanced reasoning (AHA), and iterative reward/planning verification provides a robust substrate for learning robust, reliable policies.
• Limitations and Future Directions: The procedural failure taxonomy is presently limited to seven types and lacks real-world tactile/force modalities. Simulation-only fine-tuning could be augmented by on-robot or teleoperated failure collection. Scaling dataset size shows continued quadratic gains in generalization efficacy.

To maximize ecosystem research value, practitioners are advised to adapt FailGen for their simulators, co-fine-tune VLMs with VQA/detection data, freeze pre-trained vision backbones, use multi-view/multi-temporal context grids, and incrementally evaluate using held-out simulators and real-robot settings (Duan et al., 2024, Cui et al., 13 Mar 2025).