RoboAuto: Autonomous Robotic Systems

• RoboAuto is a versatile research platform for autonomous robotic systems integrating modular sensing, ROS or ROS2 middleware, and decision-making capabilities, with applications in observatory control, vehicle navigation, and manufacturing, among others.
• The architecture of RoboAuto platforms typically includes layers for mission intent, task management, mapping, planning, control, monitoring, and human supervision, distinctly separating scientific or operational goals from executable tasks.
• Key features of RoboAuto systems encompass autonomous decision-making with robust recovery mechanisms, closed-loop control, and persistent state monitoring, ensuring compliance with safety protocols under a human-supervised fallback model.

RoboAuto is a generic designation for an autonomous robotic-vehicle research platform or system architecture rather than a single uniformly specified implementation. Across the associated research, the term encompasses autonomous observatory control, mobile-robot navigation, autonomous driving, robotic inspection, CAD-to-robot assembly, heterogeneous ROS-based computation, teleoperation, ethical negotiation, and closed-loop robot manipulation. Its recurring architectural characteristics are modular sensing and actuation, ROS or ROS2 middleware, autonomous decision-making, human-supervised fallback, persistent state and monitoring, simulation-to-real transfer, and explicit treatment of safety, uncertainty, and recovery.

1. Conceptual scope and historical development

The earliest operational model associated with RoboAuto is Robo-AO, an autonomous laser-adaptive-optics observatory instrument. Robo-AO integrated telescope control, laser propagation, adaptive-optics correction, target acquisition, observing-sequence execution, data acquisition, monitoring, and fault recovery into a largely unattended robotic system. It began operation at the Palomar Observatory 60-inch telescope in June 2012 and had completed 78 science nights and more than 9,000 robotic observations by August 2013. Its average target-to-target operational overhead was 86 seconds, enabling approximately 20 observations per hour (Baranec et al., 2013).

Robo-AO’s importance to the RoboAuto concept lies in its integration of scientific intent, scheduling, hardware control, safety, and fault recovery. Its software used independent subsystem daemons, continuous status reporting, automatic daemon restart, and a central robotic control daemon. The resulting architecture treated observing as a recoverable stateful workflow rather than as a collection of manually executed instrument commands.

The subsequent Robo-AO queue system formalized the separation between scientific intent and operational execution. Scientists described programs and targets in XML; the queue eliminated invalid or unsafe candidates, computed priority weights, returned the highest-weight object, and persisted completion state after each decision. Its scoring model combined slew efficiency, airmass, scientific priority, and time criticality. The scheduler processed approximately 12,000 targets in about 2.5 seconds and supported an average observing rate of approximately 20 targets per hour (Riddle et al., 2014).

Later work broadened the architecture from observatory automation to autonomous vehicles and robots. ROS-based mobile-robot systems combined Android teleoperation with sonar-based wall following (Pinheiro et al., 2015). AutoRally demonstrated a reproducible 1:5-scale autonomous-driving platform with onboard sensing, dynamics identification, state estimation, visual perception, and model-predictive control (Goldfain et al., 2018). Roboat II extended the pattern to a holonomic autonomous surface vessel using LiDAR–IMU–GPS SLAM, nonlinear moving-horizon estimation, nonlinear model-predictive control, and path planning (Wang et al., 2020).

The term RoboAuto therefore denotes an architectural family. Its common objective is the transformation of high-level goals into autonomous, sensor-driven, monitored, and recoverable physical action across different robotic platforms.

2. System architecture and software abstractions

A RoboAuto system is most naturally organized as a layered architecture:

Mission or scientific intent ↓ Scheduling, task allocation, or behavior management ↓ Mapping, localization, perception, and world modeling ↓ Planning and trajectory generation ↓ Control, actuation, and hardware interfaces ↓ Monitoring, verification, logging, and recovery ↺ Human supervision, teleoperation, and external coordination

The boundaries between layers are explicit in several systems. Robo-AO separated queue scheduling from robotic sequencing: the queue decided what should be observed and with what parameters, while the sequencing system executed the observation. The Cambridge RoboMaster platform similarly separated planners and learned policies from the Freyja state-estimation and optimal-feedback control stack, allowing learned policies to output desired velocities while lower-level software generated wheel commands (Blumenkamp et al., 2024).

Auto-Assembly applies the same principle to manufacturing. Its pipeline proceeds from CAD design through design analysis, assembly-sequence generation, tooling and cell matching, bill-of-process generation, robot control-code generation, motion planning, simulation, and physical execution (Zobov et al., 2023). The bill of process acts as an intermediate representation between product-level intent and robot-specific execution.

For autonomous driving, AutonomROS divides functions between CPU software and FPGA-capable ROS2 nodes. Localization, vehicle communication, cruise control, and Navigation 2 remain software components, while point-cloud generation, obstacle detection, and lane detection can execute in programmable logic through ReconROS (Lienen et al., 2023). This preserves ROS2 interfaces while allowing computationally intensive perception functions to be accelerated.

A generalized RoboAuto interface should distinguish:

• Intent: goals, scientific programs, missions, destinations, or assembly objectives.
• Tasks: semantically meaningful operations such as entering an intersection, using a charging station, taking an elevator, picking a part, or acquiring an image.
• Actions: executable primitives such as steering, braking, grasping, placing, moving, sensing, or exposing a camera.
• Resources: robots, tools, charging stations, lanes, elevators, observation time, or protected airspace.
• Constraints: safety, legal, temporal, environmental, geometric, and hardware restrictions.
• State: completed, active, failed, uncertain, recovering, or aborted operations.
• Evidence: sensor observations, health telemetry, calibration provenance, and verification results.

The AutoInspect system illustrates this abstraction at mission scale. It separates platform-independent mapping, localization, topological navigation, mission execution, scheduling, monitoring, and user interfaces from robot-specific interfaces for odometry, navigation commands, cameras, radiation measurements, and docking (Staniaszek et al., 2024). New robots need a localizer, an edge-traversal interface, and registered action capabilities rather than a complete redesign of the mission layer.

3. Perception, localization, planning, and control

Perception and world modeling

RoboAuto platforms use heterogeneous sensing appropriate to their domain. The JKU-ITS Automobile integrates multilayer LiDAR, cameras, radar, GPS/GNSS, wheel encoders, and a 9-DOF IMU through ROS1 and a Toyota RAV4 research vehicle (Certad et al., 2023). AutoRally combines GPS, IMU, wheel-speed sensors, global-shutter cameras, and battery and temperature monitoring (Goldfain et al., 2018). Roboat II uses 3D LiDAR, IMU, GPS, and a camera, although the camera is not used in the reported SLAM formulation (Wang et al., 2020).

The perception pipeline can be accelerated when data-parallel operations dominate. AutonomROS converts depth images into point clouds, filters and projects points into an obstacle grid, and performs lane detection using FPGA-capable ROS nodes. With Iceoryx shared-memory transport, point-cloud generation reached 29–30 FPS with 11–15 ms turnaround in software and 17–18 ms in hardware, while hardware obstacle detection reduced CPU use from 50–60% to 4–8% at 29–30 FPS (Lienen et al., 2023).

Localization and mapping

Localization architectures range from probabilistic particle filters to factor graphs and model-based estimators. AUTONAV uses Google Cartographer for 2D SLAM and ROS AMCL for localization against a known occupancy-grid map (Sarwar et al., 10 Apr 2025). Roboat II uses incremental factor-graph smoothing with LiDAR odometry, GPS factors, and loop closures. In a canal experiment involving 107,656 LiDAR scans and approximately 19,065 m of trajectory, the reported loop-return translation error was 0.17 m and average keyframe registration plus graph-optimization runtime was 79.3 ms (Wang et al., 2020).

AutoRally uses factor-graph optimization to combine GPS and IMU measurements while estimating position, velocity, and inertial biases. Its graph is optimized incrementally with iSAM2, retaining states at 10 Hz and interpolating the estimate to publish a 200 Hz state estimate (Goldfain et al., 2018). Robo-AO’s robotic observatory uses a different localization problem, but its laser acquisition procedure demonstrates the same principle of quantitative state validation: acquisition is accepted when 80% of wavefront-sensor subapertures exceed 75% of typical laser-return flux (Baranec et al., 2013).

AutoInspect combines metric mapping with a topological graph. Nodes represent named locations and edges represent traversable connections with costs, traversal types, special behaviors, and operational constraints. When an edge becomes blocked, it is temporarily deactivated and the shortest path is recomputed (Staniaszek et al., 2024). This hybrid representation supports repeatable inspection missions while remaining editable by operators.

Planning

Planning mechanisms include queues, graph search, symbolic satisfiability, nonlinear optimization, model-predictive control, and learned policies. The Robo-AO queue eliminates hard-ineligible targets before applying soft priority weights. AUTONAV compares Z3-based SMT planning with BFS and A* over discretized occupancy maps. Its SMT formulation represents paths as waypoint sequences constrained by initial state, goal reachability, obstacle separation, and movement bounds (Sarwar et al., 10 Apr 2025).

For dynamic systems, geometric paths are insufficient. Roboat II uses nonlinear model-predictive control over a 4-second horizon with thruster constraints of $\pm 50$ N, while AutoRally uses model predictive path integral control for aggressive driving, including wheel slip, drifting, and nonlinear tire behavior (Wang et al., 2020, Goldfain et al., 2018). The Cambridge RoboMaster platform uses a learned multi-agent policy to produce desired velocities, followed by the Freyja controller for state estimation, trajectory tracking, and wheel-speed generation (Blumenkamp et al., 2024).

Control

Control is generally separated from higher-level planning. The Auriga 1:10 autonomous vehicle uses a bicycle model, Stanley steering, PID control, Pure Pursuit, and a finite-state machine for lane following, parking, intersections, traffic lights, and obstacle avoidance (Holighi et al., 2024). Roboat II couples nonlinear moving-horizon estimation with nonlinear model-predictive control, allowing estimated dynamic states to initialize constrained control optimization (Wang et al., 2020).

In manipulation, RoboPilot uses typed primitive actions and closed-loop execution verification rather than allowing a LLM to directly generate unrestricted low-level robot control. Its monitor checks action validity and parameters before execution, compares expected and observed outcomes afterward, and triggers recovery or replanning when displacement exceeds a 2 cm threshold (Liu et al., 30 Sep 2025).

4. Autonomy, scheduling, and human supervision

Autonomous scheduling

RoboAuto scheduling converts priorities and constraints into executable decisions. Robo-AO’s queue represents programs, targets, objects, and observations hierarchically. A target can include required visits, cadence, time windows, seeing limits, and metadata; an object contains coordinates and motion information; an observation contains exposure time, adaptive-optics requirements, filter, camera mode, and repetition count (Riddle et al., 2014).

The scheduler first eliminates candidates violating declination, hour-angle, airmass, solar, lunar, time-critical, laser-closure, weather, dome, or system-status constraints. It then applies a configurable score:

$$\text{total weight} = a\cdot\text{slew weight} + b\cdot\text{airmass weight} + \text{scientific priority} + \text{time criticality}.$$

The same architectural distinction applies to mobile and industrial RoboAuto systems. AutoInspect schedules repeated inspection missions, suppresses missions when battery state is low, and sends the robot to the nearest charging station. Its deployments at RACE B1 and the JET fusion reactor lasted 49 and 35 days, respectively, demonstrating that long-term autonomy requires resource scheduling and operational recovery in addition to navigation (Staniaszek et al., 2024).

Human-in-the-loop operation

Autonomy does not eliminate human involvement. Robo-AO automated routine telescope, laser, adaptive-optics, acquisition, monitoring, and recovery tasks, while people defined survey targets, allocated time, configured programs, maintained hardware, and handled exceptional conditions (Baranec et al., 2013). The Pioneer 3-DX system similarly combines autonomous wall following with Android teleoperation for critical situations (Pinheiro et al., 2015).

TUM Teleoperation provides a modern ROS2 architecture for Remote Driving and Trajectory Guidance. Direct Control sends steering, speed, pedal, gear, indicator, and horn commands, whereas Trajectory Guidance allows an operator to specify a path and velocity profile that the vehicle’s onboard controller executes (Kerbl et al., 16 Jun 2025). In the reported setup, LTE glass-to-glass video latency had a median of approximately 160 ms, while command latency was approximately 15.5 ms over LTE for both TCP and UDP.

A RoboAuto system should therefore distinguish:

• Autonomous execution: routine operation under validated policies.
• Remote assistance: human selection or approval of high-level goals, maneuvers, or trajectories.
• Remote driving: direct operator control of vehicle motion.
• Safe stop: autonomous transition to a non-actuating or otherwise safe state.
• Maintenance and recovery: intervention after hardware, software, or environmental failure.

The command arbiter must ensure that human commands, autonomous commands, and safety overrides cannot produce contradictory actuation.

5. Safety, security, ethics, and recovery

Fault tolerance and operational safety

RoboAuto safety is distributed across hardware interlocks, software supervisors, environmental monitors, and recovery procedures. AutoRally uses a wireless deadman relay, a PWM multiplexer, and a software runstop, although it does not provide a true power-disconnecting emergency stop (Goldfain et al., 2018). Robo-AO combines weather safety, laser shutdown, predictive satellite deconfliction, subsystem health reporting, daemon restart, and procedural recovery (Baranec et al., 2013).

AutoInspect’s long-term deployments reveal the importance of non-obvious failure modes. At RACE B1, a localization failure emerged after extended operation because of CPU throttling associated with heat buildup. At JET, fatal interventions were attributed to a user-facing process manager that shut down ROS nodes when operators reconnected to the system (Staniaszek et al., 2024). These results demonstrate why long-duration testing, thermal monitoring, operator-workflow testing, and failure injection are required.

Cybersecurity

RoboMal addresses static malware detection for robotic controller executables. It extracts up to 2,000 bytes from the .pydata section of ELF files and classifies the resulting byte sequence with a bidirectional LSTM. On a dataset of 450 controller executables, the reported 10-fold cross-validation results were 85% accuracy, 87% precision, 87% recall, and 87% F1 (Kaur et al., 2022).

RoboMal is relevant as a pre-execution admission-control layer but is not a complete cybersecurity architecture. Its dataset contains controller executables for a small-scale autonomous car, and its extraction method depends on a particular ELF layout and .pydata section. Secure boot, signed updates, hardware-backed keys, provenance verification, rollback protection, runtime monitoring, sandboxing, and fail-safe recovery remain necessary.

Ethical negotiation

RobEthiChor adds context-aware negotiation to robots that represent users with different preferences. It distinguishes hard ethics—laws, safety constraints, and organizational rules—from soft ethics—user-specific preferences among otherwise feasible alternatives. Robots exchange offers containing tasks and selectively disclosed user-status conditions, then evaluate received offers using local ethical profiles rather than exchanging complete profiles (Memon et al., 30 Jul 2025).

In controlled TurtleBot 4 experiments, agreement rates were 77.8% in an airport context and 73.3% in a hospital context, with an average negotiation time of 0.672 seconds. The system is bilateral and based on manually specified profiles. It does not establish fairness, legal compliance, strategic robustness, or safety in real traffic.

For RoboAuto, ethical negotiation must remain subordinate to hard safety and legal constraints. It may select among safe alternatives such as lane access, charging priority, loading-zone use, or scheduling, but it should not directly override collision avoidance, emergency rules, or certified control constraints.

Recovery and infeasibility recognition

Closed-loop recovery is a defining characteristic of the RoboAuto architecture. RoboPilot’s fast and slow thinking modes select between rapid primitive planning and more deliberate reasoning for spatially complex or ambiguous tasks. Its benchmark includes error recovery and infeasibility recognition; the combined system achieved 92.5% average simulation success, compared with 66.6% for Instruct2Act* (Liu et al., 30 Sep 2025).

The distinction between failure and infeasibility is important. A failed action may be retried or replanned; an infeasible task should terminate with an explicit status; a safety failure should stop and request authorization. A robust RoboAuto state machine should therefore include at least:

planned → ready → executing → verified ↓ uncertain ↓ recovering ↓ completed / failed / infeasible / aborted

6. Research applications, evaluation, and future directions

RoboAuto architectures support several application domains.

• Autonomous observatories: Robo-AO demonstrated more than 9,000 robotic observations, visible diffraction-limited imaging, large stellar-multiplicity surveys, and high-volume Kepler-object validation (Baranec et al., 2013).
• Autonomous road vehicles: AutoRally, the JKU-ITS Automobile, Auriga, AutonomROS, and OpenPodcar2 provide platforms for perception, localization, control, energy studies, human interaction, and hardware–software co-design (Goldfain et al., 2018, Certad et al., 2023, Holighi et al., 2024, Lienen et al., 2023, Soni et al., 27 Apr 2026).
• Multi-robot systems: The Cambridge RoboMaster platform supports onboard optimal control, peer-to-peer communication, visual localization, distributed SLAM, and multi-agent reinforcement learning. It demonstrated a maximum speed of 4.45 m/s, approximately 5 m/s² acceleration, and onboard GPPO inference in about 0.5 ms (Blumenkamp et al., 2024).
• Autonomous inspection: AutoInspect completed 84 missions at RACE B1 and 81 at JET, with mean times between non-minor interventions of 78 and 140 hours, respectively (Staniaszek et al., 2024).
• Robotic manufacturing: Auto-Assembly generated candidate assembly sequences from CAD, matched tooling and cells, produced bills of process, simulated execution, and transferred generated process code to a two-robot physical cell (Zobov et al., 2023).
• Cooperative traffic and human interaction: ROBOPOL combines social robotics, V2X, pedestrian interaction, and formal traffic reasoning. Its proof of concept used ARI to detect a pedestrian, receive CAMs from an e-bike, issue gestures and speech, and transmit a DENM warning (Bied et al., 30 Dec 2025).
• Remote operation: TUM Teleoperation provides a modular ROS2 substrate for direct control, trajectory guidance, operator visualization, monitoring, and integration with automated-driving frameworks (Kerbl et al., 16 Jun 2025).

Evaluation should be decomposed into subsystem and end-to-end metrics. Relevant quantities include:

• planning and scheduling latency;
• localization error and covariance;
• perception precision, recall, and uncertainty;
• trajectory-tracking error;
• task success and infeasibility-recognition rates;
• intervention frequency;
• mean time between interventions;
• safe-stop success;
• recovery time;
• network latency, jitter, packet loss, and stale-data behavior;
• energy consumption;
• cyber-detection false positives and false negatives;
• ethical agreement, fairness, and privacy leakage;
• simulation-to-real degradation;
• operation under thermal, environmental, and hardware faults.

The most persistent limitations across the research are incomplete dynamics, static-world assumptions, under-specified interfaces, limited quantitative evaluation, dependence on calibration, insufficient treatment of uncertainty, and reliance on human-authored policies. Many systems demonstrate feasibility without establishing production-level safety, reliability, legal compliance, or generalization.

Future RoboAuto research is likely to emphasize heterogeneous compute, shared-memory transport, multimodal foundation models, formal runtime assurance, uncertainty-aware planning, fleet-scale scheduling, cooperative perception, authenticated negotiation, resilient teleoperation, and long-duration autonomous deployment. The common direction is toward systems that do not merely generate actions, but continuously verify whether actions remain safe, feasible, interpretable, and aligned with mission objectives.