CISRU: Autonomous Robotic Suite for ISRU

• CISRU is a hardware-agnostic, AI-enabled robotics software suite that supports collaborative in-situ resource utilization across rovers, astronauts, and mission control.
• The system integrates modular components for perception, navigation, manipulation, and cooperative behavior using ROS2 and ERGO architecture to adapt high-level goals into executable plans.
• Field validations in Mars analogue settings show high detection accuracies and low latencies, demonstrating its efficacy for tasks like sample collection, inspection, and astronaut support.

CISRU denotes the Collaborative In-Situ Resources Utilisation project and, more specifically, an AI-enabled robotics software suite for planetary and terrestrial robotics that is fully abstracted from the robotic platform and intended to support complex rover-rover and astronaut-rover interaction in non-structured scenarios. Its design couples high-level autonomy, AI-based perception, guidance, navigation and control, manipulation, and cooperative behaviour/human-robot interaction in a modular stack that accepts high-level goals, decomposes them on board into executable plans, and adapts those plans when execution deviates from expectation. The suite is explicitly aimed at ISRU-relevant activities such as sampling, inspection, tool handover, and astronaut support, and it inherits multi-agent planning concepts from ERGO/ADE and the PERASPERA program while aligning with ECSS E4 autonomy (Romero-Azpitarte et al., 2023).

1. Scope, abstraction, and system model

CISRU was developed as a hardware-agnostic autonomy and interaction framework rather than as a controller for a single rover configuration. The suite is described as fully abstracted from specific robotic platforms, deployable on a variety of robots and on astronauts’ technical gear, and interoperable with mission control infrastructure. Its interfaces leverage ROS2 and ERGO Agent messages, and the same architectural logic is intended to operate in simulation, terrestrial analog environments, and planetary scenarios. This platform abstraction is central to the project’s scope: CISRU is meant to transfer collaborative autonomy functions across heterogeneous assets rather than to bind them to a single embodiment (Romero-Azpitarte et al., 2023).

At the architectural level, the system is organized around agents and shared services. A control centre exchanges information with robotic agents and astronauts; each agent converts high-level mission goals into subsystem actions such as movement and manipulation; and a set of shared components provides perception, navigation, cooperative behaviour, manipulation, and human-robot interaction. A later systems paper positions CISRU as the “command and services” layer within a broader AI-enabled rover operations stack, complementing navigation-oriented and terrain-analysis subsystems rather than replacing them (Luna et al., 7 Oct 2025).

Module	Role	Representative technologies
Multi-Agent Autonomy	Goal-based coordination	ERGO, MAS reactor, ROS2
Perception	Scene understanding and safety	MobileNet-SSD, DeepLabV3+
GNC	Mapping, planning, control	Visual SLAM, NAV2, FM2
Manipulation	Tool use and sampling	MoveIt2, tool-changer
Cooperative Behaviour/HRI	Shared awareness and supervision	Hololens 2, map fusion

2. Autonomy regime and multi-agent organization

The autonomy core of CISRU is derived from the ERGO architecture and uses a goal/observation-based interface together with a Multi-Agent Synchronisation reactor for cross-agent communication. The operating regime is ECSS E4 autonomy, summarized in the project material as “execute high-level goals and self-plan/adapt.” In practical terms, only high-level goals are accepted; these are decomposed by on-board planners into executable plans; and if execution diverges from plan, adaptation occurs autonomously. The suite therefore targets supervisory autonomy rather than direct teleoperation, with inter-agent goal and observation exchange as a primary mechanism for coordination (Romero-Azpitarte et al., 2023).

This autonomy model is coupled to an explicit agent hierarchy. CISRU uses leader and secondary agents, but the hierarchy is dynamic rather than fixed to a single operational mode. In human-robot collaboration modes, leader robots aid and supervise astronauts during tasks such as inspection, monitoring, and emergency assistance. In fully robot-robot modes, leader robots manage high-level mission objectives while secondary agents perform specialized tasks such as distributed sampling or tool handling. The same framework supports role changes driven by agent status, environmental evolution, or emergencies, which suggests that CISRU treats mission execution as a distributed planning problem rather than as a static command chain (Luna et al., 7 Oct 2025).

A common simplification is to describe CISRU merely as multi-robot task allocation. The project descriptions are broader. Astronauts, robots, and mission control are all represented within the same coordination logic, and the cooperative behaviour layer is described as ontologically grounded in shared agency, with astronauts and robots acting as peer agents capable of monitoring one another. In this formulation, supervision is bidirectional: robots support astronauts, astronauts can task robots, and mission control remains integrated into the same goal and observation structure (Romero-Azpitarte et al., 2023).

3. Perception, tracking, and situational awareness

The perception subsystem is AI-based and modular, with distinct models assigned to environment segmentation, object and agent detection, pose estimation, anomaly detection, emergency recognition, and interaction detection. Semantic segmentation uses DeepLabV3+ to classify scene regions such as ground, obstacle, little rock, and far rock, while MobileNet-SSD is used for most detection tasks, including astronauts, rovers, rocks, and solar panels. Dedicated models are also described for panel crack detection and astronaut fall detection. The resulting outputs support both operational safety and downstream autonomy, since perception feeds navigation, manipulation, cooperative behaviour, and visualization (Romero-Azpitarte et al., 2023).

Resource-aware deployment is a notable characteristic of this stack. MobileNet-SSD runs on MyriadX VPU as a low-power, space-qualified inference platform, while DeepLabV3+ is reported on Zynq UltraScale+ MPSoC or Jetson Orin AGX. For temporal consistency, CISRU uses a Kalman filter-based tracker to maintain persistent instance identity across frames. Spatial localization is computed from depth, including a simplified mean-depth estimate over a detected bounding box,

$$z_{mean}(B) = \frac{1}{|B|} \sum_{(x,y) \in B} D(x,y),$$

after which persistent identity is maintained by the tracker. Related project documentation also reports 3D mapping through RGB-D sensor fusion and Extended Kalman Filtering, indicating that CISRU’s spatial reasoning layer can accommodate multiple sensing and filtering configurations depending on deployment constraints (Romero-Azpitarte et al., 2023).

The training data strategy reflects the domain specificity of astronaut-rover interaction. Self-labelled datasets from ESTEC, SPoT, and HI-SEAS were used, and the labeling process is described as semi-automated, including partially automated bounding box annotation, adversarial validation, and use of the Segment Anything Model for mask generation in segmentation workflows. Reported field metrics in later validation include 86% accuracy for human detection, interaction tracking, and pose estimation with MobileNet-SSD; 92% accuracy for equipment anomaly detection; and sub-second or approximately 1.2 s emergency response latencies, with 100% fall detection success in controlled tests (Luna et al., 7 Oct 2025).

4. Navigation, mapping, and manipulation

The guidance, navigation, and control subsystem combines perception-aware mapping with autonomous path planning. Visual SLAM fuses stereo camera, inertial, and odometry measurements, and filtered point clouds from perception are used to improve mapping quality. Path planning is implemented through NAV2 in ROS2 with a custom plugin based on the Fast Marching Square algorithm. This navigation layer is not limited to obstacle avoidance: the project documentation emphasizes social navigation in the presence of astronauts, dynamic replanning in response to map updates or execution changes, and compatibility with shared multi-agent world models (Romero-Azpitarte et al., 2023).

In the manipulation domain, CISRU targets ISRU-relevant tasks rather than generic pick-and-place. The reference hardware includes a ROBOTIS Manipulator-H controlled through MoveIt2, equipped with an autonomous multi-tool changer at the end-effector. The suite integrates tool changing, sample collection, sample storage, and error recovery with the planning system, so manipulation primitives can be invoked as part of higher-level collaborative plans. Tool pose is estimated visually relative to the end-effector, and when a tool or sample is unreachable the system can escalate to cooperative behaviour, allowing another agent to assist (Romero-Azpitarte et al., 2023).

The interaction between navigation and manipulation is especially important in the reported field scenarios. A leader rover may identify a target or inspection waypoint, navigate there autonomously, execute or supervise manipulation, and then hand over subtasks to another rover or to an astronaut. This tight coupling suggests that CISRU’s manipulation components are intended as plan-executable services in a broader distributed autonomy framework, not as isolated arm-control modules. The emphasis on sample passing, tool handover, and dynamic role assignment further distinguishes the stack from single-robot manipulation architectures (Romero-Azpitarte et al., 2023).

5. Cooperative behaviour and astronaut-robot interaction

The cooperative behaviour and human-robot interaction layer is responsible for shared situational awareness, supervision, and command exchange among rovers, astronauts, and mission control. Two functions are particularly central. First, position sharing and map fusion allow multiple agents to maintain a fused global occupancy grid. Second, the HRI layer provides direct astronaut interfaces through Mixed Reality and wearable displays. The project material describes Microsoft Hololens 2 as the MR platform, enabling visualization and command through gaze, voice, and optionally gestures, while a wrist console or forearm display provides manual monitoring and control (Romero-Azpitarte et al., 2023).

Map fusion is more than a visualization aid. The system merges local SLAM maps from different rovers into a unified representation, and companion material describes this process in terms of key-point feature detection and matching with a Brute-force matcher to align different partial maps. The resulting shared map supports global awareness, coordinated navigation, and flexible hand-off of tasks such as sample storage or follow-on inspection. This is directly tied to the suite’s conception of shared agency: astronauts and robots are not only visible to one another but are represented within a common state and task model (Romero-Azpitarte et al., 2023).

Supervision and emergency handling are also embedded in this layer. Rovers can detect astronaut falls, deviations in location, or communication loss and then autonomously notify or instruct other agents accordingly. The same infrastructure relays unhandled errors to mission control. This makes CISRU’s HRI model fundamentally bidirectional: astronauts can command and supervise robots, but robots also supervise astronaut status when safety-critical events occur. A plausible implication is that the suite treats EVA support as a cooperative autonomy problem with continuous state monitoring, rather than as a simple user-interface extension to rover control (Romero-Azpitarte et al., 2023).

6. Validation scenarios, measured performance, and significance

CISRU was tested incrementally at component level and then as an integrated system in analogue planetary environments, including the GMV SPoT facility and ESA ESTEC Planetary Lab. The reference robotic platforms included LAMARR as leader rover and MAE as secondary rover, and the scenarios covered solar panel inspection, collaborative sample collection, astronaut detection and tracking, emergency detection, map fusion, tool exchange, and Mixed Reality-supported mission activities. The suite is reported as field-validated at Technology Readiness Level 4 in Mars analogue environments, and later work presents it as one of three integrated AI systems intended to improve next-generation rover surface operations (Romero-Azpitarte et al., 2023).

Two primary operational scenarios recur in the reported evaluations. In multi-robot planetary exploration, a leading rover explores and identifies points of interest while a second rover provides tools or samples, and the multi-agent planner adapts plans in real time for collaborative sampling. In astronaut-rover settlement maintenance, a rover inspects solar panels, detects cracks, and the multi-agent system dispatches an astronaut with MR support for repair; if an emergency such as an astronaut fall occurs, detection and notification are handled autonomously. These scenarios are significant because they span perception, autonomy, navigation, manipulation, and HRI within the same mission thread rather than evaluating the modules in isolation (Romero-Azpitarte et al., 2023).

Reported quantitative outcomes are specific and operationally oriented. Solar panel inspection protocols achieved 0.5 m positioning accuracy, 92% anomaly detection accuracy, and 200 ms latency for reporting to mixed-reality interfaces. Collaborative sample collection completed tool exchange cycles in under 3 minutes, achieved sample transport with 1 m GPS accuracy, and yielded 15% improved area coverage relative to single-robot setups. Emergency response experiments reported 100% detection in 50 controlled fall trials, 1.2 s average alert transmission time, and medical emergency recognition above 90% accuracy. These results support the project’s stated claim that E4 autonomy and AI can materially benefit astronaut-robot collaboration, while also delimiting the evidence to analogue-environment testing rather than operational planetary deployment (Luna et al., 7 Oct 2025).

A recurrent misconception is that CISRU is only a perception package for astronaut detection or, conversely, only a multi-robot ISRU planner. The published descriptions consistently define it as a five-module software suite spanning autonomy, perception, navigation, manipulation, and cooperative behaviour. Another misconception is that high-level autonomy obviates human oversight. The architecture instead retains mission control, Mixed Reality interfaces, and wearable consoles within the operational loop, indicating that CISRU pursues supervised collaborative autonomy rather than autonomy without human participation (Romero-Azpitarte et al., 2023).