Unified Autonomy Stack Overview

• Unified Autonomy Stack is an integrated framework unifying modules for perception, planning, control, simulation, deployment, and assurance across varied domains.
• It uses configuration-driven abstractions and stable interfaces like YAML, ROS 2, and ECS to ensure seamless module substitution and lifecycle continuity.
• It advances safety and performance via formal verification, conservative data fusion, and real-to-sim-to-real toolchains, ensuring reliable autonomous operations.

Unified autonomy stack denotes an integrated software and systems architecture that coordinates perception, state estimation, planning, control, simulation, deployment, and assurance within a common set of abstractions. In recent research, the term is used for configuration-driven maritime autonomy frameworks, learned-and-verified spacecraft GNC, semantic shared-autonomy for driving, multi-sensor multi-agent autonomy platforms, real2sim2real digital-twin toolchains, autopilot-agnostic aerial stacks, and embodied-agent systems that unify cyber and physical skills (Vasstein et al., 6 Jun 2025, Shojaei, 24 Jun 2026, Nguyen et al., 6 Nov 2025, Hallyburton et al., 2023, Sánchez-Alzola et al., 2024, Panerati et al., 6 Feb 2026, Shi et al., 25 Jun 2026). The common objective is not merely software consolidation, but a single operational substrate spanning algorithm development, simulation, testing, deployment, and, in some cases, certification.

1. Conceptual scope and representative interpretations

The literature does not present a single canonical stack. Rather, it presents a recurring architectural idea: autonomy should be expressed through stable interfaces that decouple modules while preserving end-to-end operability. PyGemini describes a “single, extensible autonomy stack for maritime systems—from early R&D through cloud-scale data-generation to certification-grade acceptance testing” (Vasstein et al., 6 Jun 2025). The NTNU-ARL “Unified Autonomy Stack” is a “system-level autonomy ‘blueprint’” organized around multi-modal perception, multi-behavior planning, and multi-layered safe navigation (Dharmadhikari et al., 12 May 2026). OmniAct defines a unified stack as a “hierarchical, asynchronous software architecture” that bridges cyber and physical action domains while separating planning, memory, and verification (Shi et al., 25 Jun 2026).

This suggests that “unified” refers less to one algorithmic doctrine than to a systems property: a stack is unified when the same abstractions survive across modalities, robot morphologies, and lifecycle stages.

Framework	Domain	Unification axis
PyGemini	Maritime autonomy	ECS, YAML CDD, containerization, acceptance testing
AMPLE-GNC	Spacecraft autonomy	Learned capability tiers bounded by verified shielding
SAFe-Copilot	Shared autonomy for driving	Semantic arbitration via VLMs
AVstack	Multi-sensor, multi-agent autonomy	Dataset generation, training, fusion, evaluation
AutoDRIVE–Autoware	Digital-twin deployment	Unified real2sim2real toolchain
aerial-autonomy-stack	Aerial robotics	Autopilot-agnostic ROS 2 stack with FTRT simulation
OmniAct	Embodied cyber-physical agents	Unified skill routing, memory, visual preemption

A persistent misconception is that a unified stack must be monolithic. OmniAct explicitly argues for “not a monolithic model but a hierarchical asynchronous architecture with explicit separation of planning, memory, and verification” (Shi et al., 25 Jun 2026). Another misconception is that unification eliminates domain adaptation. AVstack reports that camera networks trained on infrastructure vantage points recover “>500 % more detections” than ego-trained models on the same infra imagery, underscoring the necessity of per-application re-training (Hallyburton et al., 2023).

2. Architectural abstractions and software organization

A major line of work unifies autonomy through explicit architectural formalisms. PyGemini adopts an entity–component–system formulation in which a world is a triple

$W=(E,\mathrm{Assign},P),$

with entities $E$, component types $C$, and processors $P$, and with tick-wise data-flow given by processor composition

$$\mathrm{Assign}_1=P_k\circ\cdots\circ P_2\circ P_1(\mathrm{Assign}_0).$$

The same framework builds a world directly from configuration,

$$\texttt{world}=\mathsf{build\_world}(\texttt{config.yaml}),$$

and treats the YAML description as both specification and executable acceptance test (Vasstein et al., 6 Jun 2025).

Other stacks formalize unification through layered software boundaries. Aerostack2 uses seven layers, from middleware and inter-process communication up to basic robotics functions, behaviors, mission control, and applications, with each layer allowed to call only the layer immediately below (Fernandez-Cortizas et al., 2023). TCS-AD similarly isolates Localization, Perception, Mission/Maneuver, Planning, Execution, Diagnostics, and V2X in separate ROS 2 nodes with fixed message schemas and topic names, so that a SLAM node can be substituted for GNSS-RTK “without touching downstream code” (Ochs et al., 2024). The Chrono-based toolkit combines a ROS 2–Chrono interface, ROS 2 autonomy modules, and a Dockerized development ecosystem, so that identical autonomy nodes can run on the real vehicle or in simulation (Elmquist et al., 2022).

Embodied-agent work extends these abstractions from control pipelines to unified action spaces. OmniAct defines actions as tuples

$a_t=(\tau_t,g_t),$

where $\tau_t\in\mathcal{T}$ indexes a cyber or physical skill and $g_t$ contains skill-specific parameters (Shi et al., 25 Jun 2026). This is conceptually parallel to SAFe-Copilot’s semantic-level arbitration, where the decisive object is not a low-level trajectory alone but a high-level plan label such as “stop,” “turn left,” or “drive forward” (Nguyen et al., 6 Nov 2025).

Across these systems, the architectural constant is explicit interface discipline: ECS ports, ROS 2 topics, action servers, plugin descriptors, or YAML commands. A plausible implication is that the unification problem is increasingly being treated as an interface-design problem rather than solely as an algorithm-selection problem.

3. Perception, mapping, and world representation

Unified stacks differ most visibly in their world models. The NTNU-ARL blueprint fuses LiDAR, FMCW radar, cameras, and IMUs in a windowed factor-graph smoother over states

$x_k=[p_{WB}^k,v_{WB}^k,R_{BW}^k,b_a^k,b_g^k]^\top,$

solved in iSAM2 using IMU, LiDAR, radar, and optional vision factors to obtain drift-corrected pose and an updated TSDF/voxel map (Dharmadhikari et al., 12 May 2026). It augments geometry with semantics through YOLOe or GPT-5, back-projecting detections into the voxel grid to form 3D bounding boxes and scene graphs (Dharmadhikari et al., 12 May 2026).

A different unification strategy is to use one representation for geometry, photometry, and semantics. In the Gaussian-splatting framework, the map is a set of anisotropic Gaussians with parameters $E$0 and density

$E$1

with global map density $E$2 (Ong et al., 17 May 2025). The same structure supports rendering, collision checking, semantic retrieval, and large-scale task-driven navigation. Semantic vectors are PCA-compressed CLIP features, and clustering uses a joint geometric-semantic distance

$E$3

This representation is explicitly proposed as a “unified representation for autonomous robot navigation in large-scale unstructured outdoor environments” (Ong et al., 17 May 2025).

Several other stacks unify autonomy by unifying the data-production substrate. AVstack uses a dataset generation pipeline that instantiates arbitrary combinations of RGB, semantic-segmentation, depth cameras, LiDAR, and radar on both vehicles and infrastructure, registers them in a single global frame through RefChoC, and emits data in NuScenes/KITTI/Waymo-compatible formats (Hallyburton et al., 2023). AutoDRIVE–Autoware builds “autonomy-oriented digital twins” of vehicles and environments, with rigid-body, suspension, tire, powertrain, LiDAR, camera, and environment models connected to Autoware Universe over standard ROS 2 topics (Samak et al., 2024).

Neuromorphic work adds another representational regime. The event-based stack models each DVS event as

$E$4

feeds events to a leaky integrate-and-fire network, and derives obstacle or gate observations through spike clustering (Sanyal et al., 11 Mar 2025). Here, unification lies in reconfigurable perception, planning, and control modules driven by a YAML “platform profile” that swaps a differential-drive TurtleBot for a quadrotor (Sanyal et al., 11 Mar 2025).

4. Planning, control, and decision arbitration

In a unified stack, planning is rarely a single algorithmic block. It is instead a family of planners connected by common abstractions. The NTNU-ARL architecture builds a dense local graph $E$5 and a sparse global graph $E$6 and supports Planning-to-Target, Exploration, and Inspection as distinct optimization problems over the same online map (Dharmadhikari et al., 12 May 2026). TCS-AD uses a behavior tree for mission validation and a particle-swarm planner for collision-free, comfort-maximizing trajectories, while execution adds guard layers that can replace the planner’s speed profile with a predefined braking curve (Ochs et al., 2024). Aerostack2 exposes behaviors as ROS 2 action servers with start, pause, resume, stop, and modify interfaces, and composes missions through behavior trees with Sequence, Fallback, and Parallel semantics (Fernandez-Cortizas et al., 2023).

Shared-autonomy systems make the arbitration problem explicit. SAFe-Copilot first abstracts raw plans and driver actions into textual labels, then computes uncertainty

$E$7

and invokes a VLM to choose among human, autonomous, or merged plans when $E$8 or when driver input is requested (Nguyen et al., 6 Nov 2025). On Bench2Drive, “VAD + SAFe-Copilot” reduces collision rate from $E$9 to $C$0, raises completion from $C$1 to $C$2, and improves average score from $C$3 to $C$4 (Nguyen et al., 6 Nov 2025). The paper emphasizes that arbitration at a semantic, language-based level preserves intent better than low-level trajectory arbitration alone (Nguyen et al., 6 Nov 2025).

Control unification likewise tends to mix analytic and learned modules. AMPLE-GNC’s fault-adaptive controller uses Rapid Motor Adaptation to infer latent actuator faults online and applies

$C$5

achieving $C$6 recovery on actuator-sign faults and $C$7 on continuous-gain faults within the training randomization envelope (Shojaei, 24 Jun 2026). The racing stack combines behavioral primitives—Maintain, Trail, Safe Merge—with a lateral controller based on pure-pursuit look-ahead plus LQR gain scheduling and a longitudinal P controller with feedforward drag compensation, enabling operation at $C$8 mph and multi-agent passing (Saba et al., 2024). The neuromorphic navigation stack combines PID, MPC, and PiNN variants under task-specific reconfiguration, reporting reduced maneuver time and energy relative to RGB-only and perception-only baselines (Sanyal et al., 11 Mar 2025).

5. Deployment, simulation, and lifecycle integration

One of the strongest interpretations of unification is lifecycle continuity: the same stack should run as a stand-alone application, in simulation, in CI/CD, in cloud services, and on hardware. PyGemini is explicit on this point. Each layer can run natively in Python, inside a Docker container, or embedded into an external process such as a ROS 2 node; it also supports stand-alone execution, cloud-based service deployment, and embedded-library usage (Vasstein et al., 6 Jun 2025). Its CDD loop combines configuration search, library extension, YAML authoring, acceptance testing, and pull-request publication (Vasstein et al., 6 Jun 2025).

The aerial-autonomy-stack generalizes this principle to aerial deployment. It decomposes autonomy into Perception, Planning, Control, Simulation, and Networking containers, offers a unified autopilot interface for PX4 and ArduPilot, and synchronizes the entire system with a synthetic ROS 2 clock (Panerati et al., 6 Feb 2026). The reported real-time factors include $C$9–$P$0 for a single PX4 instance with two sensors and $P$1 for a dual-instance PX4 configuration, supporting “over 20x faster-than-real-time” end-to-end simulation (Panerati et al., 6 Feb 2026).

Digital-twin toolchains push the same idea toward real2sim2real continuity. The AutoDRIVE–Autoware system tele-operates physical vehicles through a common API/HMI layer, identifies digital-twin parameters from streamed sensor data, runs Autoware Universe in the simulator through the same ROS 2 graph used in hardware, and then transfers the unchanged graph to real vehicles through the Devkit (Samak et al., 2024). The Chrono-based toolkit makes an analogous claim at smaller scale: swapping simulation for reality is largely a matter of changing the ROS 2 namespace or replacing the Chrono bridge with real sensor drivers (Elmquist et al., 2022).

These results indicate that “stack unification” often encompasses software supply chain, middleware, timing model, and deployment packaging. It is not confined to runtime autonomy alone.

6. Assurance, evaluation, and open technical tensions

The most consequential divergence among unified stacks concerns assurance. PyGemini uses hash-based regression testing: every configuration can be submitted as a test, the system runs for $P$2 ticks, changed component attributes are hashed with SHA-256, and the resulting tuples are compared against golden hashes in CI (Vasstein et al., 6 Jun 2025). The artifact is small enough to attach to certification reports, and the stated purpose is to prevent future code changes from silently breaking a scrutinized configuration (Vasstein et al., 6 Jun 2025).

AMPLE-GNC takes a stronger formal route. Its runtime shield monitors a 6-DOF spacecraft state every $P$3 s, checks nine LTL invariants, proves predictor soundness in Kind 2, and uses a split-conformal recovery deadline to determine when to latch into a verified safe-hold controller (Shojaei, 24 Jun 2026). The paper reports $P$4 invariants proved sound, $P$5 unsound events after the second-order predictor fix over $P$6 “safe” verdicts, and a shield that keeps the controller $P$7 autonomous while still catching non-recovery (Shojaei, 24 Jun 2026).

Safe navigation stacks also incorporate conservative fusion or last-resort safety filters. AVstack’s post-tracking decentralized data fusion uses covariance intersection to remain conservative under unknown correlations and maintains mAP near $P$8 even under major correlation, whereas naïve track-fusion drops to about $P$9 (Hallyburton et al., 2023). The NTNU-ARL stack adds a control barrier function layer beneath map-based planning and exteroceptive policies; in Gazebo quadrotor ablations, crash rates “drastically drop” when adding the CBF safety filter (Dharmadhikari et al., 12 May 2026).

By contrast, not all unified stacks claim formal guarantees. SAFe-Copilot explicitly states that no formal proof of safety is given, even though the system can be viewed as enforcing a “safety contract” through VLM-mediated arbitration (Nguyen et al., 6 Nov 2025). The AutoDRIVE–Autoware report likewise states that no formal quantification of sim-to-real fidelity and no error-bound metrics were provided (Samak et al., 2024). PyGemini emphasizes real-time or near-real-time promise but “does not report raw FPS numbers” (Vasstein et al., 6 Jun 2025).

These tensions define the current frontier. Unified stacks have clearly moved beyond ad hoc module collections: they now unify representations, middleware, simulation, deployment, and, increasingly, assurance. Yet the literature also shows that unification does not by itself resolve the hardest problems. Semantic reliability in Gaussian maps remains an open issue; CPU resources remain the primary bottleneck in faster-than-real-time aerial simulation; long Autoware build times and high RAM usage complicate deployment on small platforms; and broad domain randomization, rather than raw data volume, is what drove held-out fault generalization in AMPLE-GNC (Ong et al., 17 May 2025, Panerati et al., 6 Feb 2026, Samak et al., 2024, Shojaei, 24 Jun 2026). A plausible implication is that future unified autonomy stacks will be judged less by how many modules they contain than by how rigorously they coordinate capability, safety, and lifecycle reproducibility across heterogeneous platforms.