XAUTO: Multi-XPU Runtime for Autonomous Apps

• XAUTO is a runtime system that optimizes latency in autonomous applications by exposing fine-grained XNODE tasks across heterogeneous processors.
• It employs an ILP-based policy solver to jointly assign XPUs and set static priorities, ensuring deterministic, low-latency execution.
• The system integrates a profiler, runtime, and scheduler to enforce fixed-priority preemptive queues on CPUs, GPUs, and DLAs for real-time performance.

Searching arXiv for the XAUTO paper and closely related work to ground the article. XAUTO is a runtime system for latency-sensitive autonomous applications that cooperatively manages multiple heterogeneous processors, or XPUs, through a fine-grained, multi-XPU programming abstraction called XNODE. Its central premise is that existing runtime systems for such applications, including ROS, perform module-level task management but lack awareness of the fine-grained usage of multiple XPUs. XAUTO addresses that gap by exposing stage-level tasks, jointly selecting XPU assignments and static priorities, and scheduling execution so as to minimize end-to-end latency in autonomous workloads such as perception pipelines for autonomous driving (Han et al., 13 Aug 2025).

1. Problem setting and design objective

Modern autonomous applications increasingly distribute different stages of their algorithm modules across CPUs, GPUs, DLAs, and related accelerators. In the formulation used by XAUTO, the relevant granularity is not the coarse application module but the individual stage inside a module, such as preprocessing, inference, or postprocessing. This finer granularity is the basis for the system’s claim that holistic management is necessary: without stage-level visibility, a runtime cannot jointly optimize XPU allocation and execution order across the full task graph (Han et al., 13 Aug 2025).

XAUTO is organized around four cooperating components: an offline Profiler, an offline Policy Solver, an online Runtime, and an online Scheduler. The Profiler measures worst-case execution times, or WCETs, for each XNODE implementation on each supported XPU in isolation. The Policy Solver then takes the directed acyclic graph, or DAG, topology together with those WCETs and solves an integer linear program, or ILP, that determines both the XPU chosen for each XNODE and the static priority assigned to it. The Runtime loads that configuration and instantiates XNODE tasks on incoming data, while the Scheduler enforces partitioned, fixed-priority, preemptive queues on each XPU (Han et al., 13 Aug 2025).

The design target is deterministic latency reduction rather than adaptive online optimization. The paper explicitly frames static ILP-based scheduling as providing deterministic bounds and as being vital for safety-critical latency requirements. This suggests that XAUTO is positioned in the real-time scheduling tradition, even though its execution substrate includes accelerators such as GPUs and DLAs rather than only CPUs.

2. Programming abstraction: XNODE and XMODULE

The core abstraction is the XNODE, defined as one stage inside an autonomous-application module. An XNODE is finer-grained than a ROS node, and it may have multiple implementations, each tagged with an XPU type. The programming model therefore allows a developer to provide multiple realizations of the same logical stage, for example a GPU implementation and a DLA implementation, each with its own initialization and execution callbacks (Han et al., 13 Aug 2025).

At initialization time, each implementation’s init callback runs once and returns a scheduling handle, such as one associated with a CUDA or TensorRT context. At execution time, the exec callback is invoked on each data frame. This separation matters because it allows the runtime to treat stage invocations as schedulable tasks while preserving the implementation-specific state needed by the underlying accelerator.

XAUTO also defines XMODULE, which groups a sequence of XNODEs into a module, manages shared state, and exposes input and output topics. Internally, it chains the constituent XNODEs in sequence and publishes to topics when the last XNODE finishes. The notable consequence is that XAUTO can retain a modular application structure while still exposing a stage-level DAG to the policy solver and scheduler. By explicitly declaring stage-to-XPU mappings at the XNODE level, the runtime can observe both the full stage graph and the set of alternative XPU implementations available for each stage (Han et al., 13 Aug 2025).

A concise summary of the major components is as follows.

Component	Role
Profiler	Runs each XNODE implementation in isolation to measure WCETs on CPU, GPU, DLA, and related XPUs
Policy Solver	Solves an ILP to choose XPU assignments and static priorities
Runtime	Instantiates XNODE tasks on incoming data and submits them with priorities
Scheduler	Enforces partitioned, fixed-priority, preemptive queues on each XPU

3. Formal scheduling model

XAUTO formalizes XPU assignment and scheduling over a DAG $G=(V,E)$ of XNODEs. Let $M$ be the set of XPU instances, and let each node $v_i$ have WCET $\mathrm{wcet}_i^m$ on XPU $m$, with $\infty$ used when an implementation is unsupported. The model introduces binary assignment variables $p_i^m \in \{0,1\}$, start and finish times $s_i, f_i \ge 0$, and demand variables $d_{ijk}^m \ge 0$ (Han et al., 13 Aug 2025).

The constraints are given in four groups. First, each XNODE must be assigned to exactly one XPU:

$\sum_m p_i^m = 1 \qquad \forall i$

Second, precedence constraints must hold for each edge $M$0:

$M$1

Third, finish times must respect WCET:

$M$2

Fourth, XPU capacity is enforced in a demand-bound-function style. For any interval $M$3,

$M$4

with

$M$5

if $M$6 (Han et al., 13 Aug 2025).

The optimization objective is

$M$7

that is, minimization of the end-to-end makespan. After solving, XPU assignments are read directly from the $M$8 variables, and static priorities are derived by ordering nodes in increasing $M$9, so that earlier-finishing tasks receive higher priority. This is a particularly strong design choice: priority is not separately optimized but induced from the optimal finish-time ordering. A plausible implication is that XAUTO treats assignment and priority synthesis as a coupled offline problem rather than as independent stages.

4. Online runtime and scheduler behavior

The online execution path follows the offline policy. For each incoming message, the runtime creates a task for each XNODE whose inputs are ready, attaches the precomputed priority and the handle returned by initialization, and submits that task to the worker associated with the assigned XPU. Each worker maintains a priority queue for its XPU. When a higher-priority task arrives, the worker preempts the running task through XPU-specific suspend and resume hooks; otherwise the new task waits until higher-priority work completes (Han et al., 13 Aug 2025).

This design is explicitly partitioned: each XPU coordinator only manages tasks assigned to that XPU. The scheduler therefore does not attempt global run-time migration across devices. Instead, the global coupling happens offline in the ILP, and run-time enforcement is local, fixed-priority, and preemptive where the hardware permits it.

Several implementation details are central to that behavior. GPU preemption uses the NVIDIA driver’s Time Slice Groups; each CUDA context is mapped to a Time Slice Group, and suspend and resume are performed through driver calls. The reported GPU preemption latency is approximately $v_i$0. DLA preemption suspends and resumes the driver’s command queue, with latency approximately $v_i$1. CPU tasks are run as SCHED_FIFO threads pinned to cores, relying on native preemptive real-time scheduling. XPUs without hardware preemption fall back to non-preemptive, priority-ordered queueing under the same coordinator framework. The scheduler’s CPU overhead is reported as negligible, below $v_i$2 on a 40-node DAG (Han et al., 13 Aug 2025).

The runtime can load XMODULEs either in separate processes or in one unified process for lower IPC cost. This indicates that XAUTO does not prescribe a single deployment boundary between functional modules and scheduling infrastructure; instead, it permits a trade-off between isolation and inter-process communication overhead.

5. Experimental evaluation

The evaluation is carried out on an NVIDIA Jetson AGX Orin with a 12-core ARM CPU, 1 GPU, and 2 DLA, in MAXN power mode. Two workload families are used: synthetic DAGs and a real driving perception pipeline consisting of LA, TD, TC, FCD, RCD, LD, TR, and PR. The synthetic setup includes 7 groups of 50 random DAGs with 10 to 40 XNODEs each, CPU WCETs from approximately 5 to 95 ms, GPU speedups of $v_i$3 to $v_i$4, DLA speedups of $v_i$5 to $v_i$6, and probabilistic assignment of implementations. Two hardware configurations are considered for the synthetic case: a small configuration with 4 CPU cores, 1 GPU, and 1 DLA, and a large configuration with 8 CPU cores, 1 GPU, and 2 DLA (Han et al., 13 Aug 2025).

The baselines are ROS2, characterized as module-level scheduling with locally optimal XPU assignment to the fastest device, and XAUTO+HEFT, which uses XAUTO’s runtime together with HEFT assignment. The main quantitative result is that XAUTO reduces geomean end-to-end latency relative to ROS2 across all tested DAG sizes.

System	10 nodes	20 nodes	30 nodes	40 nodes
ROS2	1.00	1.00	1.00	1.00
XAUTO+HEFT	1.31×	1.47×	1.56×	1.58×
XAUTO (ILP)	1.38×	1.59×	1.72×	1.65×

Across synthetic benchmarks, XAUTO’s speedup over ROS2 ranges from 1.23× to 2.01× as the setting varies from small to large and from 10 to 40 nodes. In the driving case, ROS2 with all-GPU inference reaches a 99th-percentile latency of 42.0 ms, while XAUTO, using an ILP solution that places FCD and TD on DLA, reaches 26.1 ms, which is 1.61× faster. Even when the XPU assignment is held fixed, XAUTO’s scheduler alone contributes an additional gain of approximately 1.29× over ROS2’s round-robin GPU scheduling (Han et al., 13 Aug 2025).

These results support the paper’s central claim that both components of the method matter: the offline assignment policy and the online priority-driven scheduler. The comparison to XAUTO+HEFT indicates that the scheduling substrate alone is not the entire explanation for the latency reduction.

6. Limitations, future extensions, and naming

The paper identifies several trade-offs. Static ILP-based scheduling provides deterministic bounds and predictable, repeatable behavior, but it cannot adapt online to thermal throttling, input-dependent WCET, or dynamic load. It also depends on the accuracy of offline profiling, and profiling must disable all contention. Proposed extensions include dynamic priority adjustment per frame, online XPU reassignment with state migration, pre-computed ILP solutions for different load scenarios with dynamic switching, and combination with reinforcement learning or lightweight heuristics for unpredictable workloads (Han et al., 13 Aug 2025).

The name “XAUTO” can be confused with other systems whose names begin with “X-Auto” or “XAuto,” but they address different problems. X-AutoMap is a modular framework for autonomous X-ray fluorescence mapping with correlative feature detection, tight integration with Bluesky and EPICS at the NSLS-II Hard X-ray Nanoprobe, and a coarse-to-fine scan orchestration workflow for chemically heterogeneous systems (Deleon et al., 13 Nov 2025). AutoXAI is a framework for selecting and tuning explainable AI solutions using context-aware recommendation and AutoML-style hyperparameter optimization (Cugny et al., 2022). XAutoML is a visual analytics tool for understanding and validating automated machine learning systems, centered on interactive visualizations and XAI analyses inside JupyterLab (Zöller et al., 2022).

Accordingly, in the technical literature, XAUTO most precisely denotes the runtime system built around XNODE, ILP-based XPU assignment, and partitioned priority scheduling for latency-sensitive autonomous applications. Its contribution lies in combining stage-level abstraction, holistic multi-XPU policy synthesis, and lightweight online enforcement into a single runtime framework (Han et al., 13 Aug 2025).