Isolation-aware Scheduling Framework for DNN-based End-to-End Autonomous Driving System on Tile-based Accelerators

Abstract: Level-4+ autonomous driving systems (ADS) must run dozens of heterogeneous deep neural networks (DNNs) as end-to-end (E2E) pipelines under a strict latency constraint (<=100 ms), even as execution time varies by up to 3.3x. Cost rules out dedicating isolated hardware to each function in mass-produced ADS, so these DNNs must be densely colocated on a single chip, which introduces shared-resource contention. Tile-based accelerators expose two scheduling opportunities that conventional ADS schedulers do not exploit. First, they provide a tunable degree of parallelism (DoP): assigning more tiles raises DoP and can shorten DNN execution time. Second, they provide hardware-native isolation: tiles can be physically partitioned among co-located DNNs. But using this flexibility is expensive: changing a task's DoP triggers a stop-migrate-restart reallocation of its weights and intermediate features. At ADS task rates of 10-240 Hz, these stalls accumulate along E2E chains and threaten deadlines. Reservation-based schedulers fix DoP and leave this flexibility unused; work-conserving schedulers exploit it but assume reallocation is cheap and treat deadlines as independent. We present ADS-Tile that combines configurable isolation and elastic reservation into a spatio-temporal isolation-sharing space that bounds where and when reallocation occurs; a probabilistic latency model and a DAG-aware runtime scheduler then use this space to decide task colocation and DoP under shared E2E deadlines. On an industry- and academia- derived ADS benchmark, ADS-Tile uses up to 32% fewer tiles than the work-conserving baseline in deadline-critical settings and cuts reallocation-induced wasted processing capacity from 17%-44% to below 1.2%. Controlled spatio-temporal sharing improves resource efficiency and latency predictability for tile-based ADS.

• The paper introduces ADS-Tile, a framework integrating spatial isolation and elastic reservation that reduces migration overhead and tile usage by up to 32%.
• It employs a Guided Hybrid Allocation method and DAG-aware runtime scheduling to dynamically adjust the degree-of-parallelism for deadline-critical DNN tasks.
• Empirical results show improved tail latency, reduced deadline misses, and scalable performance, emphasizing key trade-offs in partition configuration and workload variability.

Isolation-Aware Scheduling for DNN-Based Autonomous Driving Systems on Tile-Based Accelerators

Introduction and Problem Context

As Level-4+ Autonomous Driving Systems (ADS) progress toward full-scale deployment, meeting tight end-to-end (E2E) latency constraints while concurrently executing dozens of heterogeneous DNN pipelines remains a central systems challenge. Legacy approaches that either statically allocate dedicated hardware or rely on purely temporal isolation are not cost-effective for mass-market deployment and can be inefficient under dynamic on-road operating loads. The emergence of tile-based DNN accelerators offers both hardware-enforced isolation and a software-controlled degree of parallelism (DoP) through runtime tile allocation. However, leveraging DoP flexibility incurs significant reallocation costs due to DNN state migration, which threatens deadline adherence when naively managed.

This work presents ADS-Tile, a spatial-temporal scheduling framework that combines hardware-native isolation and elastic reservation to maximize both resource utilization and latency predictability for DNN-based ADS on tile-based multi-core systems. This essay synthesizes the key models, algorithmic advances, empirical results, and design tradeoffs in this architecture, exploring the practical and theoretical implications of its contributions within the tile-based accelerator and real-time embedded AI systems domains.

ADS Workflows and Tile-Based Accelerator Architectures

The target workload is a directed acyclic graph (DAG) abstraction of E2E ADS pipelines, integrating both core driving functions (perception, prediction, planning, and control) and auxiliary cockpit functions (e.g., semantic segmentation, depth estimation). These are implemented as sensor-driven and data-driven DNN modules interconnected by publish-subscribe message passing, each activated at rates ranging from 10 Hz to 240 Hz.

Figure 1: Workflow of a representative ADSArch end-to-end ADS benchmark, showing the DAG topology of sensing and DNN modules.

Tile-based DNN accelerators, such as those pioneered by Tesla, NVIDIA, and Tenstorrent, offer hundreds of compute tiles with isolated local memory, connected via scalable 2D mesh NoCs, and support spatial partitioning (bins) for hardware-native isolation. The scheduler can dynamically adjust the number of tiles assigned to each DNN operator (tuning DoP), yielding a direct tradeoff between per-DNN latency and system resource utilization. However, migrating DNN execution state during DoP changes is expensive relative to both task compute time and traditional CPU context switches, especially at the aggregate rates encountered in full ADS deployments.

Figure 2: Architectural overview of a tile-based accelerator platform, highlighting spatio-temporal workflow scheduling and spatial partitioning capabilities.

Scheduling Challenges and Prior Paradigms

Conventional real-time scheduling for ADS workloads manifest along two suboptimal extremes:

• Reservation-based schemes (e.g., cyclic executives, ROS2, D3) statically allocate fixed DoP and temporal budgets to each task. These maximize predictability but create significant underutilization due to over-provisioning: tail events for all tasks almost never align, so E2E reliability can be achieved with far lower resource allocation than the worst-case per-task sum.
• Work-conserving, dynamic allocation schemes (e.g., Planaria, MoCA, VELTAIR) greedily redistribute tiles among active tasks. While utilization increases, every arrival or completion event can trigger a chip-wide migration, incurring substantial stop-migrate-restart costs and unpredictable E2E latency.

Empirical evidence in the paper quantifies excessive idle or wasted tile fractions (up to 44%) and unacceptable deadline miss rates when using either approach in isolation under realistic variance in task execution and inter-task interference.

Figure 3: Comparison of basic scheduling behaviors in fully temporal-reservation (CyclicName) and throughput-driven (TpFirstName) schedulers.

The ADS-Tile Scheduling Framework

ADS-Tile addresses the above gaps by introducing a spatio-temporal isolation-sharing space, parameterized by two orthogonal mechanisms:

1. Configurable Spatial Isolation: The physical chip is partitioned into bins, each a collection of tiles with strong isolation properties. Rescheduling and DoP reallocation are strictly local to partitions, capping the cost and scope of migratory overhead. Partition granularity is tuned to balance the tradeoff between sharing opportunities and interference control.
2. Elastic Reservation: Rather than assigning fixed sub-deadlines and DoP, elastic reservations are computed using probabilistic latency models (parameterized by quantile $q$) and allow for E2E slack sharing across the DAG: if upstream tasks finish early, downstream slack is dynamically recoverable.

The offline component solves a three-phase Guided Hybrid Allocation (GHA) problem over the workflow DAG to generate initial partitioning, DoP, and reservation schedules using probabilistic execution latency and I/O models. The runtime scheduler then admits, allocates, and migrates DNN tasks according to current load, partition-specific queues, and ERT/DDL constraints, only triggering migratory events when they deliver net E2E latency benefit.

Figure 4: System overview of ADS-Tile, integrating offline GHA compilation and distributed runtime scheduling.

Figure 5: Spatio-temporal isolation-sharing, illustrating adaptive degree-of-parallelism adjustment in partition-local tile pools with DAG-aware slack management.

DAG-Aware Runtime Scheduling

The runtime architecture runs a two-level scheduler: a global host controller configures per-partition queues, while local RISC-V controllers within each partition supervise task admission, DoP allocation, and migration events. Critical features include:

• Admission by ERT: Tasks become eligible only after their ERT, preventing head-of-line blocking due to premature admissions.
• Soft Sub-Deadlines: Hard drops occur only at true E2E deadline violations, with sub-deadlines used for urgency ordering.
• Quota Control: Tile DoP is selected for each job so as to maximize deadline adherence with minimal unnecessary acceleration.
• Limited Reallocation Scope: Migration is resolved using precompiled dataflow and indirection tables, ensuring only the affected partition's tiles are paused and migrated.

Figure 6: Runtime scheduling flow within a partition, including event-driven admission, DoP fitting, and localized memory migration.

Empirical Results and Comparative Evaluation

The evaluation targets realistic L4 ADS benchmarks consisting of industry-derived heterogenous DNN chains, implemented on a multi-chip Simba-style architecture. Results are presented in terms of tail latency ($99^\text{th}$ percentile), deadline-miss rate, average tile utilization, and scaling efficiency.

• Tile Efficiency: In deadline-critical settings, ADS-Tile uses up to 32% fewer tiles than the work-conserving baseline and reduces migration-induced wasted capacity from 17–44% to below 1.2%, directly lowering area and power requirements.
• Tail Predictability: Deadline-miss rates are kept low and stable under major workload variation, irrespective of the drop policy, with tail latency tracking the deadline bound more precisely than under best-effort sharing.
• Partitioning Tradeoffs: Increasing partition count decreases reallocation overhead but increases fragmentation; optimal settings vary with workload and hardware scale.
• Scaling: Maximum admissible workload increases by 1.3–2.3$\times$ (in terms of auxiliary chains) for a given hardware budget, with lower miss rates and more stable tail behavior.

  Figure 7: 99th-percentile E2E tail latency and deadline miss rate versus tile count, workload scale, and drop policy.

  

  Figure 8: Maximum supported workflow chains under latency constraints and minimum required tiles (hardware scale) for latency compliance.

Theoretical Implications and Future Directions

This work formalizes the spatio-temporal task packing problem on malleable, partitionable accelerator architectures under E2E probabilistic deadline constraints, and demonstrates that neither classical worst-case static analysis nor unconstrained dynamic sharing is sufficient for high-throughput, real-time autonomous systems. The GHA algorithm and its runtime decomposition offer a concrete approach to scalable malleable scheduling for DAG-structured workloads—a foundational result for real-time ML system design.

Implications and prospects include:

• Applicability to heterogeneous and mixed-criticality scheduling beyond ADS, including robotics, AR/VR, and any real-time ML domain requiring both hard guarantees and efficient sharing.
• Potential for integration with model compression, dynamic precision adjustment, or accuracy-reliability tradeoff mechanisms for enhanced safety and cost efficiency.
• Extensions to distributed and cross-chip systems, including composable partitioning and inter-chip migration routing, with relevance for future exascale AI training and inference deployments.
• Building blocks for modular, programmable, and resource-isolation-aware AI middleware that support industry safety standards and certification workflows.

Conclusion

ADS-Tile exemplifies an isolation-aware scheduling framework that achieves high resource utilization and tight latency predictability for DNN-based real-time workflows on partitionable tile-based accelerators. By jointly exploiting hardware-native spatial isolation and elastic, probabilistic temporal reservation—while confining migration penalties—ADS-Tile enables dense DNN colocation within stringent cost, reliability, and latency budgets. Its methodology illustrates a path toward deployable, scalable, and certifiable AI systems in autonomous driving and other latency-critical domains (2606.10303).