Twill: Optimal GPU & Edge Scheduling

• Twill refers to advanced systems that optimize scheduling for both tensor-core GPUs and mobile edge AI, employing formal constraint optimization.
• On GPUs, Twill achieves optimal software pipelining and warp specialization by jointly solving a QFLIA-based linear constraint system.
• On mobile edge platforms, Twill dynamically manages heterogeneous AI workloads, reducing inference latency and ensuring strict power compliance.

Twill denotes two advanced systems at the intersection of high-performance computing and intelligent workload management. In the context of tensor-core GPUs, Twill refers to a heuristic-free scheduling system that delivers provably optimal software pipelining (SWP) and warp specialization (WS) for iterative GPU programs. On heterogeneous mobile edge platforms, Twill designates a run-time scheduling framework for compound AI (cAI) systems, enabling dynamic, priority-aware orchestration of deep neural network (DNN) and transformer/LLM inference under strict power constraints. Both systems achieve state-of-the-art performance by formulating scheduling as formal constraint optimization problems and integrating detailed hardware knowledge, but they target different aspects of the AI and systems stack (Soi et al., 19 Dec 2025, Taufique et al., 1 Jul 2025).

1. Twill for Optimal Software Pipelining and Warp Specialization on GPUs

Twill automatically derives jointly optimal software-pipelining and warp-specialization schedules for iterative programs on modern tensor-core GPU microarchitectures such as NVIDIA Hopper (H100) and Blackwell (B200). Its design objectives are: (i) maximize instruction-level parallelism (ILP) via minimal initiation interval (II) pipelines, (ii) allocate GPU resources (registers, shared memory, warps) within exact hardware constraints, and (iii) guarantee global optimality using only off-the-shelf constraint solvers. Modern tensor-core GPUs expose asynchronous fixed-function units—e.g., matrix-multiply and tensor-memory accelerators—with long and explicitly managed latencies. SWP maximizes ILP across loop iterations, while WS assigns operations to warps to mitigate stalls from resource contention or synchronization. Twill’s principal insight is to formulate SWP and WS as a single, joint optimization problem in quantifier-free linear integer arithmetic (QFLIA), automatically yielding the best achievable II and valid code-generation assignments (Soi et al., 19 Dec 2025).

2. Technical Workflow and Formalization

Twill accepts a tile-level intermediate representation, such as Triton's TTGIR, and constructs a dependence graph $G=(V,E)$ where nodes represent tile-operations (e.g., GEMM, EXP, TMA) annotated with resource-reservation tables and memory footprints. Cost normalization via a preprocessing integer linear program (ILP) adjusts operation latencies to tractable magnitudes while preserving ratios. An initial ILP-based modulo scheduler computes the minimal II for pure SWP. Twill then "unrolls" the steady-state schedule, formulates a QFLIA constraint system over operation stages, warp assignments, and II, and uses an SMT solver (Yices2) to jointly solve SWP and WS subject to:

• Data-dependence latencies
• Resource/budget capacity (registers, shared memory) per warp and per cycle
• Cross-warp spill and synchronization penalties
• Blocking synchronization window constraints

The overall objective is $\min_{s, w, II} II$, subject to all machine constraints encoded as linear integer inequalities. This formulation ensures no heuristic schedule can outperform Twill under the explicit hardware model (Soi et al., 19 Dec 2025).

3. Implementation and System Extensibility

Twill’s implementation leverages open-source solvers (CBC for initial II computation, Yices2 for final SMT scheduling) and is parameterized by the hardware’s resource-timing and capacity figures. Adapting to new architectures involves updating resource-reservation tables, register/shared memory budgets, spill latencies, and synchronization patterns. There are no hard-coded heuristics, ensuring performance portability. Twill currently models singly-nested loops without conditionals and requires external tile sizes, focusing strictly on II minimization. Extending Twill to handle nested/branched control flow (hierarchical modulo scheduling), automated tile-size selection, and end-to-end integration with modern GPU compilers are ongoing research directions (Soi et al., 19 Dec 2025).

4. Empirical Evaluation and Optimality Guarantees

Applied to Flash Attention kernels on NVIDIA H100 and B200 architectures, Twill rediscovered the expert-tuned SWP+WS schedules of Flash Attention 3 and 4, achieving within 1–2% of hand-tuned throughput (e.g., ≈645 TFLOP/s) in 19–28 seconds of solve time. For the backward pass, Twill validated that H100’s register constraints preclude cross-iteration pipelining, aligning with manual expert findings. Because Twill proves infeasibility for any II lower than its returned value, it operationalizes a formal optimality guarantee: no SWP+WS schedule with a smaller II exists for the given kernel and architecture, under the precise capacity and timing model (Soi et al., 19 Dec 2025).

5. Twill for Scheduling Compound AI on Mobile Edge Platforms

A distinct system also called Twill manages real-time scheduling of heterogeneous AI inference on resource- and power-constrained mobile edge devices, such as the Nvidia Jetson Orin NX (Taufique et al., 1 Jul 2025). Twill addresses the online scheduling of DNN, transformer, and LLM inference requests that form compound-AI (cAI) workloads. Each incoming task is assigned a model type, arrival time, latency requirement, and priority. The scheduler must jointly (i) map tasks to heterogeneous clusters (CPU, GPU, DLA), (ii) dynamically migrate or freeze/resume running tasks, and (iii) adjust dynamic voltage-frequency scaling (DVFS) levels on each cluster, all under a strict global power budget.

Twill employs an affinity-driven cluster mapping metric, considering both latency and power efficiency. Runtime migration is triggered when it reduces cumulative system latency after accounting for migration cost. Priority-aware freeze/unfreeze logic employs a FIFO "freeze queue" to ensure high-urgency tasks oust lower-priority work while maintaining responsiveness. A feedback-based DVFS controller iteratively tunes compute frequencies to keep total board power below $P_{max}$. These mechanisms exploit (i) online layer-level model profiling (for per-cluster latency/power estimation), (ii) exact power-domain monitoring, and (iii) fine-grained OS control of process state and cluster bindings (Taufique et al., 1 Jul 2025).

6. Performance Analysis, Complexity, and Experimentation

On Jetson Orin NX (10W TDP), Twill achieves 20–54% lower end-to-end cAI workload completion time compared to MapFormer, Tango, and Band baselines. Average inference queues wait times decrease by 83–88%, and Twill meets 95th-percentile latency targets for high-priority tasks, where baselines miss these targets in 30–60% of runs. Crucially, Twill guarantees strict power compliance, with 0% of time above TDP (±0.1W hysteresis), while MapFormer violates TDP nearly 50% of the time. Twill adds only ~15ms per control event, under 2% of inference latency even at high load. All main scheduling operations (decision, migration, DVFS adjustment) are constant-time due to the small number of clusters and queued tasks on edge platforms (Taufique et al., 1 Jul 2025).

7. Limitations and Future Directions

Both Twill systems are limited by externalities not captured in their explicit models. On GPUs, Twill presumes externally provided tile sizes and does not optimize for startup/drain code, nor for code size or spill-aware latency beyond II minimization. Support is currently scope-limited to singly-nested loop programs. On edge platforms, DVFS control granularity and ONNX state management cap the efficiency of migration and freeze/unfreeze. Expanding Twill’s applicability to control-flow-rich kernels, end-to-end compiler integration, and automatic tile-size or model partitioning are ongoing research challenges. These limitations highlight directions for extending the formal constraint-based approach to richer workloads, broader architectures, and deeper compiler-system integration (Soi et al., 19 Dec 2025, Taufique et al., 1 Jul 2025).