CUDA Tile (CuTile) Overview
- CUDA Tile (CuTile) is a tile-centric GPU programming model that abstracts tensor computations into fixed-size tiles to streamline kernel development.
- It leverages front ends in Python and Rust to enforce safety via partitioned mutability and schedule-visible tile programs, reducing data races.
- CuTile demonstrates competitive performance through careful hardware-aware scheduling, optimized memory traversal, and scalable tile operations.
CUDA Tile, often shortened to CuTile in recent literature, denotes a tile-centric approach to GPU kernel development in which computation is formulated over tensor tiles rather than explicit SIMT index arithmetic. In contemporary usage, the term most often refers to NVIDIA’s CUDA Tile / Tile IR stack and to front ends built on that stack, including cuTile Python and cuTile Rust; in earlier CUDA-era work, “tiling” referred more narrowly to the 2D shape of a CUDA thread block used to partition an output domain (Elibol et al., 14 Jun 2026).
1. Terminology and historical usage
The literature uses “CUDA Tile” and closely related terms in more than one sense. The most important distinction is between early block-geometry tuning and later tile-programming systems.
| Usage | Meaning | Representative source |
|---|---|---|
| Early CUDA tiling | 2D CUDA thread-block dimensions used to partition the output image domain | (Xu et al., 2010) |
| CUDA Tile / Tile IR | Python-based, tile-centric abstraction with tile loads, stores, and MMA | (Yadav et al., 25 Apr 2026) |
| cuTile Rust | Safe, tile-based GPU kernel programming system for Rust built on CUDA Tile / Tile IR | (Elibol et al., 14 Jun 2026) |
In the 2010 CUDA 2.1-era study on bilinear image interpolation, tiling meant the choice of block-level thread-block shape such as , , or . The paper explicitly studied block-level tiling, not deeper register tiling, and treated tile shape as a performance-tuning parameter that changes occupancy, memory traversal, and latency hiding (Xu et al., 2010). By contrast, the 2026 work on cuTile Rust describes a new programming model in which kernels are written over tiles, mutable outputs are partitioned into disjoint sub-tensors before launch, and the compiler/runtime preserve the host’s ownership contract during execution (Elibol et al., 14 Jun 2026).
A closely related but distinct abstraction is CuTe, which is presented as a mathematical specification for representing and manipulating tensors through hierarchical layouts and a layout algebra, and is described as the foundation of CUTLASS and related efforts including CuTe DSL (Cecka, 2 Mar 2026). This suggests that recent CUDA tile programming spans at least three layers: layout algebra, tile-centric kernel front ends, and backend lowering to Tensor Core and memory-movement instructions.
2. Tile-centric programming model
In the contemporary CuTile literature, the tile is the core abstraction. A kernel is not written in SIMT style with explicit per-thread indexing, warp intrinsics, or hand-managed shared-memory protocols. Instead, the programmer writes sequential code over fixed-size immutable tiles and mutable output tile views, and a GPU launch is modeled as a grid of tile programs, where each tile program is a single logical thread executing over one or more tiles of tensor data (Elibol et al., 14 Jun 2026).
This programming style is visible in both the Rust and Python-facing descriptions. In cuTile Rust, loading from memory produces a tile value; tile operations such as elementwise arithmetic or mma produce new tiles; and storing writes a whole tile into a mutable output sub-tensor. In the independent evaluation of NVIDIA’s CUDA Tile, the programming model is described as Python-based and tile-centric, with three core operations: ct.load / ct.store for tile-based memory movement, ct.mma for tile-level Tensor Core matrix multiply-accumulate, and @ct.kernel with ByTarget for kernel definition and architecture-specific targeting or tuning (Elibol et al., 14 Jun 2026).
The same tile-programming model is used for research kernels. TiledAttention, for example, is a scaled dot-product attention forward operator implemented in cuTile Python (TileIR) and exposed as a PyTorch-callable function. Its stated value is that schedule choices remain directly editable from Python: tile sizes, staging or prefetch depth, and shared-memory layouts are all explicit schedule-level controls rather than buried in template-heavy CUDA or CUTLASS rewrites (Khan, 2 Mar 2026). This suggests that CuTile’s main abstraction boundary is not “operator call” but “schedule-visible tile program.”
3. Ownership, memory semantics, and host execution
A major development in the modern CuTile line is the extension of language-level safety into GPU tile programs. cuTile Rust is presented as a safe, tile-based GPU kernel programming system for Rust, built on NVIDIA’s CUDA Tile / Tile IR stack, with the central claim that a large and important class of GPU kernels can be made data-race-free by construction without measurable runtime cost (Elibol et al., 14 Jun 2026).
The key safety mechanism is partitioned mutability. Mutable outputs are split into disjoint sub-tensors before launch, and the mapping from tile programs to mutable sub-tensors is injective: no output partition is assigned to more than one tile program. Immutable inputs are broadcast as shared read-only references. Within a tile program, mutable tensor accesses are lowered to token-ordered Tile IR operations, while immutable reads remain unconstrained. The paper summarizes this as &mut Tensor mapping to token-ordered operations and &Tensor mapping to unconstrained operations, and states: “Every execution of a kernel written in safe cuTile Rust is data-race-free under Tile IR’s memory model” (Elibol et al., 14 Jun 2026).
The host-side execution model is also part of the abstraction. cuTile Rust’s DeviceOp trait represents lazy, typed GPU work and composes with synchronous launches, asynchronous pipelines, and CUDA graph replay. The paper emphasizes a “same-type-in-same-type-out” invariant: if the caller passes an Arc<Tensor<T>>, the recovered value is again an Arc<Tensor<T>>; if the caller passes a borrow such as &Tensor<T>, that borrow form is preserved after execution (Elibol et al., 14 Jun 2026). This allows Rust’s borrow checker to reason about the whole launch lifecycle.
The system does not claim to eliminate unsafe entirely. It explicitly provides unchecked_accesses and raw-pointer kernel parameters as local escape hatches, and the Grout inference engine uses safe cuTile Rust kernels where possible but unchecked accesses or raw pointers for performance-critical attention and fused normalization kernels. The paper therefore characterizes the contribution as narrowing and localizing the unsafe surface, not removing it (Elibol et al., 14 Jun 2026).
4. Scheduling, locality, and portability
A persistent theme in the literature is that tile abstraction does not remove the need for hardware-conscious scheduling. The early CUDA-era interpolation study already showed that an optimized tiling strategy on one GPU model is not always a good solution on another, and that the best tile shape depends on both hardware resources and external conditions such as image scale. On the tested GTX 260 and GeForce 8800 GTS, appeared to provide the best performance for larger scales, but the paper explicitly rejected the idea of a universally optimal tile (Xu et al., 2010).
Recent CuTile work reaches a similar conclusion at a different abstraction level. In the GB10 FlashAttention study, CuTile is described as helping express a high-performance tiled kernel, but the paper argues that the abstraction does not automatically give good cache behavior. Its baseline CuTile traversal uses a regular cyclic KV sweep, which creates a poor L2 reuse distance once KV exceeds L2 capacity. The proposed Sawtooth Wavefront Reordering changes the order in which each CTA scans KV tiles for successive Q tiles, and the paper reports 50% or greater reduction in L2 misses and up to 60% increase in throughput on GB10 (Zhu et al., 22 Jan 2026). This suggests that tile algebra and Tensor Core structure are not sufficient; inter-CTA temporal ordering remains a first-class optimization knob.
Cross-architecture evaluation reinforces the same point. An independent study on H100 NVL, B200, and RTX PRO 6000 Blackwell Server Edition describes CuTile effectiveness as strongly workload- and architecture-dependent. On datacenter-class Blackwell, CuTile achieves up to 1007 TFLOP/s for fused attention and outperforms FlashAttention-2 by 2.5x, while the same CuTile attention kernel achieves only 53% of FlashAttention-2 throughput on RTX PRO 6000. The paper contrasts this with Triton, which sustains 62–101% of cuBLAS performance across the tested platforms without architecture-specific tuning (Yadav et al., 25 Apr 2026). A plausible implication is that CuTile’s current optimization stack is powerful but not yet uniformly portable.
5. Attention kernels and inference systems
Attention has become a central proving ground for CuTile. TiledAttention implements a scaled dot-product attention forward operator in cuTile Python and exposes it as a PyTorch-callable function. The kernel uses online softmax and tiled streaming, avoids materializing the full score matrix, and keeps schedule controls explicit at the Python level. Across an 80-point benchmark grid on DGX GB10, it reaches a mean throughput ratio of fused PyTorch SDPA, but in the regime it reaches a mean ratio of with 4 wins out of 20 cases, while delivering very large gains over unfused baselines: over the PyTorch SDPA math path and 0 over standard eager attention (Khan, 2 Mar 2026).
The independent CuTile evaluation reports even stronger fused-attention results on B200. For causal FMHA with 32 heads, head dimension 1, batch size 2, and sequence lengths up to 3, the CuTile kernel rises from 405.9 TFLOP/s at sequence length 4 to 1007.4 TFLOP/s at 5, versus 400.7 TFLOP/s for FlashAttention-2 and 525.6 TFLOP/s for Triton at the same 6 point. The same paper also emphasizes that the CuTile FMHA kernel is around 60 lines of Python kernel code (Yadav et al., 25 Apr 2026).
cuTile Rust extends this story from kernels to an end-to-end inference engine. Grout, built on cuTile Rust, is described as a lean, model-specialized batch-1 Qwen3 engine rather than a general serving stack. Its abstract reports 171 generated tokens/s for Qwen3-4B on the NVIDIA GeForce RTX 5090 and 82 generated tokens/s for Qwen3-32B on the B200 in batch-1 decode, competitive with vLLM and SGLang and consistent with an HBM roofline sanity check (Elibol et al., 14 Jun 2026). The paper’s framing is that tile-based Rust kernels, typed host execution, and CUDA graph replay can support a realistic inference path, not merely isolated microbenchmarks.
6. Performance profile, limitations, and related research directions
The strongest empirical argument for contemporary CuTile is that high-level tile abstractions can remain close to backend limits on selected workloads. On the NVIDIA B200 GPU, cuTile Rust reaches 7.02 TB/s for elementwise add at 7 and 2.07 PFlop/s for 8 f16 GEMM, which the paper interprets as 96.4% of cuBLAS. The same evaluation reports that safe Rust matches unsafe Rust within 9, and that cuTile Rust matches cuTile Python within measurement noise (Elibol et al., 14 Jun 2026). The host execution model also spans distinct latency regimes: chained sync and async converge to about 0, while CUDA graph replay approaches about 1 on the tested RTX 5090 launch-overhead benchmark (Elibol et al., 14 Jun 2026).
The limitations are equally explicit. cuTile Rust works well for kernels naturally expressed through loads, stores, reductions, matrix multiplies, normalization, and attention-style tensor operations, but it gives up SIMT-level control such as explicit warp primitives and manual shared-memory management in exchange for single-threaded tile semantics and tractable static safety checking. The safe mutable partition model is tied to CUDA’s 3D grid, limiting mutable tensor rank in the safe partitioned form, and some optimized kernels still require unchecked_accesses or raw pointers. GEMM still trails cuBLAS at some matrix sizes, so Grout uses cuBLAS for model GEMMs (Elibol et al., 14 Jun 2026).
Related work suggests broader directions for tile-centric GPU programming. Hexcute presents a tile-based programming language that exposes shared memory and register abstractions and automates layout and task-mapping synthesis (Zhang et al., 22 Apr 2025). Cypress presents a task-based programming model with sequential semantics and a mapping specification that lowers to Hopper-style CUDA with TMA- and Tensor-Core-oriented pipelines (Yadav et al., 9 Apr 2025). A systematic study of tile-program code generation bugs collected 301 analyzable tile-program codegen bugs across open-source repositories and explicitly noted that CUDA Tile had too few publicly confirmed codegen bug reports during the collection window to support reliable analysis (Rathnasuriya et al., 19 May 2026). This suggests that the maturation of CuTile is likely to depend not only on faster kernels, but also on tile-aware debugging, testing, and repair infrastructure.
In that broader context, CUDA Tile is best understood not as a single kernel template or a synonym for block size, but as a family of tile-centric abstractions for expressing GPU computation, movement, and scheduling. The modern literature treats tiles simultaneously as units of data layout, units of Tensor Core computation, units of ownership and aliasing control, and units of host-visible execution composition. The resulting systems can be concise and fast, but they remain highly sensitive to layout algebra, cache behavior, and backend specialization.