CuTe Layouts: Hierarchical Tensor Mapping

• CuTe layouts are algebraic specifications that map multi-dimensional tensor data, threads, and instructions onto linear resources using hierarchical, shape–stride representations.
• The layout algebra supports operations like composition, coalescence, inversion, and logical division, enabling efficient kernel construction and static verification.
• CuTe layouts underpin NVIDIA's CUTLASS library, offering a unified framework to express complex memory, thread, and instruction mappings in modern GPU systems.

Searching arXiv for papers on CuTe layouts and closely related formalizations. CuTe layouts are abstract specifications of how multi-dimensional tensor data and threads are mapped to linear resources such as memory and registers. In the CuTe formulation, a layout is written as a shape–stride object, typically $L = S : D$ or $H = s : d$, and is equipped with a layout algebra including composition, coalescence, complementation, division, tiling, and inversion. This representation directly extends traditional flat shape/stride tensor layouts to hierarchical forms required by modern GPU tensor instructions, and it is used as the foundation of NVIDIA’s CUTLASS library and related efforts including CuTe DSL (Cecka, 2 Mar 2026).

1. Hardware motivation and conceptual scope

CuTe was introduced for the regime in which tensor cores, hardware-optimized copy instructions, and thread-cooperative memory movement prescribe fixed and often complex data layouts. The cited treatment places this in the context of Volta, Turing, Ampere, Hopper, and Blackwell GPUs, where MMA/WMMA, TMA, and TMEM instructions require layouts that are not naturally expressible by ordinary flat row-major or column-major indexing (Cecka, 2 Mar 2026).

In this setting, a layout is not restricted to a memory layout for tensor elements. The same formal object is used for data layouts, thread layouts, and instruction layouts. A data layout maps logical tensor coordinates to offsets; a thread layout maps thread indices to logical coordinates; an instruction layout maps architecturally specified coordinates to offsets or fragment positions. CuTe’s central claim is that these three cases should be manipulated by a single algebra rather than by unrelated ad hoc index formulas (Cecka, 2 Mar 2026).

This scope distinguishes CuTe from conventional flat tensor descriptors. Traditional shape/stride representations can encode row-major, column-major, padded, and uniformly strided layouts, but they do not naturally encode hierarchical tiles, interleavings, bank-swizzling patterns, or thread/value layouts. CuTe generalizes the familiar “shape + stride” idiom rather than discarding it: flat layouts remain special cases, while hierarchical and instruction-specific layouts become first-class objects (Cecka, 2 Mar 2026).

2. Hierarchical representation: shape, stride, coordinates, and codomain

A CuTe shape is an HTuple of positive integers. An HTuple(T) is either an element of $T$ or a Tuple(HTuple(T)), so shapes may be arbitrarily nested. Two such tuples are congruent when they have the same tree shape, and weakly congruent when one is at least as refined as the other. The shape and stride of a layout must be congruent (Cecka, 2 Mar 2026).

The size of a shape is the product of its leaf integers. For a flat layout $H = s : d$ with $s = (s_0,\dots,s_{n-1})$, the ISL-based formalization defines the integral coordinate space

$$C = \{ [c] : 0 \le c < P \}, \quad P = \prod_{i=0}^{n-1} s_i,$$

and the natural coordinate space

$$C_n = \{ [c_0,\dots,c_{n-1}] : 0 \le c_i < s_i \text{ for all } i \}.$$

The coordinate mapping $M_C^H$ uses colexicographic ordering:

$$M_C^H = \{ c \rightarrow [c \bmod s_0,\ \lfloor c/s_0 \rfloor \bmod s_1,\ \dots,\ \lfloor c/\prod_{i=0}^{n-2} s_i \rfloor ] : 0 \le c < P \},$$

while the index mapping is affine:

$$M_I^H = \{[c_0,\dots,c_{n-1}] \rightarrow [\sum_{i=0}^{n-1} c_i d_i] : \forall i,\ 0 \le c_i < s_i \}.$$

Their composition,

$H = s : d$0

is the layout mapping itself (Bhaskaracharya et al., 13 Nov 2025).

The 2026 representation paper gives the same idea in semimodule form. A stride is an HTuple(D) congruent with the shape, where $H = s : d$1 may be $H = s : d$2, a coordinate space, or a bit-vector space such as $H = s : d$3. The layout is then $H = s : d$4, with evaluation by an inner product on natural coordinates. This permits ordinary integer offsets, coordinate-valued layouts, and binary swizzles to be expressed within one framework (Cecka, 2 Mar 2026).

Standard flat layouts remain immediate special cases. Row-major $H = s : d$5 yields $H = s : d$6; column-major $H = s : d$7 yields $H = s : d$8. Hierarchical layouts such as $H = s : d$9 preserve the same underlying mapping while exposing a nested coordinate structure that can be folded, unfolded, and reinterpreted algebraically (Bhaskaracharya et al., 13 Nov 2025).

3. Layout algebra

The defining feature of CuTe is not only its representation but its algebra. Every layout can be written as a concatenation of sublayouts,

$T$0

with evaluation

$T$1

This exposes the per-mode decomposition on which “by-mode” operations are built (Cecka, 2 Mar 2026).

Coalescence flattens or simplifies a layout while preserving its action on integral coordinates. The result has the same size and the same mapping $T$2 offset, but reduced hierarchy or rank. The cited examples include $T$3, which coalesces to $T$4, and $T$5, which coalesces to $T$6 (Cecka, 2 Mar 2026).

Composition is the central transformation operator. Given layouts $T$7 and $T$8, the composition $T$9 satisfies

$H = s : d$0

when the codomain of $H = s : d$1 is compatible with the domain of $H = s : d$2. The representation paper makes explicit that composition is governed by stride divisibility and shape divisibility conditions; if these fail, no CuTe layout exists that exactly represents the requested composed mapping over the intended finite domain (Cecka, 2 Mar 2026). This is the algebraic mechanism behind transposes, reblocking, thread partitioning, and instruction-specific reinterpretations.

Inversion appears in right, left, and full forms. A right-pseudo-inverse $H = s : d$3 and a left-pseudo-inverse $H = s : d$4 support reasoning about contiguity, injectivity, and membership of architecturally required offsets in a data layout. Full inverses exist for bijective or “compact” layouts (Cecka, 2 Mar 2026).

Complementation constructs a layout whose image fills ordered holes outside the image of a given layout. CuTe then defines logical product

$H = s : d$5

and logical divide

$H = s : d$6

These operators are the basis of tiling and partitioning. The paper identifies zipped_divide as the practical operator that repacks the result into a tile coordinate and a grid coordinate, which is then used pervasively to partition matrices across blocks, warps, and threads (Cecka, 2 Mar 2026).

4. Formal semantics: integer-set relations and categorical foundations

A major subsequent development is the formalization of CuTe layouts as integer set relations in ISL. In that account, coordinate spaces, index spaces, and layout mappings are all explicit Presburger-style relations. Composition, inversion, and complement are implemented directly by ISL operations such as relation composition, inverse, set difference, range, domain, and lexicographic minimization. This gives mathematically checkable semantics for basic layouts, hierarchical layouts, and swizzles, and it supports correctness verification and cross-system reasoning with Triton linear layouts (Bhaskaracharya et al., 13 Nov 2025).

This formalization treats hierarchical layouts by flattening them, computing the flat layout mapping, and then reconstructing index mappings for alternative compatible shapes. A key consequence is that some equivalent coordinate organizations yield affine index mappings, while others yield quasi-affine ones. Since CuTe syntax expects shape/stride-based layouts with strictly affine index mappings, whether a compatible shape is representable in CuTe becomes a precise semantic question rather than an informal judgment (Bhaskaracharya et al., 13 Nov 2025).

The same paper also models CuTe swizzles $H = s : d$7 as integer set relations. Scalar bitwise definitions are translated into binary coordinate mappings, binary swizzle mappings, and scalar layout mappings. The resulting relations are quasi-affine and precise enough to verify properties such as involution, while also showing that CuTe swizzles are a special case of the binary linear transformations associated with Triton layouts (Bhaskaracharya et al., 13 Nov 2025).

A different line of formalization gives categorical foundations for a class of tractable layouts. It defines two categories, Tuple and Nest, whose morphisms encode flat and nested layouts, respectively. The paper proves a one-to-one correspondence between non-degenerate tractable layouts and non-degenerate standard-form Nest morphisms, and establishes compatibility theorems such as

$H = s : d$8

for composition, along with analogous results for coalesce, complement, logical division, and logical product (Carlisle et al., 9 Jan 2026).

These two formalisms are complementary. The ISL semantics emphasizes full relation-level analysis, quasi-affinity, swizzles, and cross-system unification; the categorical account emphasizes the internal algebra of tractable layouts and proves that the operations used in CuTe practice correspond exactly to categorical constructions on morphisms (Bhaskaracharya et al., 13 Nov 2025, Carlisle et al., 9 Jan 2026).

5. Role in CUTLASS, thread/value layouts, and compile-time verification

CuTe layouts are described as the foundation of NVIDIA’s CUTLASS library and of CuTe DSL, and the 2026 representation paper states that they have been successfully deployed in production systems (Cecka, 2 Mar 2026). This deployment context matters because the layout algebra is not presented as a purely descriptive notation. It is the mechanism by which kernels derive, propagate, and statically verify the operand layouts prescribed by modern hardware.

The same layout type is used for memory tensors, thread arrangements, and instruction fragments. A representative example is the Ampere FP64 tensor core $H = s : d$9-matrix partition, expressed as a thread/value layout

$s = (s_0,\dots,s_{n-1})$0

When a data layout $s = (s_0,\dots,s_{n-1})$1 is composed with this thread/value layout, the result is a mapping from (thread, value) to memory offset, and slicing that composition yields the values owned by a given thread (Cecka, 2 Mar 2026).

The paper gives analogous instruction-layout examples for TMEM loads. For tcgen05.ld.32x32b.x1, the layout is represented as

$s = (s_0,\dots,s_{n-1})$2

and for tcgen05.ld.16x256b.x1 as

$s = (s_0,\dots,s_{n-1})$3

Because these are ordinary CuTe layouts, one can use left inverses, compositions, and compatibility checks to determine whether a data layout matches the instruction’s required addressing pattern (Cecka, 2 Mar 2026).

This is also the basis for generic kernel construction. The paper’s high-level GEMM is parameterized by layout types ALayout, BLayout, and CLayout; algorithmic logic is separated from the tensor-memory mapping. Tiling, partitioning, transposition, and thread assignment are introduced by layout operations such as zipped_divide, logical_product, and composition, rather than by hand-written index arithmetic (Cecka, 2 Mar 2026).

The productivity claim is concrete. The representation paper contrasts CUTLASS v2’s roughly 300 separate layout types, amounting to roughly 55,000 lines of code, with CuTe’s core implementation of roughly 3,000 lines, while asserting that CuTe represents all of those prior cases and additional ones (Cecka, 2 Mar 2026). In this sense, CuTe layouts function as a compact intermediate language for architecturally constrained tensor indexing.

6. Scope, limitations, and related directions

The most important limitation is that not every useful layout formalism coincides with every other one. The categorical framework applies to a naturally occurring class of tractable layouts and explicitly does not cover arbitrary swizzles or Triton-like $s = (s_0,\dots,s_{n-1})$4-linear layouts. Its closure properties and theorems therefore hold for a large practical subset rather than for all conceivable CuTe-style mappings (Carlisle et al., 9 Jan 2026).

Conversely, the ISL-based formalization is deliberately more expressive than CuTe’s native shape/stride layouts plus swizzles. It notes an expressiveness gap: there are integer relations, including modulo-based permutations such as

$s = (s_0,\dots,s_{n-1})$5

that are naturally expressible in ISL but are not expressible as CuTe layouts or Triton linear layouts (Bhaskaracharya et al., 13 Nov 2025).

A second limitation is reconstruction. Given only a one-dimensional layout relation $s = (s_0,\dots,s_{n-1})$6, inferring both shape and strides in full generality is left open. The ISL paper provides algorithms when either the shape or the strides are known, but not for the unrestricted inverse problem (Bhaskaracharya et al., 13 Nov 2025).

A third limitation concerns complexity. Presburger arithmetic and polyhedral operations can be expensive in the worst case, although the same paper argues that realistic deep-learning layouts remain within practical scale, citing ranks up to 6 and tiling depths around 4, corresponding to at most 24 dimensions in the polyhedral representation (Bhaskaracharya et al., 13 Nov 2025).

The forward direction is therefore twofold. One line seeks richer compiler integration and cross-system optimization, including MLIR- or FPL-oriented layout-aware compilation and unified reasoning across CuTe and Triton (Bhaskaracharya et al., 13 Nov 2025). The other line seeks deeper internal understanding of CuTe’s existing algebra, as in the categorical account’s characterization of tractable layouts and its proof that composition, complement, logical division, and logical product on morphisms align with CUTLASS behavior (Carlisle et al., 9 Jan 2026). Together, these developments position CuTe layouts as both a practical kernel-construction language and a subject of ongoing formalization.