Tile-Major Layout in Memory Systems

Updated 8 July 2026

Tile-major layout is a tile-centric data organization strategy that partitions matrices into sub-blocks aligned with hardware execution and memory access patterns.
It encompasses various schemes, including UMDAM’s column-major tile ordering, TileLens’ 2D memory reshaping, and DSLs like TileLang and Hexcute for composable layout mappings.
Performance evaluations report up to 3× reduction in time-to-first-token and significant bandwidth and latency improvements by closely matching data layout to computational needs.

Tile-major layout is a tile-centric organization of data or work in which sub-blocks rather than full rows or columns become the primary unit of storage, movement, and execution. In recent systems literature, the term denotes a family of closely related schemes rather than a single canonical format: UMDAM uses a column-major, tile-based layout for unified NPU-PIM co-execution, TileLens uses a two-dimensional global memory layout that reshapes each contiguous memory block into a rectangle aligned with compute tiles, and tile DSLs such as TileLang and Hexcute treat layouts as explicit mappings from logical indices to hardware-oriented tile placements (Huang et al., 5 Nov 2025, Ju et al., 4 Jul 2026, Wang et al., 24 Apr 2025, Zhang et al., 22 Apr 2025). Across these settings, the recurring objective is to match data organization to tiled computation so as to preserve locality, interleaving, bandwidth utilization, or hardware legality without incurring redundant layout conversion.

1. Definition and structural variants

In UMDAM, the tile-major layout is defined as a tile-oriented column-major layout for a 2D weight matrix of size $M \times N$ . The matrix is partitioned into tiles of size $tile\_height \times tile\_width$ ; inside each tile, weights are stored in column-major order; and the tiles themselves are placed in column-major order with respect to the overall matrix. The tile height aligns with DRAM interleaving granularity, and the tile width is often set to the number of DRAM banks so that each tile maps to available hardware parallelism (Huang et al., 5 Nov 2025).

TileLens defines tile-major layout differently but with the same tile-centric premise. Rather than storing matrix data as a one-dimensional strip in row-major or column-major order, it reshapes each contiguous memory block into a two-dimensional rectangle. For a matrix in column-major layout, a tile-major mapping with memory tile size $a \times b$ packs all elements of an $a \times b$ tile contiguously in memory and aligns that tile to the storage granularity used by large-granularity memory systems such as HBF and RoMe (Ju et al., 4 Jul 2026).

TileLang and Hexcute generalize the notion further. TileLang represents layout as a mapping from logical indices to physical addresses and supports composition with blocking, padding, swizzling, and fragment placement. Hexcute makes tiles the central abstraction for both computation and memory layout, with tile-level operations described via thread-value layouts and synthesized memory layouts (Wang et al., 24 Apr 2025, Zhang et al., 22 Apr 2025).

A common simplification is to treat tile-major layout as merely “blocked row-major.” The literature considered here does not support that simplification. UMDAM is explicitly column-major within and across tiles, TileLens is a 2D reshaping of coarse memory blocks, and TileLang and Hexcute allow swizzled or hierarchical variants. This suggests that “tile-major layout” is best understood as a hardware-affinity design space centered on tiles, not as a single fixed linearization.

2. Formal mappings and address generation

UMDAM gives an explicit address construction algorithm. For each tile $(tile\_row\_idx, tile\_col\_idx)$ and each element $(local\_row\_idx, local\_col\_idx)$ , the mapping extracts address fields from tile-local and tile-global coordinates and composes them as

$addr \gets (row, col_M, ba, ra, ch, col_L, offset).$

The DRAM address order is

$\text{(MSB)}\quad \text{Row}~|~\mathrm{Col\_M}~|~\mathrm{Bank}~|~\mathrm{Rank}~|~\mathrm{Channel}~|~\mathrm{Col\_L}~|~\mathrm{Offset}\quad \text{(LSB)}.$

The field widths are configurable; for LPDDR5, the paper gives the example: burst size $32$B, row size $2$KB, $tile\_height \times tile\_width$ 0 column bits, $tile\_height \times tile\_width$ 1 offset bits, with $tile\_height \times tile\_width$ 2 bits and $tile\_height \times tile\_width$ 3 bits for $tile\_height \times tile\_width$ 4B interleaving (Huang et al., 5 Nov 2025).

TileLens provides a complementary mapping at coarse memory granularity. For a tile-major layout with memory tile size $tile\_height \times tile\_width$ 5 and per-element size $tile\_height \times tile\_width$ 6, element $tile\_height \times tile\_width$ 7 is decomposed by

$tile\_height \times tile\_width$ 8

and the address offset is

$tile\_height \times tile\_width$ 9

This packs the full $a \times b$ 0 tile contiguously and aligns it to the access granularity of the memory system (Ju et al., 4 Jul 2026).

TileLang states the layout abstraction in more general algebraic form. A layout function is expressed as

$a \times b$ 1

with physical address modeled by

$a \times b$ 2

In this formulation, tile-major layout is not a special case hard-coded into the language; it is one class of mapping that can be composed with scheduling and memory-scope decisions (Wang et al., 24 Apr 2025).

These formulations are closely related in intent. UMDAM decomposes an address into DRAM controller fields, TileLens decomposes it into tile-grid and tile-local coordinates, and TileLang abstracts the same idea as layout composition. A plausible implication is that tile-major layout becomes increasingly useful as the system stack exposes more of the memory hierarchy and address-generation path.

3. Hardware affinity and execution coupling

The principal systems argument for tile-major layout is that it reconciles the access patterns of tiled compute with the constraints of the underlying memory system. In UMDAM, each DRAM bank holds a complete column, or local tile column, so that the bank’s PIM unit can perform direct column access during GEMV, while placing channel bits lower in the address order preserves DRAM interleaving for the NPU’s batched, row-wise accesses. The layout is therefore designed simultaneously for column-locality and interleaving (Huang et al., 5 Nov 2025).

TiledAttention shows the same coupling at the level of GPU shared memory. Each CUDA block owns a tile of queries $a \times b$ 3 and streams over $a \times b$ 4 tiles of size $a \times b$ 5. Shared memory is used to stage $a \times b$ 6 and $a \times b$ 7 tiles, and the tile-major layout there means that data is organized per tile, matching the program’s block decomposition rather than a purely contiguous row-major or column-major order. The kernel exposes tile sizes $a \times b$ 8, the number of streaming stages, and shared-memory swizzles as explicit tuning knobs (Khan, 2 Mar 2026).

TileLang describes the same hardware-affinity principle in a more general programming-model form. Buffers are placed in global memory, shared memory, or registers with primitives such as T.alloc_shared and T.alloc_fragment; the compiler’s Layout Inference Pass derives a Layout object for each buffer; and fragment layouts determine how a tile is distributed across threads’ registers. Hexcute likewise treats tiles as first-class types and infers layouts so that copy and mma operations satisfy instruction-level layout constraints, optionally with swizzling for bank-conflict avoidance (Wang et al., 24 Apr 2025, Zhang et al., 22 Apr 2025).

System	Tile-major interpretation	Reported purpose
UMDAM	Column-major within tiles and across tiles	Unified NPU-PIM access without re-layout
TiledAttention	Shared-memory organization per streamed tile	On-chip reuse for tiled SDPA
TileLens	Two-dimensional global memory blocks	Eliminate read amplification in LGMS
TileLang / Hexcute	Composable or inferred tile mappings	Hardware-matching kernels and layouts

What unifies these cases is not a single physical ordering, but a consistent design rule: the tile is chosen as the unit at which data placement, execution mapping, and hardware resource allocation are made compatible.

4. Performance consequences

UMDAM reports that, on OPT models, the unified column-major, tile-based layout reduces time-to-first-token (TTFT) by up to $a \times b$ 9 and time-to-last-token (TTLT) by $a \times b$ 0. The paper also states that, even with long decode phases, there is still at least $a \times b$ 1 latency reduction, and that the approach incurs no extra memory overhead or bandwidth loss because it uses a single unified weight layout (Huang et al., 5 Nov 2025).

TileLens evaluates tiled matrix-multiplication kernels from Qwen-3 30B and Llama-3.1 70B on a cycle-level simulator. With conventional layouts, the geomean slowdown on HBF-augmented GPUs is reported as 1.61–6.49x; by combining a tile-major layout with an adaptive hardware prefetcher, TileLens reduces this to within 1% of an HBM-only baseline on a system with a 5us HBF NAND read latency. The paper attributes the improvement to the elimination of read amplification, bandwidth waste, cache pollution, and straggler-induced stalls (Ju et al., 4 Jul 2026).

The kernel-programming literature reports related gains. TileLang states that it can achieve speedups of up to 1.25× over Triton on GEMM benchmarks while ensuring bank conflict-free execution across all tested devices through layout swizzling (Wang et al., 24 Apr 2025). Hexcute reports 1.7-11.28 $a \times b$ 2 speedup over existing DL compilers for mixed-type operators and up to 2.91 $a \times b$ 3 speedup in the end-to-end evaluation, with the paper connecting those gains to layouts that match Tensor Core expectations and avoid conversion overhead (Zhang et al., 22 Apr 2025). TiledAttention states that production fused baselines remain stronger overall, but also reports that it dramatically outpaces unfused and “eager” attention in PyTorch (>10×) and can reach near-parity in certain parameter regimes (Khan, 2 Mar 2026).

Comparable benefits appear outside dense tensor kernels. TASM reports that tile-based video storage can speed up subframe selection queries by an average of over 50% and up to 94%, and can improve the throughput of the full scan phase of object detection queries by up to 2X (Daum et al., 2020). Although this is a storage-layout setting rather than tensor memory layout, it reflects the same core principle: when the physical layout matches the query’s spatial access unit, unnecessary decoding work falls.

5. Programmability, synthesis, and failure modes

Recent tile DSLs treat tile-major layout as something to be synthesized or parameterized rather than manually hard-coded. TileLang decouples scheduling space (thread binding, layout, tensorize and pipeline) from dataflow and exposes layout through annotations and primitives such as T.annotate_layout, T.use_swizzle, T.Pipelined, and T.[Parallel](https://www.emergentmind.com/topics/additive-parallel-correction). The compiler then performs layout inference for shared memory, registers, and fragments, so that tiled data movement and compute can be expressed in terms of T.copy, T.gemm, and related operators (Wang et al., 24 Apr 2025).

Hexcute pushes this further with a type-inference-based synthesis procedure. Its layout constraints relate source, destination, and instruction layouts; for example, for copy(a, b) the paper gives

$a \times b$ 4

and for mma(a, b, c) it derives more elaborate constraints tying the $a \times b$ 5, $a \times b$ 6, and $a \times b$ 7 dimensions of operand layouts to the instruction interface. Shared-memory layouts are solved so that

$a \times b$ 8

optionally composed with a swizzle function to remove bank conflicts (Zhang et al., 22 Apr 2025).

The same abstraction power creates new correctness risks. A systematic study of tile-program code generation bugs curates 401 bug reports and identifies 301 tile-program codegen bugs. The reported manifestations are crashes (58%), silent wrong results (36%), and performance bottlenecks (5%). The study categorizes root causes into control-flow and scheduling (5%), IR construction and transformation (16%), tile mapping and launch (6%), memory bugs (19%), type and operator bugs (48%), and device-specific bugs (4%) (Rathnasuriya et al., 19 May 2026).

The bug study also identifies a recurring misconception in practice: shape-aligned cases are not representative. The reported bugs are often triggered by shapes that are not exact multiples of tile size, by non-contiguous or padded layouts, by exotic dtypes, or by backend-specific legality constraints. The recommended detection methods are therefore structured input generation, differential or metamorphic oracles, and canary/injection-based testing, rather than relying on random fuzzing alone (Rathnasuriya et al., 19 May 2026).

The tile-major idea has close relatives in other computational domains, although the semantics are not identical. In panoramic room-layout estimation, DMH-Net projects an equirectangular panorama into six cubemap tiles and performs prediction individually on each cubemap tile, then assembles the final 3D room layout through post-processing and optimization. The motivation is that cubemap tiles are perspective, distortion-free images aligned with the Manhattan frame, so the per-tile decomposition is both CNN-friendly and geometrically interpretable (Zhao et al., 2022).

In large-graph visualization, a tile pyramid is used for semantic zoom. The graph layout is spatially partitioned into levels of tiles, higher-ranked nodes remain readable at coarser zoom levels, and edges are rerouted at each level using sleeve routing on the dual graph of a Constrained Delaunay Triangulation. The result is a precomputed, browser-side layout whose interaction cost depends on the current viewport and tile set rather than on the entire graph (Nachmanson et al., 17 May 2026).

In video analytics, TASM uses codec-supported tiles as spatially independent rectangular regions that can be decoded separately. It chooses uniform and non-uniform tile layouts based on video content and query workload, with layout changes constrained to keyframe boundaries, and uses a regret-based dynamic tuning algorithm when workload information is incomplete (Daum et al., 2020).

These systems are not all instances of the same memory layout problem, but they show that the tile-centric viewpoint generalizes. This suggests that tile-major layout is part of a broader methodological shift: replacing monolithic linear or global representations with representations whose primitive unit is the tile, because the tile is often the unit at which access granularity, parallel execution, or semantic visibility is actually determined.