---
title: Tile-Major Layout in Memory Systems
url: https://www.emergentmind.com/topics/tile-major-layout
type: topic
---

# Tile-Major Layout in Memory Systems

Tile-major layout is a tile-centric organization of data or work in which sub-blocks rather than full rows or columns become the primary unit of storage, movement, and execution. In recent systems literature, the term denotes a family of closely related schemes rather than a single canonical format: UMDAM uses a **column-major, tile-based layout** for unified NPU-PIM co-execution, TileLens uses a **two-dimensional global memory layout** that reshapes each contiguous memory block into a rectangle aligned with compute tiles, and tile DSLs such as TileLang and Hexcute treat layouts as explicit mappings from logical indices to hardware-oriented tile placements [2511.03293][2607.04031][2504.17577][2504.16214]. Across these settings, the recurring objective is to match data organization to tiled computation so as to preserve locality, interleaving, bandwidth utilization, or hardware legality without incurring redundant layout conversion.

## 1. Definition and structural variants

In UMDAM, the tile-major layout is defined as a **tile-oriented column-major layout** for a 2D weight matrix of size $M \times N$. The matrix is partitioned into tiles of size $tile\_height \times tile\_width$; inside each tile, weights are stored **in column-major order**; and the tiles themselves are placed **in column-major order** with respect to the overall matrix. The tile height aligns with DRAM interleaving granularity, and the tile width is often set to the number of DRAM banks so that each tile maps to available hardware parallelism [2511.03293].

TileLens defines tile-major layout differently but with the same tile-centric premise. Rather than storing matrix data as a **one-dimensional strip** in row-major or column-major order, it **reshapes each contiguous memory block into a two-dimensional rectangle**. For a matrix in column-major layout, a tile-major mapping with memory tile size $a \times b$ packs all elements of an $a \times b$ tile contiguously in memory and aligns that tile to the storage granularity used by large-granularity memory systems such as HBF and RoMe [2607.04031].

TileLang and Hexcute generalize the notion further. TileLang represents layout as a mapping from logical indices to physical addresses and supports composition with blocking, padding, swizzling, and fragment placement. Hexcute makes tiles the central abstraction for both computation and memory layout, with tile-level operations described via thread-value layouts and synthesized memory layouts [2504.17577][2504.16214].

A common simplification is to treat tile-major layout as merely “blocked row-major.” The literature considered here does not support that simplification. UMDAM is explicitly column-major within and across tiles, TileLens is a 2D reshaping of coarse memory blocks, and TileLang and Hexcute allow swizzled or hierarchical variants. This suggests that “tile-major layout” is best understood as a hardware-affinity design space centered on tiles, not as a single fixed linearization.

## 2. Formal mappings and address generation

UMDAM gives an explicit address construction algorithm. For each tile $(tile\_row\_idx, tile\_col\_idx)$ and each element $(local\_row\_idx, local\_col\_idx)$, the mapping extracts address fields from tile-local and tile-global coordinates and composes them as
\[
addr \gets (row, col_M, ba, ra, ch, col_L, offset).
\]
The DRAM address order is
\[
\text{(MSB)}\quad \text{Row}~|~\mathrm{Col\_M}~|~\mathrm{Bank}~|~\mathrm{Rank}~|~\mathrm{Channel}~|~\mathrm{Col\_L}~|~\mathrm{Offset}\quad \text{(LSB)}.
\]
The field widths are configurable; for LPDDR5, the paper gives the example: burst size $32$B, row size $2$KB, $6$ column bits, $5$ offset bits, with $\mathrm{Col\_M}=3$ bits and $\mathrm{Col\_L}=3$ bits for $256$B interleaving [2511.03293].

TileLens provides a complementary mapping at coarse memory granularity. For a tile-major layout with memory tile size $a \times b$ and per-element size $s$, element $(i,j)$ is decomposed by
\[
T_i = \lfloor i/a \rfloor,\quad i' = i \bmod a,\qquad
T_j = \lfloor j/b \rfloor,\quad j' = j \bmod b,
\]
and the address offset is
\[
\text{offset}_\text{tile} = (T_j \cdot \tfrac{K}{a} + T_i) \cdot (a \cdot b \cdot s) + (j' \cdot a + i') \cdot s.
\]
This packs the full $a \times b$ tile contiguously and aligns it to the access granularity of the memory system [2607.04031].

TileLang states the layout abstraction in more general algebraic form. A layout function is expressed as
\[
f : \mathbb{K}^n \rightarrow \mathbb{K}^m,
\]
with physical address modeled by
\[
\text{address} = \sum_i y_i s_i.
\]
In this formulation, tile-major layout is not a special case hard-coded into the language; it is one class of mapping that can be composed with scheduling and memory-scope decisions [2504.17577].

These formulations are closely related in intent. UMDAM decomposes an address into DRAM controller fields, TileLens decomposes it into tile-grid and tile-local coordinates, and TileLang abstracts the same idea as layout composition. A plausible implication is that tile-major layout becomes increasingly useful as the system stack exposes more of the memory hierarchy and address-generation path.

## 3. Hardware affinity and execution coupling

The principal systems argument for tile-major layout is that it reconciles the access patterns of tiled compute with the constraints of the underlying memory system. In UMDAM, each DRAM bank holds a complete column, or local tile column, so that the bank’s PIM unit can perform direct column access during GEMV, while placing channel bits lower in the address order preserves DRAM interleaving for the NPU’s batched, row-wise accesses. The layout is therefore designed simultaneously for **column-locality** and **interleaving** [2511.03293].

TiledAttention shows the same coupling at the level of GPU shared memory. Each CUDA block owns a tile of queries $Q_{\text{tile}} \in \mathbb{R}^{T_M \times D}$ and streams over $K,V$ tiles of size $T_N \times D$. Shared memory is used to stage $K$ and $V$ tiles, and the **tile-major layout** there means that data is organized **per tile**, matching the program’s block decomposition rather than a purely contiguous row-major or column-major order. The kernel exposes tile sizes $(T_M, T_N)$, the number of streaming stages, and shared-memory swizzles as explicit tuning knobs [2603.01960].

TileLang describes the same hardware-affinity principle in a more general programming-model form. Buffers are placed in global memory, shared memory, or registers with primitives such as `T.alloc_shared` and `T.alloc_fragment`; the compiler’s **Layout Inference Pass** derives a `Layout` object for each buffer; and fragment layouts determine how a tile is distributed across threads’ registers. Hexcute likewise treats tiles as first-class types and infers layouts so that copy and `mma` operations satisfy instruction-level layout constraints, optionally with swizzling for bank-conflict avoidance [2504.17577][2504.16214].

| System | Tile-major interpretation | Reported purpose |
|---|---|---|
| UMDAM | Column-major within tiles and across tiles | Unified NPU-PIM access without re-layout |
| TiledAttention | Shared-memory organization per streamed tile | On-chip reuse for tiled SDPA |
| TileLens | Two-dimensional global memory blocks | Eliminate read amplification in LGMS |
| TileLang / Hexcute | Composable or inferred tile mappings | Hardware-matching kernels and layouts |

What unifies these cases is not a single physical ordering, but a consistent design rule: the tile is chosen as the unit at which data placement, execution mapping, and hardware resource allocation are made compatible.

## 4. Performance consequences

UMDAM reports that, on OPT models, the unified column-major, tile-based layout reduces **time-to-first-token (TTFT) by up to $3.0\times$** and **time-to-last-token (TTLT) by $2.18\times$**. The paper also states that, even with long decode phases, there is still **at least $14\%$ latency reduction**, and that the approach incurs **no extra memory overhead or bandwidth loss** because it uses a single unified weight layout [2511.03293].

TileLens evaluates tiled matrix-multiplication kernels from Qwen-3 30B and Llama-3.1 70B on a cycle-level simulator. With conventional layouts, the geomean slowdown on HBF-augmented GPUs is reported as **1.61–6.49x**; by combining a tile-major layout with an adaptive hardware prefetcher, TileLens reduces this to **within 1% of an HBM-only baseline** on a system with **a 5us HBF NAND read latency**. The paper attributes the improvement to the elimination of read amplification, bandwidth waste, cache pollution, and straggler-induced stalls [2607.04031].

The kernel-programming literature reports related gains. TileLang states that it can achieve **speedups of up to 1.25× over Triton** on GEMM benchmarks while ensuring **bank conflict-free execution across all tested devices** through layout swizzling [2504.17577]. Hexcute reports **1.7-11.28$\times$ speedup over existing DL compilers for mixed-type operators** and **up to 2.91$\times$ speedup in the end-to-end evaluation**, with the paper connecting those gains to layouts that match Tensor Core expectations and avoid conversion overhead [2504.16214]. TiledAttention states that production fused baselines remain stronger overall, but also reports that it **dramatically outpaces unfused and “eager” attention in PyTorch (>10×)** and can reach near-parity in certain parameter regimes [2603.01960].

Comparable benefits appear outside dense tensor kernels. TASM reports that tile-based video storage can speed up **subframe selection queries by an average of over 50% and up to 94%**, and can improve the throughput of the **full scan phase of object detection queries by up to 2X** [2006.02958]. Although this is a storage-layout setting rather than tensor memory layout, it reflects the same core principle: when the physical layout matches the query’s spatial access unit, unnecessary decoding work falls.

## 5. Programmability, synthesis, and failure modes

Recent tile DSLs treat tile-major layout as something to be synthesized or parameterized rather than manually hard-coded. TileLang decouples **scheduling space (thread binding, layout, tensorize and pipeline)** from dataflow and exposes layout through annotations and primitives such as `T.annotate_layout`, `T.use_swizzle`, `T.Pipelined`, and `T.Parallel`. The compiler then performs layout inference for shared memory, registers, and fragments, so that tiled data movement and compute can be expressed in terms of `T.copy`, `T.gemm`, and related operators [2504.17577].

Hexcute pushes this further with a type-inference-based synthesis procedure. Its layout constraints relate source, destination, and instruction layouts; for example, for `copy(a, b)` the paper gives
\[
f \circ p^{-1} = g \circ q^{-1},
\]
and for `mma(a, b, c)` it derives more elaborate constraints tying the $M$, $N$, and $K$ dimensions of operand layouts to the instruction interface. Shared-memory layouts are solved so that
\[
m \circ f = \text{desired thread-wise memory access pattern},
\]
optionally composed with a swizzle function to remove bank conflicts [2504.16214].

The same abstraction power creates new correctness risks. A systematic study of tile-program code generation bugs curates **401 bug reports** and identifies **301 tile-program codegen bugs**. The reported manifestations are **crashes (58%)**, **silent wrong results (36%)**, and **performance bottlenecks (5%)**. The study categorizes root causes into control-flow and scheduling **(5%)**, IR construction and transformation **(16%)**, tile mapping and launch **(6%)**, memory bugs **(19%)**, type and operator bugs **(48%)**, and device-specific bugs **(4%)** [2605.19652].

The bug study also identifies a recurring misconception in practice: shape-aligned cases are not representative. The reported bugs are often triggered by shapes that are not exact multiples of tile size, by non-contiguous or padded layouts, by exotic dtypes, or by backend-specific legality constraints. The recommended detection methods are therefore **structured input generation**, **differential or metamorphic oracles**, and **canary/injection-based testing**, rather than relying on random fuzzing alone [2605.19652].

## 6. Related tile-centric organizations beyond tensor memory

The tile-major idea has close relatives in other computational domains, although the semantics are not identical. In panoramic room-layout estimation, DMH-Net projects an equirectangular panorama into six cubemap tiles and performs prediction **individually on each cubemap tile**, then assembles the final 3D room layout through post-processing and optimization. The motivation is that cubemap tiles are perspective, distortion-free images aligned with the Manhattan frame, so the per-tile decomposition is both CNN-friendly and geometrically interpretable [2207.09291].

In large-graph visualization, a **tile pyramid** is used for semantic zoom. The graph layout is spatially partitioned into levels of tiles, higher-ranked nodes remain readable at coarser zoom levels, and edges are rerouted at each level using **sleeve routing** on the dual graph of a Constrained Delaunay Triangulation. The result is a precomputed, browser-side layout whose interaction cost depends on the current viewport and tile set rather than on the entire graph [2605.17498].

In video analytics, TASM uses codec-supported tiles as spatially independent rectangular regions that can be decoded separately. It chooses **uniform and non-uniform tile layouts** based on video content and query workload, with layout changes constrained to keyframe boundaries, and uses a regret-based dynamic tuning algorithm when workload information is incomplete [2006.02958].

These systems are not all instances of the same memory layout problem, but they show that the tile-centric viewpoint generalizes. This suggests that tile-major layout is part of a broader methodological shift: replacing monolithic linear or global representations with representations whose primitive unit is the tile, because the tile is often the unit at which access granularity, parallel execution, or semantic visibility is actually determined.

Source: https://www.emergentmind.com/topics/tile-major-layout