tritonBLAS: Analytical GEMM Kernel Selector

• tritonBLAS is an analytical framework that deterministically selects GEMM kernel parameters by modeling tile configurations with hardware-specific details like cache hierarchy and memory bandwidth.
• It replaces costly runtime autotuning with a closed-form, roofline-based performance model that rapidly predicts optimal tiling parameters, achieving near-optimal throughput.
• Implemented in the Triton programming environment, tritonBLAS enables reproducible and low-overhead kernel configuration for high-performance computing and machine learning workloads.

tritonBLAS is a deterministic, analytical framework for General Matrix Multiplication (GEMM) kernel parameter selection, implemented entirely within the Triton programming environment. It replaces runtime empirical autotuning with a closed-form model that incorporates architectural parameters—including cache hierarchy, memory bandwidths, register and shared memory capacities, and the topology of compute units—to rapidly select performant GPU kernel configurations. tritonBLAS achieves near-optimal throughput, with selection overhead several orders of magnitude lower than autotuned solutions, making it suitable for both high-performance computing (HPC) and ML production workloads (Swann et al., 3 Dec 2025).

1. Analytical Model for GEMM Kernel Selection

tritonBLAS models the GEMM operation $$C \leftarrow \alpha \cdot A \cdot B + \beta \cdot C$$, where $A \in \mathbb{R}^{M \times K}$, $B \in \mathbb{R}^{K \times N}$, and $C \in \mathbb{R}^{M \times N}$, as a tiling problem mapped to the GPU hardware. The framework partitions the computation across compute units (CUs) placing spatial “output” tiles with dimensions $M_b \times N_b$ and reduction tiles along $K$ of size $K_b$.

The performance model builds upon the roofline approach, using the arithmetic intensity $I$ (FLOP/byte):

$I = \frac{2 M N K}{4[M K + N K + M N]}$

where 2 FLOPs per MAC and 2 bytes per FP16 element are assumed. The maximal achievable performance is:

$P \leq \min (F_{peak}, B_{mem} \cdot I)$

Per-tile latency comprises:

• Compute latency $A \in \mathbb{R}^{M \times K}$0, derived from matrix instruction shape $A \in \mathbb{R}^{M \times K}$1 and instruction latency $A \in \mathbb{R}^{M \times K}$2; for a tile, $A \in \mathbb{R}^{M \times K}$3, with $A \in \mathbb{R}^{M \times K}$4.
• Memory latency $A \in \mathbb{R}^{M \times K}$5, modeled across L1, L2 caches, and DRAM with bandwidths $A \in \mathbb{R}^{M \times K}$6, $A \in \mathbb{R}^{M \times K}$7, $A \in \mathbb{R}^{M \times K}$8, and hit rates $A \in \mathbb{R}^{M \times K}$9 estimated from tile footprints and reuse. Per-CU load latency is assigned by the bottleneck among memory hierarchy levels, accounting for uncached and cached loads.

Pipeline overhead (prologue/epilogue) and occupancy details, such as waves ($B \in \mathbb{R}^{K \times N}$0) and active compute units, are incorporated. The total latency formula is:

$B \in \mathbb{R}^{K \times N}$1

where $B \in \mathbb{R}^{K \times N}$2 includes all compute, memory, pipeline, and store latencies, as explicated in the model.

Optimization seeks to select $B \in \mathbb{R}^{K \times N}$3 minimizing $B \in \mathbb{R}^{K \times N}$4, equivalently maximizing $B \in \mathbb{R}^{K \times N}$5.

2. Architecture-Driven Parameterization

All model inputs are obtained from microbenchmarks executed on the target GPU. Key parameters include:

• $B \in \mathbb{R}^{K \times N}$6: Number of compute units (e.g., 80 on MI300X)
• Matrix instruction shape $B \in \mathbb{R}^{K \times N}$7 and latency $B \in \mathbb{R}^{K \times N}$8
• Registers per SIMD, registers per thread (impose loop unroll limits)
• Shared memory per CU, L1/L2 cache size and bandwidth, DRAM bandwidth and latency
• Architectural constraints: e.g., $B \in \mathbb{R}^{K \times N}$9elem_bytes $C \in \mathbb{R}^{M \times N}$0elem_bytes $C \in \mathbb{R}^{M \times N}$1 smem_size

These values determine feasible tile factors and inform the performance cost function.

3. Blocking Notation and Optimization Constraints

Within this framework, tiling factors are notated as:

• $C \in \mathbb{R}^{M \times N}$2, $C \in \mathbb{R}^{M \times N}$3: spatial output tile sizes
• $C \in \mathbb{R}^{M \times N}$4: reduction axis tile size

Constraints restrict candidate tiles:

• $C \in \mathbb{R}^{M \times N}$5 must respect shared memory and register file capacities
• Shared memory per block: $C \in \mathbb{R}^{M \times N}$6elem_bytes $C \in \mathbb{R}^{M \times N}$7 SMEM_per_CU
• Approximate register demand: $C \in \mathbb{R}^{M \times N}$8

The optimization solves:

$C \in \mathbb{R}^{M \times N}$9

subject to these constraints, over the valid factor set.

4. Static Enumeration and Selection Algorithm

tritonBLAS proceeds by statically enumerating potential tile sizes, invoking the analytical model for each candidate, and selecting the configuration with minimal predicted latency. No empirical testing or JIT compilation during selection is required.

Pseudocode encapsulates this process:

$K$5

The candidate set $M_b \times N_b$0 is typically $M_b \times N_b$1–$M_b \times N_b$2 for FP16 kernels on contemporary GPUs, enabling sub-millisecond selection latency.

5. Triton-Based Implementation

The tritonBLAS workflow is embedded in Triton as a pure-Python module. The analytical selector and GEMM kernel are coupled as follows:

• Selector determines optimal ($M_b \times N_b$3)
• Kernel launch uses Triton’s grid mapping: $M_b \times N_b$4
• All tile sizes are resolved pre-launch; no calls to @triton.autotune are issued

Example function signature:

$K$6

and the user-facing routine:

$K$7

Selection and launch overhead is $M_b \times N_b$5–$M_b \times N_b$6s, independent of matrix shape.

6. Performance Characterization

Empirical evaluation demonstrates:

GEMM Shape	Tile Sizes	Predicted Latency (μs)	Measured Latency (μs)	Error (%)
512×512×512	32×32×8	120	125	+4.2
1024×512×256	64×32×8	80	83	+3.8
2048×1024×512	128×64×16	240	253	+5.4

Over 150K random shapes up to $M_b \times N_b$7K dimension, tritonBLAS attains 94.7% of the peak performance found by exhaustive autotuning, with a median performance near 97%. Selection time is $M_b \times N_b$8–$M_b \times N_b$9 orders faster than runtime autotuning (e.g., $K$080 μs vs. 12–50 s). GEMM throughput on MI300X (FP16) is within $K$13% of vendor-optimized torch.matmul(), and achieves $K$295% of the performance of autotuned cuBLAS/CUTLASS kernels across a wide arithmetic intensity range (0.5–50 FLOP/byte).

The timing model’s prediction error remains within $K$3 across representative shapes and correctly ranks candidate tile configurations.

7. Strengths and Limitations

Strengths:

• Eliminates runtime autotuning overhead, enabling rapid deployment and dynamic batching
• Tile selection is deterministic and reproducible; identical shapes yield identical parameter choices
• Portable across GPU architectures; calibration of model inputs suffices for adaptation
• Delivers near-optimal performance (≥95% of exhaustive search), even for memory-bound workloads

Limitations:

• Model abstracts away cache associativity and replacement details; not critical for GEMM’s regular access pattern
• Framework operates under a single-GPU model; multi-GPU scenarios demand separate interconnect modeling
• For extremely small GEMM problems ($K$4), kernel launch overhead dominates achievable performance; tiling decisions have marginal effect
• Model accuracy depends on precision of measured architectural parameters; recalibration is required for new hardware, but can be achieved rapidly (few μs)

A plausible implication is that tritonBLAS enables analytical optimization in dynamic and rapidly changing workload scenarios with negligible selection cost, while maintaining throughput competitive with state-of-the-art autotuned frameworks (Swann et al., 3 Dec 2025).