---
title: Uniform-Stride Batched GEMM
url: https://www.emergentmind.com/topics/uniform-stride-batched-gemm-90a640db-d546-4980-bd6b-02d24227671f
type: topic
---

# Uniform-Stride Batched GEMM

A uniform-stride batched GEMM (General Matrix-Matrix Multiplication) computes a batch of independent small matrix multiplications using a memory layout and interface that exploits a regular (constant) stride between matrices. This approach delivers superior performance, reduces overhead compared to pointer-to-pointer batching, and is particularly effective for workloads with many small GEMMs (dimensions typically under 16 or 32), as encountered in modern scientific simulation, deep learning primitives, and high-throughput tensor contractions on both NVIDIA GPUs and vector CPUs [1304.7053, 1606.05696, 2501.06175].

## 1. Interface Specification and Semantics

Uniform-stride batched GEMM exposes an API where each batch of the matrix operands (A, B, C) is arranged in contiguous flat buffers with fixed strides. Instead of arrays of pointers to matrix slices, a single base pointer and stride for each operand suffices. 

For example, on a CUDA-enabled GPU (cuBLAS), the prototypical interface is:
```c
cublasStatus_t cublasSgemmStridedBatched(
    cublasHandle_t handle,
    cublasOperation_t transA, cublasOperation_t transB,
    int m, int n, int k,
    const float *alpha,
    const float *A0, int lda, long long strideA,
    const float *B0, int ldb, long long strideB,
    const float *beta,
    float *C0, int ldc, long long strideC,
    int batchCount );
```
where `strideA`, `strideB`, and `strideC` denote the element offsets between consecutive A, B, C matrices in memory [1606.05696]. TGEMM_multi_uniform, as realized by Jhurani & Mullowney [1304.7053], utilizes a similar interface where, for each matrix $p$, the base pointer and stride manage selection: $A_p = A_0 + p \cdot \text{stride}_A$, etc. This enables the kernel to process hundreds of thousands of matrices in a single, tightly-packed launch, reducing kernel-launch overhead and improving coalesced memory access.

The semantics are:
$$
C^p \leftarrow \alpha \ \mathrm{op}(A

Source: https://www.emergentmind.com/topics/uniform-stride-batched-gemm-90a640db-d546-4980-bd6b-02d24227671f