MiniTensor: Minimal CPU Tensor Library

• MiniTensor is a minimal, open-source tensor library offering essential tensor operations and reverse-mode autodiff for CPU-based research and education.
• It uses a PyTorch-like Python API backed by a Rust engine to deliver correct and performant tensor computations via vectorization and multithreading.
• Its compact design and auditable codebase make it ideal for teaching, prototyping, and embedding deep learning components in constrained environments.

MiniTensor is an open-source tensor operations library designed around minimalism, correctness, performance, and a small installation footprint. It exposes a PyTorch-like Python API while executing performance-critical code in a Rust engine, and it targets the subset of deep learning functionality needed for CPU-based research, teaching, prototyping, and deployment in constrained environments rather than industrial-scale, GPU-heavy workloads (Sarkar, 27 Jan 2026).

1. Definition and design goals

MiniTensor is a CPU-focused tensor and automatic differentiation library whose stated aim is to keep only the essential features of a deep learning framework in a codebase and package size that are easy to understand, audit, and deploy. Its core functionality includes dense $n$-dimensional tensors, basic arithmetic and linear algebra, reverse-mode automatic differentiation, a compact set of neural network layers and loss functions, and the standard optimizers SGD, Adam, and RMSProp (Sarkar, 27 Jan 2026).

The project is defined by four explicit design goals. Minimalism refers to a small codebase and a deliberately limited operator surface. Correctness is framed in terms of mathematically standard derivatives together with unit tests and finite-difference checks. Performance is CPU-oriented and relies on Rust and LLVM, including vectorization and multithreading. Small footprint refers to a binary wheel of only a few megabytes, in contrast to the substantially larger binary distributions of mainstream frameworks (Sarkar, 27 Jan 2026).

The intended users are correspondingly specific. The library is presented as suitable for education and teaching of autodiff and deep learning, research and prototyping where gradients and standard layers are needed without a large framework ecosystem, CPU-only or constrained environments, and application embedding scenarios where a small and auditable dependency is preferred. It is explicitly not positioned as a drop-in replacement for PyTorch or TensorFlow in large-scale training or GPU-centric deployment (Sarkar, 27 Jan 2026).

2. Tensor model and supported operations

MiniTensor’s tensor model is based on dense $n$-dimensional tensors with contiguous row-major layout. A tensor consists of a typed buffer together with metadata, specifically shape and optional strides. The currently supported datatype is 32-bit floating point; other dtypes are described as a roadmap item rather than a present capability (Sarkar, 27 Jan 2026).

Broadcasting follows NumPy/PyTorch semantics: shapes are right-aligned, and singleton dimensions can be virtually expanded without materialization. The paper gives the standard example $x \in \mathbb{R}^{b \times d}$, $b \in \mathbb{R}^d$, where broadcasting implements

$(x+b)_{ij} = x_{ij} + b_j.$

The Rust engine also supports zero-copy views via strides, although the paper does not detail a full advanced indexing or slicing API (Sarkar, 27 Jan 2026).

The supported operator set is intentionally compact. It includes elementwise arithmetic and elementwise functions, reductions such as sum and mean, matrix multiplication, reshaping and views, and 2D convolution with stride and padding. For matrix multiplication the formulation given is

$Y = XW^\top,\qquad Y \in \mathbb{R}^{m \times d},$

for $X \in \mathbb{R}^{m \times k}$ and $W \in \mathbb{R}^{d \times k}$. The layer set includes dense layers, 2D convolution, ReLU, Sigmoid, Tanh, GELU, BatchNorm, and Dropout, while the loss set includes cross entropy and mean squared error (Sarkar, 27 Jan 2026).

This operator profile indicates a deliberate emphasis on the “research essentials” of tensor computation rather than on exhaustive coverage. Specialized operations not mentioned in the paper, such as advanced indexing or FFT, are not guaranteed to be present. A plausible implication is that MiniTensor’s conceptual accessibility derives partly from this restricted surface area.

3. Reverse-mode automatic differentiation

MiniTensor includes a built-in reverse-mode autodiff system based on a dynamic computation graph $\mathcal{G}$. When a tensor is marked as requiring gradients, the forward pass records a graph whose nodes contain references to parent nodes together with a local pullback, i.e., a vector–Jacobian product rule. During .backward() a seed gradient, typically $1.0$ for a scalar loss, is initialized at the loss node, and the graph is traversed in reverse topological order while cotangents are accumulated into gradient buffers (Sarkar, 27 Jan 2026).

The paper formulates the mechanism using standard reverse-mode notation. For a primitive $n$0,

$n$1

and for compositions, the chain rule is applied in reverse as a product of Jacobians. The central claim is conventional: for a scalar loss, parameter gradients can be computed with time complexity that is a small constant multiple of the forward cost (Sarkar, 27 Jan 2026).

Several local pullback rules are made explicit. For addition $n$2, the cotangent is propagated unchanged to both inputs. For the Hadamard product $n$3, the propagated cotangents are $n$4 and $n$5. For matrix multiplication,

$n$6

The paper does not fully formalize backpropagation through broadcasting and reductions, but it states forward semantics consistent with NumPy/PyTorch behavior, and standard reduction over broadcasted dimensions is therefore implied rather than explicitly derived (Sarkar, 27 Jan 2026).

Correctness is reinforced by unit tests and finite-difference checks over tensor arithmetic, broadcasting, autograd rules, and layer gradients. The paper also mentions end-to-end examples that train small models and confirm consistent loss descent, although it does not provide named benchmark datasets in the provided text (Sarkar, 27 Jan 2026).

4. Architecture and implementation

MiniTensor is organized as a three-layer architecture. The front end is a Python API with imperative, eager execution semantics modeled after PyTorch. The binding layer uses PyO3 to expose Rust functions and types as a Python module and to mediate conversions between Python objects, including NumPy arrays, and Rust buffers. The execution layer is a Rust engine that implements tensor storage, primitive operations, and reverse-mode autodiff (Sarkar, 27 Jan 2026).

The Python/Rust boundary is a central architectural feature. PyO3 is used to bind Python and Rust, with zero-copy conversions to and from NumPy when shapes and memory layouts are compatible; otherwise, a copy is required. Performance-critical kernels—elementwise loops, reductions, matrix multiplication, and convolution—reside in Rust, while Python is used primarily for model composition, optimizer loops, and high-level training logic (Sarkar, 27 Jan 2026).

Memory management is described in terms of typed buffers, shape/stride metadata, view support, and lazy gradient allocation. Views and slices share an underlying buffer through shape-plus-stride metadata. Gradient buffers are allocated lazily, so memory is not committed until a backward pass requires it. The paper emphasizes that Rust’s ownership model and PyO3’s lifetime management are used to avoid unnecessary copies without resorting to manual memory unsafety (Sarkar, 27 Jan 2026).

Performance-oriented implementation choices include row-major contiguous layout, inner loops written to be auto-vectorizable by LLVM, use of Rust core::simd where appropriate, and multithreading over independent chunks to scale across CPU cores. The paper does not specifically describe bundled GPU backends or BLAS integration; instead, it characterizes the system as CPU-centric and self-contained (Sarkar, 27 Jan 2026).

5. Footprint, performance claims, and evaluation scope

The paper’s most concrete quantitative comparison concerns installation footprint rather than runtime benchmarking. The reported wheel sizes are as follows (Sarkar, 27 Jan 2026):

Package	Wheel size
MiniTensor	2.6 MB
PyTorch	887.9 MB
TensorFlow	620.7 MB

This places MiniTensor two to three orders of magnitude below the example PyTorch and TensorFlow wheels in binary size. The paper attributes the reduction to the absence of bundled GPU backends, a minimal operator set, reliance on Rust’s standard library rather than large external native dependencies, and the omission of distributed training, JIT compilers, and other large subsystems (Sarkar, 27 Jan 2026).

By contrast, the runtime performance discussion is deliberately cautious. The paper states that PyTorch and TensorFlow “beat MiniTensor” in optimizations and performance as of the reported release. It also claims that MiniTensor can approach the speed of production frameworks on CPU tasks for many elementwise operations and reductions because of Rust and LLVM, but it does not provide detailed microbenchmark tables or plots in the provided text (Sarkar, 27 Jan 2026).

A common misconception is therefore that MiniTensor is primarily a throughput-oriented competitor to mature frameworks. The paper does not support that reading. Its quantified contribution is footprint, architectural clarity, and preservation of a basic research workflow on CPUs; its performance claims are qualitative and bounded by an explicit acknowledgment of the present optimization gap relative to mainstream systems (Sarkar, 27 Jan 2026).

6. Use cases, limitations, and relation to broader tensor research

MiniTensor is presented as useful where a full industrial framework would be disproportionate: teaching reverse-mode autodiff, building small feed-forward or convolutional models, CPU-only experimentation, and embedding differentiable components in larger Python or Rust applications. Because the codebase is small and the pullback rules are inspectable, the library is also positioned as a vehicle for understanding how gradients flow through layers, normalizations, and losses (Sarkar, 27 Jan 2026).

Its limitations are equally explicit. The current release is CPU-only, with no CUDA, ROCm, TPU, or distributed training support. The operator and layer sets are limited relative to PyTorch and TensorFlow. Optimizer logic currently lives in Python rather than fused Rust kernels, which the paper notes may incur overhead for very large models. No JIT compilation, TorchScript, ONNX export, or graph-level compiler stack is described. Only 32-bit floating point tensors are currently supported (Sarkar, 27 Jan 2026).

Future directions mentioned in the paper include GPU backends, expanded operator coverage, more datatypes, and moving Python-side optimizer loops into batched Rust kernels. The architectural separation between Python API, PyO3 bindings, and Rust kernels is described as facilitating extension: adding a new operation means implementing its forward kernel in Rust, writing its local pullback rule, and exposing it through PyO3 (Sarkar, 27 Jan 2026).

Within the broader arXiv literature, MiniTensor occupies a distinct niche. It is not a tensor-network compression pipeline of the kind used for LLM deployment, as in Minima, which mixes Tucker, TT, and TR decompositions and custom Triton/CUDA kernels for serving gains (Kozyrev et al., 2 Feb 2026). It is not a generalized task-algebra framework based on Einstein products and tensor-valued parameters, as in Multidimensional Task Learning (Ichi et al., 26 Feb 2026). Nor is it a t-product-based stable tensor network in the sense of $n$7-NNs (Newman et al., 2018), a generalized tensorial neural network framework for compression (Su et al., 2018), or a tensorized GAN architecture built from multilinear layers (Cao et al., 2017). MiniTensor instead preserves a conventional eager deep learning workflow while stripping it down to a minimal CPU-oriented tensor and autodiff core (Sarkar, 27 Jan 2026).

This distinction matters because the term “tensor” spans several research programs: tensor-network compression, tensor algebra for task formulation, multilinear neural parameterizations, and lightweight autodiff libraries. MiniTensor belongs to the last category. Its contribution is not a new tensor decomposition or learning theory; it is an implementation strategy for retaining the essential mechanics of modern tensor programming in a substantially smaller and more auditable package (Sarkar, 27 Jan 2026).