Architecture Matrix: Unified Design Framework

• Architecture Matrix is a framework that represents model structures, computation flows, and hardware configurations using matrix patterns like sparsity, block reuse, and aspect-ratio indexing.
• It unifies diverse concepts by expressing neural operators, feature hierarchies, and matrix-native accelerators under a common, structured design paradigm.
• This approach bridges the gap between algorithm design and hardware mapping, enabling advancements in efficiency through sparse, low-latency, and hierarchical matrix architectures.

Searching arXiv for papers relevant to matrix-based architectures and architecture-as-matrix formulations. “Architecture matrix” denotes a family of representations in which model structure, computation flow, or hardware organization is expressed through matrices, matrix tiles, or matrix-indexed grids rather than only through conventional layer taxonomies. Across recent literature, the term does not refer to a single canonical formalism; instead, it appears in several technically distinct senses: as a matrix-parameterized neural operator, as a matrix of feature layers for detection, as a sparse-matrix unification of CNN/RNN/Transformer operators, and as a matrix-centric hardware substrate for analog, digital, sparse, or compute-in-memory acceleration. In all cases, the common theme is that architectural behavior is specified by structured matrix organization—through sparsity pattern, block reuse, aspect-ratio indexing, or reconfigurable matrix datapaths—so that representational design and execution efficiency become tightly coupled (Zhu, 11 May 2025, Rashwan et al., 2019, Pan et al., 3 Jan 2025).

1. Conceptual scope and definitions

An architecture matrix can be understood as any architectural construction in which the principal design object is a matrix-structured arrangement of parameters, operators, feature layers, or hardware compute units. In one line of work, the architecture itself is recast as a sparse linear operator: convolution becomes an upper-triangular or banded matrix, recurrence becomes a lower-triangular block matrix, and self-attention becomes a lifted third-order tensor or matrix factorization (Zhu, 11 May 2025). In another line, the architecture is literally a matrix of feature maps, as in Matrix Nets, where layers are indexed by two spatial downsampling axes so that scale and aspect ratio are jointly encoded (Rashwan et al., 2019).

A related but distinct use appears in operator design. NeoNeXt defines NeoCell as a patch-wise matrix-multiplication operator in which each channel applies a left matrix multiplication and a right matrix multiplication to each patch, yielding the channelwise form

$Y_{c,i,j} = L_c\, X_{c,i,j}\, R_c,$

with an equivalent block-diagonal realization $Y_c = A_c X_c B_c$ (Korviakov et al., 2024). This suggests that “architecture matrix” can also denote a design principle in which local computation is built from repeated structured matrix transforms rather than sliding kernels or tokenwise attention.

In hardware-oriented work, the phrase extends further to matrix-native accelerators. GRAMC is presented as a general-purpose, reconfigurable analog matrix computing architecture based on a $128 \times 128$ 1T1R RRAM array and reconfigurable operational amplifiers, while CUTEv2 proposes a unified and configurable CPU matrix extension architecture decoupled from the CPU pipeline (Pan et al., 3 Jan 2025, Ye et al., 13 Apr 2026). D-Legion, CAMP, GEM3D-CIM, SpArch, and several systolic-array designs similarly treat matrix multiplication, sparse matrix multiplication, or general matrix operations as the architectural core (Abdelmaksoud et al., 5 Feb 2026, Nojehdeh et al., 10 Apr 2025, Chakraborty et al., 15 Apr 2026, Zhang et al., 2020).

A plausible implication is that the phrase has become an umbrella for a broader shift: architecture design is increasingly being formulated in algebraic or matrix-structured terms so that modeling choices and hardware mapping can be reasoned about within the same representation.

2. Matrix representations of neural operators

The most explicit unification appears in the matrix-order framework of “Matrix Is All You Need,” which argues that major neural network layers can be rewritten as sparse matrix multiplications once the input is flattened or lifted into an appropriate matrix or tensor space (Zhu, 11 May 2025). For images, an $m\times n$ image is flattened into $X\in\mathbb{R}^{L}$ with $L=mn$, and a convolution with an $r\times r$ kernel is represented by a sparse upper-triangular or banded matrix $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$, with flattened kernel offsets

$N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$

and

$$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$

Each row places learnable weights at columns $Y_c = A_c X_c B_c$0 for $Y_c = A_c X_c B_c$1, yielding

$Y_c = A_c X_c B_c$2

The claimed algebraic isomorphism covers the linear part of convolution under flattening, banded sparsity, and fixed padding conventions (Zhu, 11 May 2025).

For recurrent models, the same paper writes the unrolled linearized recurrence as a lower-triangular matrix: $Y_c = A_c X_c B_c$3 with

$Y_c = A_c X_c B_c$4

so that

$Y_c = A_c X_c B_c$5

Causality appears as lower-triangular structure, and the paper states that a unidirectional linearized RNN can be written as a strictly lower-triangular and block-banded matrix (Zhu, 11 May 2025).

For self-attention, the same framework emphasizes a higher-order structure. Starting from

$Y_c = A_c X_c B_c$6

the linearized form of single-head attention is written as a third-order tensor interaction, and in lifted coordinates $Y_c = A_c X_c B_c$7, attention becomes a matrix acting on pairwise features: $Y_c = A_c X_c B_c$8 The paper repeatedly states that these constructions are algebraically isomorphic to the corresponding standard linear operators, omitting nonlinearities such as ReLU or softmax from the equivalence claim (Zhu, 11 May 2025).

This matrix-order formulation reframes architecture choice as sparse pattern selection. The paper explicitly states that one can choose sparsity pattern, block structure, bandwidth, factorization rank, and lifting space rather than introducing a bespoke operator for each modality (Zhu, 11 May 2025). This suggests an “architecture matrix” in the strongest possible sense: the architecture is the matrix pattern.

3. Matrix-structured feature hierarchies and local operators

Matrix Nets, or xNets, introduce a different sense of architecture matrix: a two-dimensional matrix of feature layers for object detection (Rashwan et al., 2019). Instead of a one-dimensional feature pyramid arranged only by scale, xNets define layers $Y_c = A_c X_c B_c$9 in a grid where each entry has width downsampling $128 \times 128$0 and height downsampling $128 \times 128$1 relative to $128 \times 128$2. Moving right halves width resolution, and moving down halves height resolution. Diagonal layers are square-like, while off-diagonal layers are rectangular and specifically model non-square aspect ratios (Rashwan et al., 2019).

The assignment rule is likewise matrix-indexed. If $128 \times 128$3 covers

$128 \times 128$4

then $128 \times 128$5 covers

$128 \times 128$6

The paper further relaxes boundaries by multiplying the lower bound by $128 \times 128$7 and the upper bound by $128 \times 128$8, so that small perturbations do not cause abrupt layer reassignment during training (Rashwan et al., 2019). In this construction, the architecture matrix jointly encodes scale and aspect ratio, and the paper argues that this makes the regression problem easier because the dynamic range of widths and heights is smaller within any given layer.

NeoNeXt offers another matrix-centric operator view. NeoCell acts on patches of size $128 \times 128$9 by applying left and right matrices $m\times n$0 and $m\times n$1 per channel. The resulting output patch is $m\times n$2, so the same operator can preserve resolution, down-sample, or up-sample depending on whether $m\times n$3 equal, undercut, or exceed $m\times n$4 (Korviakov et al., 2024). The paper also gives a block-diagonal formulation: $m\times n$5 where $m\times n$6 and $m\times n$7 replicate $m\times n$8 and $m\times n$9 along a block diagonal across patches. This turns the local operator into a structured matrix multiplication rather than an im2col-style reformulation (Korviakov et al., 2024).

The same paper explicitly compares NeoCell to depthwise convolution. For a $X\in\mathbb{R}^{L}$0 depthwise convolution, the multiplication count is

$X\in\mathbb{R}^{L}$1

whereas NeoCell with left and right square matrices of size $X\in\mathbb{R}^{L}$2 has

$X\in\mathbb{R}^{L}$3

The paper states

$X\in\mathbb{R}^{L}$4

This is a direct example of architectural specification by matrix pattern rather than stencil semantics (Korviakov et al., 2024).

A plausible implication is that these works separate two meanings of architecture matrix. In xNets, the matrix is a feature-space indexing device. In NeoNeXt and the matrix-order framework, the matrix is the operator itself.

4. Hardware-aware matrix architectures for dense computation

Several works use “architecture matrix” in a hardware sense, where the principal architectural object is a programmable or specialized matrix engine. GRAMC is an example of a general-purpose, reconfigurable analog matrix computing architecture built from a $X\in\mathbb{R}^{L}$5 1T1R RRAM array, reconfigurable OPAs, DACs, ADCs, drivers, buffers, and a register array storing configuration messages (Pan et al., 3 Jan 2025). Its central claim is that prior AMC designs fixed the connection topology to a single function such as MVM, INV, PINV, or EGV, whereas GRAMC reconfigures the connections between the memory array and amplifier circuits so that the same macro can implement multiple matrix primitives (Pan et al., 3 Jan 2025).

The macro-level reconfiguration is controlled by transmission gates, and the paper states that configuration messages are stored in the register array in advance and control whether those gates are on or off. At system level, GRAMC uses a digital control module, a group of 16 AMC macros, and hybrid analog-digital data paths. The on-chip write-verify loop programs conductances in the range $X\in\mathbb{R}^{L}$6, corresponding to 16 conductance states, i.e. 4-bit quantization, with SET and RESET pulse width of $X\in\mathbb{R}^{L}$7 (Pan et al., 3 Jan 2025). In neural-network mapping, one array stores the most significant 4 bits and another the least significant 4 bits, giving an effective 8-bit weight representation; the paper reports $X\in\mathbb{R}^{L}$8 recognition accuracy with 4-bit weights and $X\in\mathbb{R}^{L}$9 with 8-bit weights on LeNet-5 for MNIST, close to the $L=mn$0 float32 result (Pan et al., 3 Jan 2025).

CUTEv2 presents a different architectural response: a configurable CPU matrix extension decoupled from the CPU pipeline (Ye et al., 13 Apr 2026). The paper’s case-study configuration uses a $L=mn$1 PE array, PE reduce width 512 bits, scratchpad-resident tile sizes $L=mn$2, $L=mn$3, $L=mn$4 bytes, $L=mn$5 data bandwidth, and 4 TOPS at 8-bit. Its throughput for an $L=mn$6-bit format is given as

$L=mn$7

The paper emphasizes asynchronous matrix multiplication with explicit synchronization so that matrix work can overlap with vector epilogues (Ye et al., 13 Apr 2026). Matrix unit utilization under GEMM workloads exceeds $L=mn$8 across four open-source CPU RTL platforms, and the paper attributes more than $L=mn$9 of the gains in fused workloads to overlapped matrix-vector execution (Ye et al., 13 Apr 2026).

CAMP, by contrast, augments vector architectures and SIMD units with a Cartesian Accumulative Matrix Pipeline for quantized GEMM (Nojehdeh et al., 10 Apr 2025). It introduces an outer-product-oriented camp(VR0, VR1, VR2, mode) instruction and a hybrid multiplier constructed recursively from 4-bit multipliers. In 8-bit mode, CAMP processes $r\times r$0 by $r\times r$1 tiles to produce a $r\times r$2 result; in 4-bit mode, it uses $r\times r$3 by $r\times r$4 tiles. The paper reports up to $r\times r$5 performance improvement on ARM A64FX and up to $r\times r$6 on a RISC-V edge SoC, with area overhead of $r\times r$7 and $r\times r$8, respectively (Nojehdeh et al., 10 Apr 2025).

These works share a common architectural principle: the matrix engine is not merely an implementation detail of a higher-level model but the first-class architectural abstraction around which control, precision, storage, and scheduling are organized.

5. Sparse and low-latency matrix architectures

A central subset of architecture-matrix research concerns sparse or low-latency execution. VVMA—Vector-Vector-Matrix Architecture—proposes a hardware-aware neural network compression and execution scheme for TPU-like accelerators (Khoury et al., 2020). Its key idea is to replace a standard weight matrix $r\times r$9 by a structured composition of a shared block $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$0 and learned diagonal matrices $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$1, so that

$W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$2

with the general form being a tiling of copies of $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$3 multiplied by diagonal matrices. Since each diagonal $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$4 can be represented by a vector $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$5, the computation becomes

$W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$6

followed by multiplication with the shared matrix $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$7 (Khoury et al., 2020). The point is not merely compression: the vector-vector stage is pipelined and overlapped with matrix loading, so the architecture reduces the load-heavy term that dominates batch-size-1 inference on TPU-style hardware. The paper reports about $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$8 inference-latency reduction for NMT models, with sequence-to-sequence and Transformer examples showing substantial reductions in clocks, parameters, and FLOPs (Khoury et al., 2020).

Sparse matrix multiplication motivates a different class of architecture matrices. The InCRS format and synchronized systolic mesh in “Sparse Matrix to Matrix Multiplication: A Representation and Architecture for Acceleration” address the problem that in $W_{\mathrm{conv}}\in\mathbb{R}^{P\times L}$9, the first matrix is naturally consumed in row order, but the second must often be accessed in column order (Golnari et al., 2019). InCRS divides rows into sections of size $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$0 and blocks of size $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$1, using a compact counter-vector per section. The average random-access cost falls from approximately

$N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$2

for CRS to

$N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$3

with reduction factor

$N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$4

Using $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$5, $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$6, 64-bit counter-vectors, 16 bits for nonzeros before the section, and $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$7 bits for block counts, the paper reports $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$8 speedup in memory access or SpMM access behavior and $N_0 = \{\, i\,n + j \mid 0 \le i,j < r \,\},$9 speedup over FPIC for the proposed architecture (Golnari et al., 2019).

SpArch addresses generalized sparse matrix-matrix multiplication by jointly optimizing input and output locality (Zhang et al., 2020). It uses a streaming-based merger, matrix condensing, a Huffman tree scheduler, and a row prefetcher. Matrix condensing reduces the number of partial matrices by three orders of magnitude and lowers DRAM access by $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$0; Huffman scheduling reduces DRAM access by another $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$1; row prefetching further reduces it by $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$2, and overall total DRAM access is reduced by $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$3 over previous state of the art (Zhang et al., 2020). The architecture achieves average speedups of $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$4 over OuterSPACE, $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$5 over MKL, $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$6 over cuSPARSE, $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$7 over CUSP, and $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$8 over ARM Armadillo, with corresponding energy savings of $$P = \biggl\lfloor \frac{L - \max N_0}{s} \biggr\rfloor + 1.$$9, $Y_c = A_c X_c B_c$00, $Y_c = A_c X_c B_c$01, $Y_c = A_c X_c B_c$02, and $Y_c = A_c X_c B_c$03 (Zhang et al., 2020).

These sparse and low-latency systems make explicit that a matrix architecture is often defined less by arithmetic peak than by how matrix structure interacts with memory movement, tile reuse, and scheduling policy.

6. Hierarchical and three-dimensional matrix hardware

Another branch of the literature treats the architecture matrix as a hierarchy of tiled or stacked matrix processors. D-Legion is a scalable many-core architecture for accelerating matrix multiplication in quantized LLMs (Abdelmaksoud et al., 5 Feb 2026). Its hierarchy is explicit: $Y_c = A_c X_c B_c$04 Legions, each with $Y_c = A_c X_c B_c$05 adaptive-precision systolic array cores, each core a $Y_c = A_c X_c B_c$06 array, and each PE containing 16 2-bit multipliers. The baseline design uses $Y_c = A_c X_c B_c$07 Legions, $Y_c = A_c X_c B_c$08 cores per Legion, and $Y_c = A_c X_c B_c$09 cores, for a total of $Y_c = A_c X_c B_c$10 PEs. The tile decomposition is

$Y_c = A_c X_c B_c$11

with $Y_c = A_c X_c B_c$12 for $Y_c = A_c X_c B_c$13 and $Y_c = A_c X_c B_c$14 for $Y_c = A_c X_c B_c$15 (Abdelmaksoud et al., 5 Feb 2026). The design exploits block structured sparsity via a zero-tile book, reduces psum memory traffic with four accumulators per Legion, and uses multicast over a flexible NoC. The paper reports up to $Y_c = A_c X_c B_c$16 lower latency, up to $Y_c = A_c X_c B_c$17 higher memory savings, and up to $Y_c = A_c X_c B_c$18 higher psum memory savings compared to state-of-the-art work; the baseline reaches 135.68 TOPS at 1 GHz, and a 32-Legion version attains up to $Y_c = A_c X_c B_c$19 lower total latency and $Y_c = A_c X_c B_c$20 higher total throughput than TPUv4i (Abdelmaksoud et al., 5 Feb 2026).

The 3D systolic-array work for Intel Stratix 10 FPGAs provides a spatially stacked matrix architecture for dense floating-point multiplication (Gorlani et al., 2021). A classical $Y_c = A_c X_c B_c$21 2D systolic grid has throughput

$Y_c = A_c X_c B_c$22

whereas the proposed 3D array adds a third dimension $Y_c = A_c X_c B_c$23 with dot-product units of size $Y_c = A_c X_c B_c$24, yielding total DSP count

$Y_c = A_c X_c B_c$25

and peak throughput

$Y_c = A_c X_c B_c$26

Implemented through HLS and the Intel FPGA SDK for OpenCL, the design uses up to $Y_c = A_c X_c B_c$27 of available DSPs on a BittWare 520N Stratix 10 GX2800, reaches 410 MHz in the best-performing configuration, and reports peak throughput of 3673 GFLOPS with sustained performance above 3 TFLOPS (Gorlani et al., 2021).

GEM3D-CIM extends the idea into 3D integrated compute-in-memory (Chakraborty et al., 15 Apr 2026). It stacks an SRAM layer and an eDRAM layer using monolithic 3D integration, partitions functionality into T-SRAM, T-eDRAM, MA-SRAM, and MA-eDRAM subarrays, and supports general matrix operations directly within memory with 4-bit precision. The transpose operation of an $Y_c = A_c X_c B_c$28 matrix completes in

$Y_c = A_c X_c B_c$29

compared to about $Y_c = A_c X_c B_c$30 cycles conventionally, and for a $Y_c = A_c X_c B_c$31 matrix with 4-bit words the paper reports 264 ns latency, 320.55 nJ energy, 15.51 GOPS throughput, and 12.77 GOPS/W energy efficiency (Chakraborty et al., 15 Apr 2026). For $Y_c = A_c X_c B_c$32 addition and multiplication, the paper reports 27.86 GOPS and 13.93 GOPS, with 432.25 GOPS/W and 436.61 GOPS/W, respectively (Chakraborty et al., 15 Apr 2026).

A plausible implication is that three-dimensionality in these works is not merely packaging. It is an architectural device for redistributing matrix dataflow across levels—cores, layers, or memory tiers—to improve locality, bandwidth, or functional generality.

7. Significance, limits, and recurring controversies

Across these works, several recurrent claims define the significance of architecture matrices. First, they often reduce architecture design to structured matrix selection. “Matrix Is All You Need” states this explicitly in the language of sparsity pattern, block structure, bandwidth, factorization rank, and lifting space (Zhu, 11 May 2025). Second, they frequently align model structure with hardware behavior, as in VVMA’s TPU-style load reuse, CAMP’s outer-product vector instruction, CUTEv2’s asynchronous overlap, or D-Legion’s multicast and psum hierarchy (Khoury et al., 2020, Nojehdeh et al., 10 Apr 2025, Ye et al., 13 Apr 2026, Abdelmaksoud et al., 5 Feb 2026).

Third, they blur the boundary between algorithm and implementation. GRAMC is simultaneously a matrix solver and a neural-network inference engine; GEM3D-CIM supports transpose, addition, multiplication, and conventional dot-product CIM in the same memory substrate; xNets use a matrix of layers as a representational prior rather than only as a computational convenience (Pan et al., 3 Jan 2025, Chakraborty et al., 15 Apr 2026, Rashwan et al., 2019). This suggests that the architecture matrix functions as a co-design language linking representation, scheduling, and hardware topology.

Several limitations and misconceptions recur. One common misconception is that matrix reformulation automatically captures the full behavior of the original model. The matrix-order framework explicitly limits its isomorphism claims to the linear component of CNN, RNN, and attention operators, omitting nonlinearities and noting that repeated RNN nonlinearities are not fully captured by a single linear mapping (Zhu, 11 May 2025). Another misconception is that parameter or FLOP reduction necessarily implies latency reduction. VVMA explicitly contrasts itself with pruning, quantization, low-rank factorization, and sparse attention on the grounds that those methods do not necessarily reduce latency on a specific accelerator (Khoury et al., 2020).

A further controversy concerns precision and approximation. Analog matrix engines such as GRAMC and CIM systems such as GEM3D-CIM produce approximate results influenced by quantization error, intrinsic analog noise, and mixed-signal conversion. GRAMC reports relative errors around $Y_c = A_c X_c B_c$33 for matrix-solver demonstrations and frames outputs as seed solutions for larger algorithms, rather than exact arithmetic (Pan et al., 3 Jan 2025). Similarly, general-purpose matrix support in CIM requires additional circuitry and calibration, even when compatibility with dot-product mode is preserved (Chakraborty et al., 15 Apr 2026).

Finally, architecture matrices do not imply a uniform research agenda. The term spans at least four distinct objects: sparse matrix operators, matrices of feature layers, programmable matrix hardware, and matrix-structured local transforms. What unifies them is not a shared implementation, but a shared methodological move: the architecture is specified through matrix organization, and its performance or expressivity is analyzed through that structure.