Simultaneous Matrix Multiplication by Tensor Contraction Aggregation

Abstract: We present an interpretation of the fast matrix multiplication algorithms of Coppersmith and Winograd as aggregations of tensor contractions. This point of view provides a direct connection to older constructions by Pan and to more recent work on bilinear algorithms for symmetric tensor contraction. Further, we provide a new simple approach for exact aggregation of tensor contractions, each one yielding an independent matrix-matrix product. These constructions and border rank adaptations thereof do not come close to improving the best known bounds on $ω$ (the matrix multiplication exponent), but provide explicit moderate-size fast matrix multiplication algorithms that achieve a better exponent than Strassen's algorithm and have potential practical value. These results imply improvements to the best known bounds on the rank and border rank of a fixed number of independent square matrix products. In particular, using $\langle m,m,m\rangle$ to denote the matrix multiplication tensor for matrices of dimension $m$, we obtain constructions that give bounds such as the border rank bound [ \underline R!\left(\langle n^2,n^2,n^2\rangle^{\oplus9}\right) \le (n+1)^6+6(n+1)^5, ] and the exact rank bound [ R!\left(\langle n^3,n^3,n^3\rangle^{\oplus14}\right)\le (n+1)^9. ] We quantify the accuracy of such constructions in finite precision, and provide border-rank constructions that reduce cost with limiting retained fractions of accuracy at least $1/2$, $1/3$, or $1/4$ relative to the classical matrix-multiplication approach under the stated error model.