Beyond Bilinear Complexity: What Works and What Breaks with Many Modes?

Abstract: The complexity of bilinear maps (equivalently, of $3$-mode tensors) has been studied extensively, most notably in the context of matrix multiplication. While circuit complexity and tensor rank coincide asymptotically for $3$-mode tensors, this correspondence breaks down for $d \geq 4$ modes. As a result, the complexity of $d$-mode tensors for larger fixed $d$ remains poorly understood, despite its relevance, e.g., in fine-grained complexity. Our paper explores this intermediate regime. First, we give a "graph-theoretic" proof of Strassen's $2ω/3$ bound on the asymptotic rank exponent of $3$-mode tensors. Our proof directly generalizes to an upper bound of $(d-1)ω/3$ for $d$-mode tensors. Using refined techniques available only for $d\geq 4$ modes, we improve this bound beyond the current state of the art for $ω$. We also obtain a bound of $d/2+1$ on the asymptotic exponent of circuit complexity of generic $d$-mode tensors and optimized bounds for $d \in {4,5}$. To the best of our knowledge, asymptotic circuit complexity (rather than rank) of tensors has not been studied before. To obtain a robust theory, we first ask whether low complexity of $T$ and $U$ imply low complexity of their Kronecker product $T \otimes U$. While this crucially holds for rank (and thus for circuit complexity in $3$ modes), we show that assumptions from fine-grained complexity rule out such a submultiplicativity for the circuit complexity of tensors with many modes. In particular, assuming the Hyperclique Conjecture, this failure occurs already for $d=8$ modes. Nevertheless, we can salvage a restricted notion of submultiplicativity. From a technical perspective, our proofs heavily make use of the graph tensors $T_H$, as employed by Christandl and Zuiddam ({\em Comput.~Complexity}~28~(2019)~27--56) and [...]

• The paper develops graph-tensor methods that generalize Strassen’s bound, proving every d-mode tensor has asymptotic rank exponent at most 0.772318(d−1), while flattenings give a lower bound of ⌊d/2⌋.
• The paper shows that rank and circuit complexity diverge for d≥4 and provides generic circuit-exponent bounds of d/2+1, including improved values of 2.2967 for four modes and 2.8774 for five modes.
• The paper gives conditional evidence that circuit complexity is not submultiplicative under Kronecker products, linking potential failures to VP≠VNP, permanent lower bounds, and algebraic hyperclique hardness.

Motivation and setting

For bilinear maps—equivalently, 3-mode tensors—asymptotic tensor rank and asymptotic arithmetic circuit complexity coincide, a fact underpinned by Yates's algorithm and its converse for $d=3$. This equivalence is what makes Strassen's bound $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$ on the asymptotic rank of any 3-tensor simultaneously a statement about circuit size and a source of faster algorithms for convolution-type problems. The paper "Beyond Bilinear Complexity: What Works and What Breaks with Many Modes?" (2602.11975) investigates the regime $d \ge 4$, where this correspondence collapses. The canonical witness is the product-of-inner-products (PIP) tensor $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$: it has circuits of size $O(n)$ but rank exactly $n^2$, and generalizing to $2d$ modes pushes the asymptotic rank exponent to $d$ while the circuit exponent stays at $1$. Thus, for $d \ge 4$, rank and circuit complexity are functionally independent even at the level of $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$0-formulas.

The paper makes three contributions: (i) a graph-theoretic proof of Strassen's $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$1 bound that extends to $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$2 for all $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$3 and improves beyond it using refined clique-tensor exponents; (ii) the first systematic study of asymptotic circuit complexity of tensors, including strong conditional evidence that circuit complexity fails to be submultiplicative under Kronecker products once $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$4; and (iii) new upper bounds on the asymptotic circuit exponent of generic $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$5-mode tensors, namely $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$6 in general and optimized values for $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$7.

Graph tensors as the technical bridge

The unifying device is the graph tensor $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$8 associated with an undirected multigraph $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$9: each vertex $d \ge 4$0 carries a mode indexed by assignments $d \ge 4$1 of values in $d \ge 4$2 to the incident edges, and

$d \ge 4$3

These objects admit two complementary readings. In quantum information terms, $d \ge 4$4 is the pure state of $d \ge 4$5 parties where each edge distributes a Bell pair; in counting-complexity terms, it captures every Holant problem definable on the fixed graph $d \ge 4$6, with freely varying vertex signatures supplied as inputs to the modes. The conceptually crucial observation—which the authors note was implicit but never stated in Christandl–Zuiddam's work—is that Kronecker products of graph tensors correspond to graph sums:

$d \ge 4$7

and this extends to fractional graphs with rational edge weights via the length rule $d \ge 4$8. Combined with the fact that any generic $d \ge 4$9-tensor restricts to the star-graph tensor $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$0, this converts questions about worst-case $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$1-tensors into combinatorial questions about decomposing graphs such as $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$2 into structured pieces whose tensor exponents are known or bounded by tools from algorithmic graph theory (treewidth, line-graph treewidth, fractional triangle coverings).

Asymptotic rank: extending and improving Strassen

The proof of the generalized Strassen bound rests on the identity $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$3: overlaying the $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$4 stars centered at each vertex yields the doubled complete graph. Since $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$5, one obtains $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$6 where $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$7 is the exponent per edge of the clique tensor. For $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$8 this recovers Strassen's $P_n = (\sum_j x_j y_j)(\sum_j w_j z_j)$9; for $O(n)$0 it improves on it because $O(n)$1 for $O(n)$2 and better bounds on $O(n)$3 are available.

The paper's second contribution here is a sharpened analysis of Coppersmith–Winograd tensors in the $O(n)$4-mode setting. Whereas Christandl–Vrana–Zuiddam analyzed only the "small" CW tensor, the authors incorporate the "big" CW tensor (border rank $O(n)$5), isolate a fixed-marginal variant of the laser-method bound, and verify via entropy inequalities that the binding constraint is the rank-one relation class. The result is $O(n)$6, improving the previous $O(n)$7, and hence:

Theorem. Every $O(n)$8-mode tensor ($O(n)$9) has asymptotic rank exponent at most $n^2$0.

The PIP tensor gives a matching-flattening lower bound of $n^2$1 on this exponent, and $n^2$2 follows from the $n^2$3-matching tensor for all $n^2$4. Notably, the paper leaves open whether $n^2$5 for some $n^2$6; if so, the bound would beat even the hypothetical $n^2$7 with $n^2$8. The known lower bounds are consistent with $n^2$9, which would yield $2d$0.

Submultiplicativity breaks down

For rank, submultiplicativity under Kronecker products holds unconditionally; the question is whether unrestricted circuit complexity inherits it. The answer is conditionally negative, with the conditional nature being unavoidable since an unconditional separation would imply major circuit lower bounds. The mechanism is a reduction chain built from grids: the permanent tensor $2d$1 projects onto $2d$2 for the $2d$3 grid (via a Holant-style construction in which horizontal and vertical paths flip from state 0 to state 1 exactly once, with coinciding flip vertices encoding permutation matrices), while every grid decomposes as a sum of four matchings, i.e., $2d$4, and matching tensors have circuits of size $2d$5 per mode dimension.

Consequences include:

• Against VP ≠ VNP: explicit tensor families $2d$6 such that polynomial-size submultiplicativity $2d$7 for all $2d$8 would collapse VP = VNP.
• Against fast permanents: if $2d$9 is submultiplicative on $d$0-mode tensors, then permanents have circuits of size $d$1. For instance, ruling out $d$2-size permanent circuits already falsifies submultiplicativity at $d$3.
• Against hyperclique hardness: assuming the non-uniform algebraic $d$4-hyperclique conjecture, submultiplicativity fails for every even $d$5. Here the hyperclique tensor $d$6 projects from the graph tensor of the bipartite incidence graph $d$7 (for $d$8, the graph $d$9 minus a perfect matching), which itself decomposes into few perfect matchings. The authors remark that passing to the incidence graph rather than the hypergraph tensor is essential: direct decompositions of the 4-mode hyperclique tensor would still incur $1$0 cost because of mode dimensions, whereas the incidence formulation trades degree for variable count—a point relevant when "Kroneckering up."

A sharper unconditional observation frames the stakes: submultiplicativity on merely 4-mode tensors would imply $1$1, since $1$2 splits into two matchings and $1$3 (the matrix multiplication tensor) projects from $1$4.

The paper also salvages a restricted form: given a low-rank decomposition of $1$5 and a small circuit for $1$6, one can build a small circuit for $1$7, yielding $1$8 asymptotically. This mixed rank-circuit bound is the workhorse behind the improved upper bounds below.

Upper bounds on asymptotic circuit complexity

Two techniques give upper bounds on the circuit exponent $1$9 of generic $d \ge 4$0-tensors. First, a dynamic-programming argument along tree decompositions of the line graph shows $d \ge 4$1, where $d \ge 4$2 is the treewidth of the line graph of $d \ge 4$3. Applying Harvey–Wood's exact formula for the line-treewidth of complete graphs to the reduction $d \ge 4$4 yields:

$d \ge 4$5

This sits just above the flattening lower bound of $d \ge 4$6 from the PIP tensor, and the gap is essentially forced: proving an unbounded-in-$d \ge 4$7 lower bound on $d \ge 4$8 would constitute a strong circuit lower bound. The authors note that current techniques could only rule out exponents $d \ge 4$9, e.g., via subgraph-isomorphism hardness for bounded-degree graphs.

Second, for small fixed $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$00, the treewidth bound is not tight—for $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$01 it gives $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$02, worse than the rank bound $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$03. Combining the mixed rank-circuit theorem with computer-aided conic decompositions of $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$04 and the three-star sum on five vertices into fractional triangles (handled by rank, using Le Gall–Urrutia's rectangular exponents $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$05, $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$06), leftover edges (rank $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$07), and low-line-treewidth remainders gives:

• $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$08 for every 4-mode tensor;
• $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$09 for every 5-mode tensor.

Both improve on the corresponding rank bounds ($$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$10 and $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$11). The computational search did not scale beyond $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$12, and the authors present these as demonstrations of viability rather than exhaustive optimization; they also observe that at $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$13 the improved $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$14 constant does not yet help, though it may at larger $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$15.

Limitations and open questions

Several caveats bear directly on the results. All negative results on submultiplicativity are conditional—on VP ≠ VNP, exponential-size permanent circuits, or the non-uniform algebraic hyperclique conjecture—and the paper is explicit that unconditional statements would imply strong circuit lower bounds. The hyperclique-based result reaches only $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$16, leaving open whether submultiplicativity fails at $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$17 under plausible assumptions. On the upper-bound side, whether $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$18 for some $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$19 remains open, as does the existence of the limit defining $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$20 (only the limsup is known to exist). The computational decompositions were obtained only for $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$21, and the paper does not claim optimality of the resulting constants. Finally, the lower bounds on $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$22 derivable from existing subgraph-isomorphism and set-cover machinery are loose enough that the authors decline to formalize them.

Conclusion

The paper establishes graph tensors as a systematic bridge between algorithmic graph theory and the complexity of fixed-order tensors, yielding a generalized Strassen bound of $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$23 on asymptotic rank, the first upper bounds on asymptotic circuit complexity of generic many-mode tensors ($$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$24, refined to $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$25 and $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$26 for $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$27), and conditional evidence—via permanents and hypercliques—that circuit complexity, unlike rank, is not submultiplicative under Kronecker products once the number of modes exceeds three. The central open tension is quantitative: closing the gap between the $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$28 flattening lower bound and the $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$29 rank upper bound, and determining the smallest $$\underline{\mathrm R}(g^{\otimes k}) \le n^{2\omega k/3}$$30 at which submultiplicativity provably fails.