Depth-2 Linear Circuits

• Depth-2 linear circuits are a class of matrix transforms that factorize an N×N matrix as UVᵀ, using one intermediate layer of sparse linear combinations.
• They decompose matrices into sums of rank-1 terms with complexity measured by nonzero counts and degree, combining efficiency with structured sparsity.
• Advanced techniques such as rebalancing, asymptotic spectral duality, and optimized Kronecker-power constructions significantly improve upon the classical O(N^(1.5)) benchmark.

Depth-2 linear circuits compute a linear transform $Mv$ using one layer of intermediate linear combinations between the input and output layers. For an $N\times N$ matrix $M$, this is equivalently a factorization $M=UV^T$ with $U,V\in\mathbb{F}^{N\times G}$, or a decomposition $M=\sum_{i=1}^G U_iV_i^T$ into rank-1 terms. The primary complexity measure is size, usually $\nnz(U)+\nnz(V)$, and refined analyses also track degree, i.e. the maximal row or column degree on the two outer layers. Recent work has concentrated on Kronecker-power matrices such as Walsh–Hadamard and disjointness, where the classical $O(N^{1.5})$ construction has been improved through increasingly precise rebalancing frameworks and asymptotic characterizations (Alman et al., 2022, Alman et al., 17 Sep 2025).

1. Formal model and complexity measures

A depth-2 linear circuit for an $N\times N$ matrix $M$ can be viewed as a factorization

$N\times N$0

for $N\times N$1 matrices $N\times N$2, where $N\times N$3 is the number of middle-layer gates. In this representation, the circuit computes $N\times N$4 by first forming the $N\times N$5 intermediate linear combinations $N\times N$6, then applying $N\times N$7 (Alman et al., 17 Sep 2025). The same model is equivalently expressed as

$N\times N$8

with each summand a rank-1 rectangle (Alman et al., 17 Sep 2025).

The standard size measure is

$N\times N$9

or, in the synchronous factorization notation used for Kronecker powers, $M$0 when $M$1 (Alman et al., 17 Sep 2025, Alman et al., 2022). This makes the model particularly well suited to sparse factorizations and tensor-product constructions. In the disjointness-matrix literature, the same depth-2 viewpoint is described as a decomposition of the target matrix into rank-1 matrices, with complexity measured either by size or by degree (Ye, 16 Mar 2026).

Degree is a second parameter that becomes central when the circuit is interpreted algorithmically or when the coefficient domain allows cancellations. One formulation defines row and column degrees by the support sizes of the corresponding columns of $M$2 and $M$3, with

$M$4

and asymptotically

$M$5

for Kronecker powers (Alman et al., 17 Sep 2025). In the disjointness setting, size bounds are studied over $M$6, while degree bounds are studied over $M$7, because the latter can exploit sign cancellations and transposition symmetry unavailable in the nonnegative setting (Ye, 16 Mar 2026).

A recurring source of confusion is the phrase “depth-2 linear circuits.” In this literature it refers to linear-transform circuits for matrices, not to depth-two linear threshold circuits such as $M$8. The two areas interact conceptually through low-depth complexity, but they are distinct models with different gate semantics, measures, and techniques [(Williams, 2014); (Chen, 2018)].

2. Kronecker powers and the classical $M$9 baseline

The dominant family of examples is formed by Kronecker powers of a fixed base matrix $M=UV^T$0,

$M=UV^T$1

with input length $M=UV^T$2 when $M=UV^T$3 is $M=UV^T$4. Important special cases include the Walsh–Hadamard transform

$M=UV^T$5

and the disjointness matrix

$M=UV^T$6

(Alman et al., 17 Sep 2025). The same disjointness family is denoted $M=UV^T$7 in later work on refined upper bounds (Ye, 16 Mar 2026).

The classical depth-2 construction is based on the mixed product property

$M=UV^T$8

By splitting a Kronecker power into two halves, one obtains a factorization whose two factors have sparsities that multiply to $M=UV^T$9. In the Walsh–Hadamard case,

$U,V\in\mathbb{F}^{N\times G}$0

and since $U,V\in\mathbb{F}^{N\times G}$1, this yields a depth-2 circuit of size $U,V\in\mathbb{F}^{N\times G}$2 (Alman et al., 2022). The same standard argument works for any fixed base matrix and is also described as a generalization of the Fast Walsh-Hadamard transform or Yates’ algorithm (Alman et al., 2022, Alman et al., 17 Sep 2025).

For many years this $U,V\in\mathbb{F}^{N\times G}$3 upper bound was the universal benchmark. One early development was that the disjointness matrix admits better constructions than the generic baseline: the classical mixed-product route gives about $U,V\in\mathbb{F}^{N\times G}$4, while Jukna and Sergeev improved this to $U,V\in\mathbb{F}^{N\times G}$5 (Alman et al., 2022). Later work generalized this phenomenon and showed that nontrivial savings over $U,V\in\mathbb{F}^{N\times G}$6 are available for every fixed base size $U,V\in\mathbb{F}^{N\times G}$7, not only for $U,V\in\mathbb{F}^{N\times G}$8 matrices (Alman et al., 2022).

This baseline is important because the subsequent literature is largely an attempt to answer two questions. First, how far below $U,V\in\mathbb{F}^{N\times G}$9 can one push depth-2 size for concrete Kronecker-power transforms? Second, which balancing obstructions are genuinely inherent to all rebalancing-based constructions, and which merely reflect the limitations of a particular decomposition scheme? The modern answers to these questions are formulated through rebalancing, $M=\sum_{i=1}^G U_iV_i^T$0-weighted size, and asymptotic spectral duality (Alman et al., 17 Sep 2025).

3. Rebalancing, imbalanced decompositions, and asymptotic spectrum

A major step beyond the baseline was the observation that a nontrivial decomposition of a base matrix $M=\sum_{i=1}^G U_iV_i^T$1 can be lifted to improved depth-2 circuits for $M=\sum_{i=1}^G U_iV_i^T$2. If

$M=\sum_{i=1}^G U_iV_i^T$3

then

$M=\sum_{i=1}^G U_iV_i^T$4

so the problem becomes one of controlling the imbalance between the two sides of each tensor term (Alman et al., 2022). The 2022 construction introduced multiple update rules—soft balancing via the nontrivial decomposition and hard balancing via the trivial decompositions $M=\sum_{i=1}^G U_iV_i^T$5 and $M=\sum_{i=1}^G U_iV_i^T$6—and analyzed the resulting process as a random walk on log-sparsities (Alman et al., 2022).

The key quantity in that framework is the imbalance

$M=\sum_{i=1}^G U_iV_i^T$7

The process tracks

$M=\sum_{i=1}^G U_iV_i^T$8

and shows, under an explicit “imbalanced decomposition” condition, that the probability of falling into the hard-balancing stage decays exponentially. This yields the main theorem: if $M=\sum_{i=1}^G U_iV_i^T$9 has such an imbalanced decomposition, then for every $\nnz(U)+\nnz(V)$0,

$\nnz(U)+\nnz(V)$1

has a depth-2 linear circuit of size $\nnz(U)+\nnz(V)$2, where $\nnz(U)+\nnz(V)$3 is defined from the geometric means $\nnz(U)+\nnz(V)$4 (Alman et al., 2022).

The later asymptotic-spectrum perspective recast these balancing phenomena in a more intrinsic form. For a matrix $\nnz(U)+\nnz(V)$5,

$\nnz(U)+\nnz(V)$6

and, for $\nnz(U)+\nnz(V)$7,

$\nnz(U)+\nnz(V)$8

The central duality theorem states

$\nnz(U)+\nnz(V)$9

showing that the optimal asymptotic depth-2 size is exactly governed by the best $O(N^{1.5})$0-weighted relaxation (Alman et al., 17 Sep 2025).

This formulation identifies the correct obstruction quantity for rebalancing. Given a decomposition family indexed by $O(N^{1.5})$1,

$O(N^{1.5})$2

The theory proves a weak duality lower bound

$O(N^{1.5})$3

and then a strong duality upper bound

$O(N^{1.5})$4

The paper explicitly interprets this as showing that the obstructions discovered by rebalancing are complete up to subexponential factors; in hindsight, earlier rebalancing arguments were proving special cases of Strassen’s duality theorem in an appropriate semiring (Alman et al., 17 Sep 2025).

For the disjointness matrix, this paradigm was sharpened further in 2026. The earlier “wild” rebalancing process was replaced by a tame, discretized process on a finite lattice $O(N^{1.5})$5, with recursion

$O(N^{1.5})$6

The asymptotic geometric-average degree growth is then governed by a quenched top Lyapunov exponent of a random matrix product, and the size exponent is certified through the convex-optimization upper bound of Gharavi and Anantharam (Ye, 16 Mar 2026). This is a conceptual shift from heuristic balancing to a finite-state dynamical system with rigorous spectral certification.

4. Quantitative upper bounds for canonical matrices

The modern literature provides a sequence of explicit improvements for depth-2 circuits computing Kronecker-power transforms. The results are best viewed as a hierarchy: a universal $O(N^{1.5})$7 baseline, general savings for all fixed base dimensions, sharper special-purpose constructions for Walsh–Hadamard and disjointness, and then highly optimized disjointness-specific bounds in both size and degree models.

Object	Best stated bound in the data	Source
Any $O(N^{1.5})$8 Kronecker power	$O(N^{1.5})$9, $N\times N$0	(Alman et al., 2022)
Any $N\times N$1 Kronecker power	$N\times N$2	(Alman et al., 2022)
Walsh–Hadamard transform	$N\times N$3	(Alman et al., 2022)
Disjointness matrix	$N\times N$4	(Alman et al., 2022)
Disjointness matrix over any field	$N\times N$5	(Alman et al., 17 Sep 2025)
$N\times N$6 over $N\times N$7	$N\times N$8	(Ye, 16 Mar 2026)
$N\times N$9 degree over $M$0	$M$1	(Ye, 16 Mar 2026)

The 2022 results established that every fixed base size $M$2 admits a nontrivial power saving over $M$3, with $M$4 (Alman et al., 2022). For $M$5, the paper gave $M$6 for arbitrary $M$7 Kronecker powers, $M$8 for Walsh–Hadamard, and $M$9 for disjointness. The disjointness improvement used a decomposition of $N\times N$00 that was “imbalanced but not one-sided,” precisely the regime where the new theorem was needed (Alman et al., 2022).

The 2025 asymptotic-spectrum framework pushed disjointness below the $N\times N$01 exponent barrier. The paper proves that, over any field, the $N\times N$02 disjointness matrix has a depth-2 linear circuit of size $N\times N$03, and explicitly stresses that this is the first construction which surpasses exponent $N\times N$04 (Alman et al., 17 Sep 2025). The improvement comes from an enhanced partition/decomposition of $N\times N$05, combining a generalized “take one row or column at a time” construction with a more powerful merging construction and constant-weight binary codes (Alman et al., 17 Sep 2025).

The 2026 refinement improves both the size and degree exponents for $N\times N$06. Over $N\times N$07, it achieves

$N\times N$08

improving the previous bound $N\times N$09. Over $N\times N$10, it gives degree

$N\times N$11

improving the previous $N\times N$12 (Ye, 16 Mar 2026). The degree proof is region-sensitive: it analyzes a cost landscape on the $N\times N$13-plane and shows that different families—Sonetto, Regulus, and Regulus$N\times N$14—dominate in different regions (Ye, 16 Mar 2026).

These results collectively show that depth-2 linear circuit upper bounds are now driven by fine-grained asymptotic optimization rather than by a single universal construction. A plausible implication is that the field has moved from discovering isolated decompositions to understanding a structured design space, where entropy terms, weighted degrees, and tensor asymptotics all enter the objective explicitly.

5. Lower bounds, barriers, and conditional hardness

Despite substantial progress on upper bounds, robust unconditional lower bounds for depth-2 linear circuits remain limited, and the strongest current barriers are conditional. One major line of evidence comes from reductions to fine-grained complexity. For the Walsh–Hadamard matrix $N\times N$15 of side-length $N\times N$16, the Orthogonal Vectors Conjecture or the Strong Exponential Time Hypothesis each imply that there is no depth-2 linear circuit of size $N\times N$17 (Alman et al., 17 Sep 2025). A similar result holds for the disjointness matrix $N\times N$18, with a restriction that the $N\times N$19 circuit is somewhat balanced (Alman et al., 17 Sep 2025).

The same paper also argues that proving an $N\times N$20 lower bound for these matrices would have consequences far beyond transform complexity. In particular, if $N\times N$21 of side-length $N\times N$22 does not have a depth-2 linear circuit of size $N\times N$23, then Boolean Inner Product on $N\times N$24-bit vectors does not have $N\times N$25-size $N\times N$26 circuits, and analogous implications are given for Walsh–Hadamard and Boolean Inner Product mod $N\times N$27 (Alman et al., 17 Sep 2025). This places depth-2 linear-circuit lower bounds close to major open frontiers in threshold-circuit complexity.

A different but related barrier appears in the wire-count model for encoding good codes. There, $N\times N$28 denotes the minimum number of wires of a depth-$N\times N$29 unbounded-fan-in linear circuit encoding a code $N\times N$30 with distance at least $N\times N$31. The main theorem gives

$N\times N$32

and

$N\times N$33

while any linear-size encoder must have depth at least

$N\times N$34

for sufficiently large $N\times N$35 (Drucker et al., 2024). For depth $N\times N$36, the paper gives the specific bound

$N\times N$37

which shows that depth $N\times N$38 is substantially larger than linear in the parameter regimes of interest (Drucker et al., 2024). This is a different model from matrix-factorization depth-2 circuits, but it illustrates a recurring phenomenon: depth $N\times N$39 is often too rigid to support simultaneously strong combinatorial properties and linear size.

Taken together, these barriers indicate that upper-bound improvements do not yet amount to a settled structural theory. The existing evidence suggests that unconditional $N\times N$40 lower bounds for depth-2 linear transforms are difficult precisely because they would interact with long-standing open problems in fine-grained complexity and threshold-circuit lower bounds (Alman et al., 17 Sep 2025).

6. Relation to neighboring low-depth models

Depth-2 linear circuits are part of a broader low-depth landscape, but several adjacent models should be kept separate. The most common source of ambiguity is the threshold model $N\times N$41, where each gate computes a linear threshold function

$N\times N$42

This is a Boolean circuit model rather than a linear-transform factorization model. Nontrivial lower bounds for depth-two linear threshold circuits remain open, and one paper presents an all-input evaluation algorithm for depth-two linear threshold circuits of size $N\times N$43 with exponential weights running in $N\times N$44 time, explicitly describing this as evidence that non-uniform lower bounds for $N\times N$45 are within reach (Williams, 2014). A complementary Williams-style study shows that shaving polylogarithmic factors from algorithms for problems such as Hopcroft, $N\times N$46-furthest pair, bichromatic $N\times N$47-closest pair, or Max-IP would imply that $N\times N$48 has no polynomial-size $N\times N$49 circuits (Chen, 2018).

Another nearby Boolean regime is $N\times N$50, which consists of a top symmetric or threshold gate fed by unbounded-fan-in AND gates. For this class, improved correlation bounds lead to a PRG with seed length

$N\times N$51

improving the earlier $N\times N$52 dependence (Servedio et al., 2018). These are depth-two circuits, but not depth-2 linear circuits in the transform sense.

Conversely, some papers on “linear-size” circuits are only indirectly relevant. Fast adder constructions with depth

$N\times N$53

concern binary adder circuits, not general depth-2 linear transforms (Held et al., 2015). Likewise, faster linear-size And-Or path and adder circuits achieve logarithmic depth with iterated-log additive terms, and the paper explicitly states that it is not about depth-2 linear circuits in the usual sense (Brenner et al., 2024).

These distinctions matter because the methodological overlap can be genuine while the models remain different. Rebalancing, balancing obstructions, fan-out constraints, wire complexity, and Williams-style algorithmic barriers all recur across the literature, but they apply to different notions of “linearity.” In the strict sense used for matrices and Kronecker powers, depth-2 linear circuits are best understood as sparse factorizations of linear transforms, with the current frontier centered on asymptotic size, degree, and the completeness of rebalancing-based design principles (Alman et al., 17 Sep 2025, Ye, 16 Mar 2026).