---
title: Hadamard Rank in Matrices and Tensors
url: https://www.emergentmind.com/topics/hadamard-rank
type: topic
---

# Hadamard Rank in Matrices and Tensors

Hadamard rank is a matrix and tensor invariant central to algebraic statistics, combinatorics, coding theory, graph theory, and computational linear algebra. It quantifies the minimal number or minimal rank of factors required to express a matrix, tensor, or point in a projective variety as a Hadamard (entrywise or coefficientwise) product of structured elements. The concept generalizes classical notions of matrix rank and tensor rank by leveraging the algebraic properties of entrywise product, revealing deep combinatorial and geometric structure in codes, product distributions, varieties, and matrix decompositions.

## 1. Formal Definitions and Conceptual Landscape

The Hadamard product of matrices $A,B\in\mathbb{F}^{m\times n}$ is $(A\circ B)_{ij}=a_{ij}b_{ij}$. For tensors or projective points $p,q\in\mathbb{P}^N$, $p\star q=[p_0q_0\, : \, p_1q_1\, :\,\ldots\, :\, p_N q_N]$.

Several forms of Hadamard rank are in use:
- For a matrix $M$, the **$(r,s)$-Hadamard expressibility** asks for decompositions $M=A\circ B$ with $\mathrm{rank}(A)\le r,\mathrm{rank}(B)\le s$ [2508.14901].
- The **$r$-Hadamard rank**, $\mathrm{hr}_r(M)$, is the minimal $k$ such that $M$ is an entrywise product of $k$ matrices of rank at most $r$.
- In algebraic geometry, **Hadamard rank with respect to a variety $X$** seeks the minimal $m$ for which $p=q_1\star\cdots\star q_m$ with $q_i\in X$ [2510.05231].

A matrix factorization $A\circ A^T=I_n$ leads to the **Hadamard rank of the identity**: $\min\{\mathrm{rank}(A):A\circ A^T=I_n\}$ [1310.7321].

In probabilistic and statistical models, the **Hadamard extension** of an $m\times n$ matrix $A$ is the $2^m\times n$ matrix with all possible entrywise products of subsets of rows, and its column rank—**Hadamard extension rank**—controls identifiability in mixture models [2101.11688].

## 2. Foundational Bounds, Constructions, and Algebraic Structure

### Matrix Factorization and Rank Thresholds

For the identity matrix $I_n$, any Hadamard factorization $A$ must satisfy $\mathrm{rank}(A)\ge\sqrt{n}$ by submultiplicativity: $I_n=A\circ A^T\Rightarrow n\le\mathrm{rank}(A)^2$ [1310.7321].

Explicit constructions match this threshold up to constant factors, e.g., blockwise Toeplitz matrices $A_r$ of size $n=r(r+1)/2$ and rank $r$ such that $A_r\circ A_r^T=I_n$ [1310.7321]. Boolean and finite field constructions yield similar bounds.

In the $4\times4$ case, not every full-rank matrix over $\mathbb{F}_2$ admits a Hadamard factorization into two rank-2 matrices, disproved by exhaustive enumeration: $26.3\%$ of full-rank $4\times4$ binary matrices are not Hadamard-expressible as products of two rank-2 matrices [2508.14901]. This non-expressibility persists over $\mathbb{Z}$ and likely over $\mathbb{R}$.

### Hadamard Powers and Rank-Jumping

For a symmetric DNN matrix $A$, the rank of $A^{\circ r}$ can be sharply estimated:
- For $k=\mathrm{Hrk}(A)$, $\mathrm{rank}(A^{\circ r})=k$ for $r>k-2$ [2004.03909].
- Rank-two PSD matrices with all entries positive have $\mathrm{rank}(A^{\circ r})=r+1$ for $r\in\{0,1,\ldots,k-2\}$, and $=k$ for $r>k-2$; all nonzero eigenvalues of $A^{\circ r}$ are simple.

Generic $n\times m$ real matrices of rank $r$ obey $\mathrm{rank}(A^{\circ d})=\min\left\{\binom{r+d-1}{d},n,m\right\}$ for $d\ge 1$ [2207.08853].

Boolean Gram matrices $K=X^TX$ of $2^n$ binary vectors have:
\[
\mathrm{rank}(K^{\circ d}) = 
\begin{cases}
\sum_{p=1}^d \binom{n}{p} & d\le n\\
2^n-1 & d\ge n
\end{cases}
\]
with the rank controlling the shattering capacity of kernel perceptrons.

## 3. Hadamard Rank in Algebraic Geometry and Statistical Models

In projective geometry, the Hadamard rank of a point $p$ with respect to a variety $X\subset\mathbb{P}^N$ is the minimal $m$ for which $p=q_1\star\cdots\star q_m$, $q_i\in X$ [2510.05231].

Generic finiteness is characterized by the absence of $X$ in any coordinate hyperplane or binomial hypersurface. If $X$ is concise and free of pure variables/binomials, generic points possess finite $X$-Hadamard rank.

For toric varieties and their secants, general $p$ has finite Hadamard rank. For points $p$ with all coordinates nonzero, $Hrk_X(p)\le2Hrk_X^\circ$.

Explicit formulas for Segre-Veronese varieties:
\[
Hrk_{X,r}^\circ = \left\lceil \frac{\prod_{i=1}^k\binom{n_i+d_i}{d_i}-(n_1+\cdots+n_k)}{(r-1)[(n_1+\cdots+n_k)+1]} \right\rceil
\]
for non-defective cases. For symmetric tensors (Veronese), Alexander-Hirschowitz provides expected generic ranks except four sporadic exceptions.

## 4. Hadamard Rank in Coding Theory and Graphs

### Propelinear and Generalized Hadamard Codes

Hadamard full propelinear codes of type Q, with group presentation $\langle a,b : a^{4n}=e, a^{2n}=b^2, b^{-1}ab=a^{-1}\rangle$, have ranks and kernel dimensions tightly governed by the $2$-power decomposition $4n=2^s\cdot n'$, $n'$ odd:
- $s=2$: $r=4n-1$, $k=1$
- $s=3$: $r=2n$, $k\in\{1,2\}$
- $s>3$: $r<2n$, $k\in\{1,2\}$
with $k=2$ forcing the transpose code's kernel to drop to $1$ [1709.02465].

For $\mathbb{F}_p$-additive generalized Hadamard codes, given $q=p^e$, $n=p^t$:
- Kernel dimension bounds: $1\le\kappa=\kappa_p\le 1+\lfloor t/e\rfloor$
- Rank bounds for fixed kernel: $e+t-(e-1)k\le r\le1+t-(e-1)(k-1)$
For $e=2$, $\mathrm{rank}(C)+\kappa=2+t$ and every admissible $(\kappa,r)$ is attainable [2001.11609].

### $\mathbb{Z}_2\mathbb{Z}_4Q_8$-codes

Hadamard codes arising from subgroups of $G=\mathbb{Z}_2^{k_1}\times\mathbb{Z}_4^{k_2}\times Q_8^{k_3}$ are classified into five “shapes,” with allowed pairs $(r,k)$ governed by the parameters $m+1−k \in \{0,4,\tau−1,\tau,\tau+1\}$, where $\tau$ is a shape-dependent commutator count. Each admissible $(r,k)$ is constructible via group-theoretic generator and duplication procedures [1405.3780].

### 2-Rank of Graphical Hadamard Matrices

A graphical Hadamard matrix $H$ yields $G_H$ with adjacency $A$, and Hadamard rank $\mathrm{rank}_2(A)$ over $\mathrm{GF}(2)$. Seidel and Godsil–McKay switching operations change rank by $\pm2$. Kronecker-type products combine ranks additively or with a $-2$ correction if the all-ones vector is in both constituent column spaces. For order $4^m$ matrices, unbounded but not maximal 2-rank values are realized via iterated products and switching [1801.01149].

## 5. Algorithms and Computational Aspects

Efficient algorithms for Hadamard decomposition employ block-coordinate descent (BCD) alternating optimization. Given $X\in \mathbb{R}^{m\times n}$, seek decomposition $X\approx(W_1H_1)\odot(W_2H_2)$ (elementwise), minimizing $\|X-(W_1H_1)\odot(W_2H_2)\|_F^2$ [2504.13633]. The subproblems reduce to convex quadratic least-squares, solvable exactly (via an $r\times r$ system per column) or iteratively by optimal-step gradient descent. Momentum-based Nesterov acceleration and SVD-based initialization (splitting magnitude and sign) yield rapid convergence and improved minima.

For $p>2$ factors,
\[
X\approx\bigodot_{i=1}^p(W_iH_i) \quad \text{with effective rank} \le \prod_{i} r_i
\]
The partition maximizing effective rank uses predominantly rank-$3$ factors.

Empirical comparisons show Hadamard models outperform SVD for the same parameter budget, especially on sparse graphs and networks.

## 6. Invariants, Statistical Applications, and Open Problems

Hadamard rank serves as an invariant for matrix and tensor factorization beyond classical rank. The concept enters:
- Mixture identification: Hadamard extension full column rank is necessary for identifiability in mixtures of product distributions [2101.11688].
- Entrywise powering: In kernel learning, Hadamard powers reach maximal shattering capacity at threshold degrees corresponding to classical VC dimension bounds [2207.08853].
- Geometric modeling: Algebraic varieties have Hadamard ranks regulated by tropical geometry and secant variety properties [2510.05231].

Open problems include the explicit determination of algebraic constraints governing Hadamard factorizability (observed low-dimensional varieties in expressible sets [2508.14901]), full characterization of maximal achievable ranks for graphical Hadamard constructions [1801.01149], and extension of rank-jump and monotonicity phenomena to higher-rank and sparse regimes [2004.03909].

## 7. Tabular Summary of Hadamard Rank Constructs

| Context                  | Hadamard Rank Definition                             | Key Results / Bounds                       |
|--------------------------|------------------------------------------------------|--------------------------------------------|
| Matrix factorization     | min rank of $A$: $A\circ A^T=I_n$                    | Lower bound $\ge\sqrt{n}$; $\Theta(\sqrt{n})$ explicit [1310.7321] |
| Entrywise powers         | rank of $A^{\circ r}$                                | Threshold $r>k-2\implies \mathrm{rank}=k$ [2004.03909] |
| Kernel perceptron        | rank of Boolean/real Gram Hadamard powers            | Explicit combinatorial formula for shattering [2207.08853] |
| Algebraic varieties      | $Hrk_X(p)$ for $p\in\mathbb{P}^N$, $X\subset\mathbb{P}^N$ | Finite iff $X$ not in binomial hypersurface/hyperplane [2510.05231] |
| Codes (propelinear, GH)  | rank and kernel of code over $\mathbb{F}_q$          | Tight bounds, explicit constructions [1709.02465, 2001.11609] |
| Graphical Hadamard       | binary rank (2-rank) of adjacency from $H$           | All even values in interval realized, Kronecker sums [1801.01149] |

Hadamard rank is a unifying notion connecting entrywise algebraic structure, combinatorial independence, identifiability in models, and the design of heavily structured codes and decompositions. Across fields, rank-jump phenomena, threshold bounds, and geometric properties inform both theoretical understanding and algorithmic advancements.

Source: https://www.emergentmind.com/topics/hadamard-rank