The separating variety for matrix invariants

Abstract: Let $G$ be a linear algebraic group defined over an algebraically closed field $k$, and let $V$ be a vector space on which $G$ acts linearly. The separating variety $\mathcal{S}<em>{G,V}$ is the subvariety of $V<sup>2$ consisting of pairs of points indistinguishable by invariant polynomials in $k[V]<sup>G$. Its geometry places restrictions on the existence of small separating sets, i.e. sets of invariants which distinguish the same points as the full algebra of invariants. The purpose of this article is to study the separating variety in the important special case where $G=\mathrm{GL}_p(\mathbb{C})$ acts on the set $V$ of $n$-tuples of $p \times p$ matrices by simultaneous conjugation. We define a purely combinatorial poset, $\mathcal{P}</em>{p,n}$, whose maximal elements are in 1-1 correspondence with the irreducible components of $\mathcal{S}<em>{G,V}$. We show that $\mathcal{S}</em>{G,V}$ is a variety of dimension $(n+1)p<sup>2-1$, and determine its subdimension for all $n$ and $p$. In particular we show the subdimension is $(n+1)p<sup>2-p$ if $n \geq 3$, or $n \geq 2$ and $p \geq 4$. In the case $n \geq 3$, we give a formula for the number of components of given codimension in $\mathcal{S}<em>{G,V}$. We give explicit decompositions of $\mathcal{S}</em>{G,V}$ for all $n$ where $p=2,3$ or $4$. Our results in particular show that when $n\geq 2$ and $p\geq 4$, or $n\geq 3$ and $p=3$, $\mathbb{C}[V]<sup>G$ does not contain a polynomial or hypersurface separating set. It was proven in arXiv:2202.05717 that the same is true if $n \geq 4$ and $p=2$. The author made a conjecture in arXiv:2211.17088 generalising the Skronowski-Weyman theorem for representations of quivers. The results of this paper prove that conjecture in two important special cases: for the quiver with one vertex and an arbitrary number, $n$, of loops, and for the quiver with two vertices and $n$ arrows between them.