Clifford groups are not always 2-designs

Abstract: The Clifford group is the quotient of the normalizer of the Weyl-Heisenberg group in dimension $d$ by its centre. We prove that when $d$ is not prime the Clifford group is not a group unitary $2$-design. Furthermore, we prove that the multipartite Clifford group is not a group unitary 2-design except for the known cases wherein the local Hilbert space dimensions are a constant prime number. We also clarify the structure of projective group unitary $2$-designs. We show that the adjoint action induced by a group unitary $2$-design decomposes into exactly two irreducible components; moreover, a group is a unitary 2-design if and only if the character of its so-called $U\overline{U}$ representation is $\sqrt{2}$.