On the Analytic Structure of Commutative Nilmanifolds

Abstract: In the classification theorems of Vinberg and Yakimova for commutative nilmanifolds, the relevant nilpotent groups have a very surprising analytic property. The manifolds are of the form $G/K = N \rtimes K/K$ where, in all but three cases, the nilpotent group $N$ has irreducible unitary representations whose coefficients are square integrable modulo the center $Z$ of $N$. Here we show that, in those three "exceptional" cases, the group $N$ is a semidirect product $N_1 \rtimes \mathbb{R}$ or $N_1 \rtimes \mathbb{C}$ where the normal subgroup $N_1$ contains the center $Z$ of $N$ and has irreducible unitary representations whose coefficients are square integrable modulo $Z$. This leads directly to explicit harmonic analysis and Fourier inversion formulae for commutative nilmanifolds.