Restriction of some unitary representations of O(1,N) to symmetric subgroups

Abstract: We find the complete branching law for the restriction of certain unitary representations of $O(1,n+1)$ to the subgroups $O(1,m+1)\times O(n-m)$, $0\leq m\leq n$. The unitary representations we consider belong either to the unitary spherical principal series, the spherical complementary series or are unitarizable subquotients of the spherical principal series. In the crucial case $0<m<n$ the decomposition consists of a continuous part and a discrete part. The continuous part is given by a direct integral of unitary principal series representations whereas the discrete part consists of finitely many representations which either belong to the complementary series or are unitarizable subquotients of the principal series. The explicit Plancherel formula is computed on the Fourier transformed side of the non-compact realization of the representations by using the spectral decomposition of a certain hypergeometric type ordinary differential operator. The main tool connecting this differential operator with the representations are second order Bessel operators which describe the Lie algebra action in this realization. To derive the spectral decomposition of the ordinary differential operator we use Kodaira's formula for the spectral decomposition of Schr\"odinger type operators.