On eigenfunction expansions of first-order symmetric systems and ordinary differential operators of an odd order

Abstract: We study general (not necessarily Hamiltonian) first-order symmetric systems $J y'-B(t)y=\D(t) f(t)$ on an interval $\cI=[a,b\rangle $ with the regular endpoint $a$. It is assumed that the deficiency indices $n_\pm(\Tmi)$ of the minimal relation $\Tmi$ satisfy $n_+(\Tmi)< n_-(\Tmi)$. We define $\l$-depending boundary conditions which are analogs of separated self-adjoint boundary conditions for Hamiltonian systems. With a boundary value problem involving such conditions we associate an exit space self-adjoint extension $\wt T$ of $\Tmi$ and the $m$-function $m(\cd)$, which is an analog of the Titchmarsh-Weyl coefficient for the Hamiltonian system. By using $m$-function we obtain the eigenfunction expansion with the spectral function $\Si(\cd)$ of the minimally possible dimension and characterize the case when spectrum of $\wt T$ is defined by $\Si(\cd)$. Moreover, we parametrize all spectral functions in terms of a Nevanlinna type boundary parameter. Application of these results to ordinary differential operators of an odd order enables us to complete the results by Everitt and Krishna Kumar on the Titchmarsh-Weyl theory of such operators.