The principal transmission condition

Abstract: The paper treats pseudodifferential operators $P=Op(p(\xi ))$ with homogeneous complex symbol $p(\xi )$ of order $2a>0$, generalizing the fractional Laplacian $(-\Delta )^a$ but lacking its symmetries, and taken to act on the halfspace $R^n_+$. The operators are seen to satisfy a principal $\mu $-transmission condition relative to $R^n_+$, but generally not the full $\mu $-transmission condition satisfied by $(-\Delta )^a$ and related operators (with $\mu =a$). However, $P$ acts well on the so-called $\mu $-transmission spaces over $R^n_+$ (defined in earlier works), and when $P$ moreover is strongly elliptic, these spaces are the solution spaces for the homogeneous Dirichlet problem for $P$, leading to regularity results with a factor $x_n^\mu $ (in a limited range of Sobolev spaces). The information is then shown to be sufficient to establish an integration by parts formula over $R^n_+$ for $P$ acting on such functions. The formulation in Sobolev spaces, and the results on strongly elliptic operators going beyond operators with real kernels, are new. Furthermore, large solutions with nonzero Dirichlet traces are described, and a halfways Green's formula is established, for this new class of operators. Since the principal $\mu $-transmission condition has weaker requirements than the full $\mu $-transmission condition assumed in earlier papers, new arguments were needed, relying on work of Vishik and Eskin instead of the Boutet de Monvel theory. The results cover the case of nonsymmetric operators with real kernel that were only partially treated in a preceding paper.