A fully nonlinear Sobolev trace inequality

Abstract: The $k$-Hessian operator $\sigma_k$ is the $k$-th elementary symmetric function of the eigenvalues of the Hessian. It is known that the $k$-Hessian equation $\sigma_k(D^2u)=f$ with Dirichlet boundary condition $u=0$ is variational; indeed, this problem can be studied by means of the $k$-Hessian energy $-\int u\sigma_k(D^2u)$. We construct a natural boundary functional which, when added to the $k$-Hessian energy, yields as its critical points solutions of $k$-Hessian equations with general non-vanishing boundary data. As a consequence, we prove a sharp Sobolev trace inequality for $k$-admissible functions $u$ which estimates the $k$-Hessian energy in terms of the boundary values of $u$.