The Lane-Emden system near the critical hyperbola on nonconvex domains

Abstract: In this paper we study the asymptotic behavior of minimal energy solutions to the Lane-Emden system $-\Delta u = v^p$ and $-\Delta v = u^q$ on bounded domains as the index $(p,q)$ approaches to the critical hyperbola from below. Precisely, we remove the convexity assumption on the domain in the result of Guerra \cite{G}. The main task is to get the uniform boundedness of the solutions near the boundary because it is difficulty to adapt the moving plane method for the system on nonconvex domains if $\max {p,q} > \frac{n+2}{n-2}$. For the purpose, we shall derive a contradiction by exploiting carefully the Pohozaev type identity if the maximum point approaches to the boundary.