Positive solutions for a coupled nonlinear Kirchhoff-type system with vanishing potentials

Abstract: In this paper, we consider the strongly coupled nonlinear Kirchhoff-type system with vanshing potentials: \begin{equation*}\begin{cases} -\left(a_1+b_1\int_{\mathbb{R}^3}|\nabla u|^2\dx\right)\Delta u+\lambda V(x)u=\frac{\alpha}{\alpha+\beta}|u|^{\alpha-2}u|v|^{\beta},&x\in\mathbb{R}^3,\ -\left(a_2+b_2\int_{\mathbb{R}^3}|\nabla v|^2\dx\right)\Delta v+\lambda W(x)v=\frac{\beta}{\alpha+\beta}|u|^{\alpha}|v|^{\beta-2}v,&x\in\mathbb{R}^3,\ u,v\in \mathcal{D}^{1,2}(\R^3), \end{cases}\end{equation*} where $a_i>0$ are constants, $\lambda,b_i>0$ are parameters for $i=1,2$, $\alpha,\beta>1$ and $\alpha+\beta\leqslant 4$, $V(x)$, $W(x)$ are nonnegative continuous potentials, the nonlinear term $F(x,u,v)=|u|^\alpha|v|^\beta$ is not 4-superlinear at infinity. Such problem cannot be studied directly by standard variational methods, even by restricting the associated energy functional on the Nehari manifold, because Palais-Smale sequences may not be bounded. Combining some new detailed estimates with truncation technique, we obtain the existence of positive vector solutions for the above system when $b_1+b_2$ small and $\lambda$ large. Moreover, the asymptotic behavior of these vector solutions is also explored as $\textbf{b}=(b_1,b_2)\to \bf{0}$ and $\lambda\to\infty$. In particular, our results extend some known ones in previous papers that only deals with the case where $4<\alpha+\beta<6$.