Existence of Solutions on the Critical Hyperbola for a Pure Lane–Emden System with Neumann Boundary Conditions

Abstract We study the following Lane–Emden system: $$\begin{align*} & -\Delta u=|v|^{q-1}v \quad \ \textrm{in}\ \Omega, \qquad -\Delta v=|u|^{p-1}u \quad \ \textrm{in}\ \Omega, \qquad u_{\nu}=v_{\nu}=0 \quad \ \textrm{on}\ \partial \Omega, \end{align*}$$with $\Omega $ a bounded regular domain of ${\mathbb{R}}^{N}$, $N \ge 4$, and exponents $p, q$ belonging to the so-called critical hyperbola $1/(p+1)+1/(q+1)=(N-2)/N$. We show that, under suitable conditions on $p, q$, least-energy (sign-changing) solutions exist, and they are classical. In the proof we exploit a dual variational formulation, which allows to deal with the strong indefinite character of the problem. We establish a compactness condition which is based on a new Cherrier-type inequality. We then prove such condition by using as test functions the solutions to the system in the whole space and performing delicate asymptotic estimates. If $N \ge 5$, $p=1$, the system above reduces to a biharmonic equation, for which we also prove existence of least-energy solutions. Finally, we prove some partial symmetry and symmetry-breaking results in the case $\Omega $ is a ball or an annulus.