Stationary distribution, extinction and density function of a stochastic prey-predator system with general anti-predator behavior and fear effect

In this paper, we examine a stochastic prey-predator system with fear effect and general anti-predator behavior. To tackle the impact of stochastic perturbations, we first propose a p -stochastic threshold method to construct several necessary p -Lyapunov functions. Then by defining a quasi-carrying capacity x ∗ , sufficient conditions are established for the existence and uniqueness of stationary distribution ϖ ⋅ of the system. By solving the Fokker-Planck equation, the approximate expression of probability density function of the distribution ϖ ⋅ around its quasi-positive equilibrium is further derived. Besides, the extinction of prey and predator populations is studied. Finally, some numerical examples are provided to verify our theoretical results and study two aspects: (i) the impact of anti-predator behavior; (ii) the effect of prey fear. • The stochastic prey-predator with anti-predator behavior and fear effect is established. • We obtain sufficient but almost conditions for the existence of stationary distribution. • Sufficient conditions for population extinction are obtained. • We derive the approximate expression of density function of the system. • We give parameter analysis regarding the coexistence of population prey and predator.