A handover scheme based on Elman network is proposed in this paper to reduce link failure and enhance the user experience under LTE-R system. In this handover scheme, we divide the different scenarios and set up corresponding neural network prediction system with which the handover decision parameters like RSRP and RSRQ can be continuously observed and predicted. By correlating past measurement parameters with future handover decisions, we can accelerate the handover execution and optimize the handover process. It can be seen from the experimental simulation results that the Elman-based handover algorithm has a better performance than the gray prediction model- based handover algorithm and is more suitable for the highspeed rail scene with changing geographical environment.