Introduction Rice is one of the world's leading food crops, with nearly half of the world's population eating rice as their staple food. Rice yield is directly related to varieties, and the most intuitive agronomic trait of varietal yield is the number of grains per panicle. Methods In this study, rice panicles are taken as the research object, and images of the panicles are captured using a smartphone. The CSRNet counting model based on deep learning is then improved and applied to the problem of counting the number of grains per panicle in rice. Results and discussion The results show that the method of this study has a mean error value of 3.83% on the final validation set. On this basis, the development of rice per panicle counting APP based on Android terminal and batch counting software RiceGrainCounter based on PC terminal realizes real-time counting on Android terminal and batch counting on PC terminal, which can provide theoretical basis and technical support for rice per panicle counting.