Multi-View Fusion Convolutional Neural Network for Automatic Landmark Location on Spinal X-Rays

In clinical practice, landmark location plays an important role in spine deformity assessment, which is the foundation for measurement of several spinal morphological parameters. The clinicians usually use both anterior-posterior(AP) view X-rays and lateral(LAT) view X-rays of the same patients for diagnosis. However, for automatic landmark location, the information between multi-view X-rays is seldom considered. Addressing this problem, in this paper, we propose a multi-view fusion convolutional neural network for automatic landmark location on AP X-rays and LAT X-rays simultaneously. Based on an object detection framework, for two channels representing multi-view X-rays, we first share their network parameters in convolutional backbone, and then we design an image-level fusion module and an object-level fusion module respectively, which can combine the information of both channels. Finally we insert a landmark prediction branch to the end of each channel for landmark location. The experiment results show that our proposed method achieves more accurate vertebra detection and more precise landmark location than predicting them separately, which can provide reliable assistance for clinicians.