Automated diagnosis of bone metastasis based on multi-view bone scans using attention-augmented deep neural networks

Bone scintigraphy is accepted as an effective diagnostic tool for whole-body examination of bone metastasis. However, the manual analysis of bone scintigraphy images requires extensive experience and is exhausting and time-consuming. An automated diagnosis system for such images is therefore much desired. Although automatic or semi-automatic methods for the diagnosis of bone scintigraphy images have been widely studied, they employ various steps to classify the images, including segmentation of the entire skeleton, detection of hot spots, and feature extraction, which are complex and inadequately validated on small datasets, thereby resulting in low accuracy and reliability. In this paper, we describe the development of a deep convolutional neural network to determine the absence or presence of bone metastasis. This model consisting of three sub-networks that aim to extract, aggregate, and classify high-level features in a data-driven manner. There are two main innovations behind this method; First, the diagnosis is performed by jointly analyzing both anterior and posterior views, which leads to high accuracy. Second, a spatial attention feature aggregation operator is proposed to enhance the spatial location information. A large annotated bone scintigraphy image dataset containing 15,474 examinations from 13,811 patients was constructed to train and evaluate the model. The proposed method is compared with three human experts. The high classification accuracy achieved demonstrates the effectiveness of the proposed architecture for the diagnosis of bone scintigraphy images, and that it can be applied as a clinical decision support tool.