Improving deep-learning segmentation performance in 3D neuroimaging with minimal manual annotations

One of the current challenges in applying machine learning to medical images is the difficulty in obtaining labelled training data. While medical images themselves are often available, generating high-quality training labels for them is time-consuming and often requires a trained clinician. This problem is particularly acute in 3D segmentation, which generally requires detailed voxel-by-voxel segmentation maps. This thesis proposes a series of image analysis methods to leverage additional available information to improve image segmentation quality when there are limited manual labels available. Using a dataset of 3D fetal ultrasound scans with only a small number of initial segmentation labels, we demonstrate a benefit from different information sources. The approaches used in this thesis are: • Generating segmentation labels automatically from a publicly available spatio-temporal atlas, making use of prior anatomical information. We use anatomical keypoints to guide registration of atlases across different imaging modalities. We then propagate these segmentations to individual ultrasound volumes, automatically generating a set of multi-label segmentations with an average Dice coefficient of 0.818 across the chosen anatomical structures in the range of 20-25 gestational weeks. This set of labels is used to train a segmentation CNN. We find that this produces high-quality segmentations, and that training a single network with the full multi-label training set improves segmentation quality (average Dice coefficient of 0.726 across structures for a multi-task network, versus 0.659 for individual single-task networks). • Leveraging additional, unlabelled data to improve performance for a cerebellar segmentation task. Unlabelled data does not have a ground-truth label by which segmentation quality may be compared, but measures of output uncertainty can be derived using test-time augmentation and dropout measures. We use these estimates of uncertainty to inform an iterative omni-supervised framework, using the highest-quality segmentations from the unlabelled dataset as additional training data for a segmentation CNN. We show that the use of these labels as part of a training dataset improves segmentation performance, and that this is improved further when measures of uncertainty are used in data selection (an overall improvement in Dice coefficient for a cerebellar segmentation task from 0.673 to 0.727). • Passing additional, scalar information to a neural network's input. We propose an original method by which scalar data such as gestational age, readily available as part of clinical data acquisition, may be appended to the input of a fully-convolutional neural network. We show that including this information as part of the training dataset can lead to significant improvements in segmentation quality on the same cerebellar segmentation task, in the reported experiments improving test Dice coefficient from 0.673 to 0.723. We find that this improvement is not sensitive to imperfect information passed at test time. The computational methods developed in this thesis may reduce the reliance on large manually-labelled datasets for medical image analysis and save time in generating expert annotations.