Evolving graph convolutional network with transformer for CT segmentation

Accurate and robust organ and tumour segmentation from CT scans are critical for precision diagnosis and prognosis of cancer and the development of personalised treatment planning. However, the automatic segmentation of tumours and organs they invade is challenging because of significant variations, abnormalities, and unclear boundaries. While graph convolutional networks can propagate knowledge and correlations in a flexible feature space, they suffer from information saturation during deep learning, limiting their effectiveness. To overcome this issue, we propose a hybrid graph convolution transformer (HCGT) model that consists of a channel transformer (CTrans) and a convolutional graph transformer (convG-Trans). CTrans operates along the feature channel dimension to learn contextual relationships across different feature channels. The convG-Trans learns enriched relationships among distinct elements within the image by concurrently and interactively aggregating knowledge propagation from graph convolution and cross-node similarities from the transformer. Finally, a category-level attention is designed to understand the significance of the two representations from the CTrans and convG-Trans, which help adjust the fusion process before generating the segmentation output. We evaluate the HCGT on kidney and kidney tumour, and lung and non-small cell lung cancer datasets. Our evaluations include comparisons with three-dimensional (3D) medical image segmentation benchmarks and graph- and transformer-based segmentation models. The results demonstrate improved performance in abdominal and thorax organ and tumour segmentation tasks. Additionally, ablation studies show that the major technical innovations are effective and consistent when using different 3D medical image segmentation backbones.