DLNLF-net: Denoised local and non-local deep features fusion network for malignancy characterization of hepatocellular carcinoma

Hepatocellular carcinoma (HCC) is a primary liver cancer with high mortality rate. The degree of HCC malignancy is an important prognostic factor for predicting recurrence and survival after surgical resection or liver transplantation in clinical practice. Currently, deep features obtained from data-driven machine learning algorithms have demonstrated superior performance in characterising lesion features in medical imaging processing. However, previous convolutional neural network (CNN)-based studies on HCC lesion characterisation were based on traditional local deep features. The aim of this study was to propose a denoised local and non-local deep features fusion network (DLNLF-net) for grading HCC. Gadolinium-diethylenetriaminepentaacetic-acid-enhanced magnetic resonance imaging data of 117 histopathologically proven HCCs were collected from 112 patients with resected HCC between October 2012 and October 2018. The proposed DLNLF-net primarily consists of three modules: feature denoising, non-local feature extraction, and bilinear kernel fusion. First, local feature maps were extracted from the original tumour images using convolution operations, followed by a feature denoising block to generate denoised local features. Simultaneously, a non-local feature extraction block was employed on the local feature maps to generate non-local features. Finally, the two generated features were fused using a bilinear kernel model to output the classification results. The dataset was divided into a training set (77 HCC images) and an independent test set (40 HCC images). Training and independent testing were repeated five times to reduce measurement errors. Accuracy, sensitivity, specificity, and area under the curve (AUC) values in the five repetitive tests were calculated to evaluate the performance of the proposed method. Denoised local features (AUC 89.19%) and non-local features (AUC 88.28%) showed better performance than local features (AUC 86.21%) and global average pooling features (AUC 87.1%) that were derived from a CNN for malignancy characterisation of HCC. Furthermore, the proposed DLNFL-net yielded superior performance (AUC 94.89%) than a typical 3D CNN (AUC 86.21%), bilinear CNN (AUC 90.46%), recently proposed local and global diffusion method (AUC 93.94%), and convolutional block attention module method (AUC 93.62%) for malignancy characterisation of HCC. The non-local operation demonstrated a better capability of yielding global representation, and feature denoising based on the non-local operation achieved performance gains for lesion characterisation. The proposed DLNLF-net, which integrates denoised local and non-local deep features, evidently outperforms conventional CNN-based methods in the malignancy characterisation of HCC.