QNMF: A quantum neural network based multimodal fusion system for intelligent diagnosis

The Internet of Medical Things (IoMT) has emerged as a significant research area in the medical field, enabling the transmission of various types of data to the cloud for analysis and diagnosis. Fusing data from multiple modalities can enhance accuracy but requires substantial computing power. Theoretically, quantum computers can rapidly process large volumes of high-dimensional medical data. Despite accelerated developments in quantum computing, research on quantum machine learning (QML) for multimodal data processing remains limited. Considering these factors, this paper presents a quantum neural network-based multimodal fusion system for intelligent diagnosis (QNMF) that can process multimodal medical data transmitted by IoMT devices, fuse data from different modalities, and improve the performance of intelligent diagnosis. This system employs a quantum convolutional neural network (QCNN) to efficiently extract features from medical images. These QCNN-based features are then fused with other modality features (such as blood test results or breast cell slices), and used to train an effective variational quantum classifier (VQC) for intelligent diagnosis. The experimental results demonstrate that a QCNN can effectively extract image data features. Furthermore, QNMF achieved an accuracy of 97.07% and 97.61% on breast cancer diagnosis and Covid-19 diagnosis experiments, respectively. In addition, the QNMF exhibits strong quantum noise robustness.