A two-stage deep transfer learning model and its application for medical image processing in Traditional Chinese Medicine

People are deepening the study on model explainability as well as performance to better understand models’ decisions from the human perspective. However, the lack of rare clinical diagnosis data always limits the power of the emerging data-driven deep diagnosis methods, and the traditional deep transfer learning (DTL) applicable for the case is often insufficient to learn specific features in medical image processing, leading to poor explainability. To address those challenges, a two-stage deep transfer learning model is proposed and applied to assist in the Traditional Chinese Medicine (TCM) tongue diagnosis. Especially, a two-stage transfer learning training strategy is designed to loose the data dependence of deep learning on the domain data, which is composed of the imitate stage that discovers shared basic source features and the transfer stage to relearn target patterns, with good explainability. The corresponding deep squeeze-and-excitation convolutional network is proposed to learn the clinical patterns of tongue symptoms, in which a three-layer feature pyramid network fuses the multi-scale tongue features. Extensive experiments are conducted on the real clinical dataset in terms of classification accuracy and learning efficiency. The resulting accuracy of the proposed model proves its performance advantage with the recognition time achieving real-time performance.