HSI-DETR: A DETR-based Transfer Learning from RGB to Hyperspectral Images for Object Detection of Live and Dead Cells

Traditional cell viability judgment methods are invasive and damaging to cells. Moreover, even under a microscope, it is difficult to distinguish live cells from dead cells by the naked eye alone. With the development of optical imaging technology, hyperspectral imaging is more and more widely used in various fields. Hyperspectral imaging is a non-contact optical technique that provides both spectral and spatial information in a single measurement. It becomes a fast, non-invasive option to differentiate between live and dead cells. In recent years, the rapid development of deep learning has provided a better way to distinguish the difference between living and dead cells through a large amount of data. However, it is often necessary to acquire large amounts of labeled data at an expensive cost to train models. This is more difficult to achieve on medical hyperspectral images. Therefore, in this paper, a new model called HSI-DETR is proposed to solve the above problem on the target detection task of live and dead cells, which is based on the detection transformer (DETR) model. The HSI-DETR model suitable for hyperspectral images (HSI) is proposed with minimal modification. Then, some parameters of DETR trained on RGB images are transferred to HSI-DETR trained on hyperspectral images. Compared to the general method, this method can train a better model with a small number of labeled samples. And compared to the DETR-R50, the AP50 of HSI-DETR-R50 has increased by 5.15%.