Boosting bacterial detection with hyperspectral mining

Automatically detecting bacteria from pathological sections is of great significance in clinical practice, providing precious information for accurate decision-making in disease diagnosis and treatment. However, traditional bacterial identification methods require professional medical equipment and operations, making them costly and time-consuming. Learning-based methods can detect bacteria through spatial features, but their accuracy is unsatisfactory due to the limited modeling capabilities of existing deep-learning models. Considering that RGB images contain both spatial and spectral target information, here we propose to investigate the latent spectral features to enhance accurate and efficient bacterial detection. Specifically, we first performed hyperspectral image (HSI) reconstruction from RGB bacterial images, which can investigate underlying spectral features without additional cumbersome and expensive spectral imaging systems. The HSI reconstruction network builds on the spatial-frequency block (SF-block) under the U-shaped architecture. The SF-block combines frequency-wise self-attention (FWSA) and spatial-wise local-window self-attention (LWSA) modules in a parallel design. Such a framework can effectively model the spatial sparsity of bacteria and the interspectral similarity of HSI. It also enables the complementary fusion of spatial and spectral features, establishes cross-window connections, and expands the receptive field while maintaining linear complexity. Then, by stacking SF-blocks at multiple scales, we can effectively detect bacteria from the reconstructed HSIs by integrating both spectral and spatial features, and output each bacterium’s location, size, and category. We constructed a large-scale bacterial detection data set for network training and testing that contains 2910 labeled images over four common bacterial categories. Extensive experiments show that our method achieved state-of-the-art bacterial detection accuracy of 92.4% at a speed of 11 FPS, which is 3 orders of magnitude faster than traditional methods.