Multi-Modal Lightweight Deep Learning Model for Typhoon Prediction

This research proposes a multimodal deep learning model (VIT-DBN) to enhance typhoon forecasting capabilities. Traditional machine learning methods relying on unimodal data have limited risk assessment accuracy. Meanwhile, overlooking temporal signals hinders deep learning approaches. The VIT-DBN model assimilates diverse data modalities to confer a comprehensive understanding of typhoon dynamics. Specifically, a Vision Transformer (ViT) and Deep Belief Network (DBN) are integrated to concurrently process imagery and time series data. The model optimizes architecture, parameters and complexity for lightweight deployment on resource-constrained devices. Comparative analyses demonstrate substantial gains in prediction accuracy over conventional machine learning, evidencing the efficacy of fusing multimodal data. This technique promotes real-time forecasting by enabling integration with mobile and edge devices. Explicit focus on spatial relationships in imagery via positional embeddings and self-attention in the ViT, along with temporal pattern capture by the DBN, underpins the performance gains. By leveraging diverse data modalities, this robust methodology enhances early warning systems and improves timeliness of typhoon predictions.