AI-Powered Potato Plant Disease Detection: A Vision-Language Framework

Abstract Potato crops are a vital part of global food security. Potato leaf and crop health play a crucial role in determining the yield and quality of potato production. This paper presents a novel approach to potato disease detection by integrating a Vision Transformer (ViT) model with a Large Language Model (LLM) for enhanced classification of potato plant diseases. We developed a multi-modal pipeline that not only accurately identifies diseases affecting potato leaves and tubers but also provides contextual explanations for the diagnoses. Experimental results demonstrate that our integrated approach outperforms traditional individual models, with the potato leaf disease classifier achieving 99.44% validation accuracy and the potato tuber disease classifier reaching 76.19% accuracy when trained separately, while the combined model maintains excellent performance of 95.06% on the validation set. The fusion of computer vision with Mistral AI's LLM capabilities creates an interpretable system that can assist agricultural experts with both disease identification and recommended treatment actions. This paper contributes to the growing field of AI-assisted agriculture by demonstrating how multi-modal deep learning systems can provide more comprehensive solutions to potato disease management challenges, potentially reducing crop losses and improving food security.