VishGuard: AI-Powered Real-Time Defense Against Voice Phishing

Voice phishing, or “vishing,” has emerged as one of the most deceptive and rapidly growing cyber threats in recent years. Exploiting the psychological trust associated with real-time voice communication, attackers often impersonate legitimate entities-such as banks, government agencies, or tech support-aiming to coerce individuals into revealing sensitive information including personal identification numbers, bank details, and login credentials. As vishing tactics grow more sophisticated through the use of AI-generated voices and spoofed caller IDs, traditional rule-based or static detection systems are proving inadequate. These legacy approaches typically lack real-time capabilities, suffer from high falsepositive rates, and struggle to adapt to evolving fraud patterns. This research proposes a comprehensive, AI-driven voice phishing detection system capable of analyzing calls in real time by leveraging a fusion of state-of-the-art technologies. The core architecture integrates Automatic Speech Recognition (ASR) tools such as Whisper or Wav2Vec for live speech-to-text conversion, enabling immediate linguistic analysis. It employs Natural Language Processing (NLP) techniques using models like BERT to detect deceptive language patterns, urgency, and manipulative cues. Simultaneously, the system extracts acoustic features such as pitch, tone, speech rate, and vocal stress, which are often indicative of deceptive intent. These multimodal features are fed into a hybrid ensemble of machine learning models, including Random Forest, Support Vector Machines (SVM), and deep learning architectures like Long Short-Term Memory (LSTM) networks, to classify the call as legitimate or suspicious. A real-time web-based interface allows for live monitoring of conversations between two entities, while dynamically updating the phishing probability score based on ongoing interactions. The system further enhances user trust by offering explainable AI outputs, providing reasons behind each classification decision. Evaluation on a curated dataset demonstrates a detection accuracy exceeding 98 %, with strong performance metrics across precision, recall, and latency. The system is scalable, user-friendly, and well-suited for integration into modern communication infrastructures including VoIP services.