Development and implementation of a LabVIEW-based virtual instrumentation system for enhancing communication with hearing impaired and visually impaired persons

People without sensory impairments may find it challenging to communicate effectively with those who have hearing or visual impairments. Further, hearing-impaired and visually impaired people are not interested in maintaining interpersonal relationships with others due to the difficulty in communicating. As a result, considerable research has been conducted to find an innovative way that make the process of communication easier for them. To overcome the weaknesses and shortcomings of these studies, the authors introduce a unique application. Our application model focuses on developing a virtual instrument that recognises hand gestures, displays a corresponding message, and provides a voice message to assist people with hearing or visual impairments in communication. A flex sensor glove was used for hand gesture detection, and a virtual instrumentation system was developed using LabVIEW software. Ten volunteers with no previous disabilities were initially used to test the system. Seven gestures were selected for the initial evaluation, and each gesture was performed five times, completing a total of 350 trials. The selected seven hand gestures for the initial evaluation were successfully communicated through both audio and visual means by providing them in all three languages: Sinhala, Tamil, and English. Furthermore, our system achieved a gesture detection accuracy of 95.42%, with an average response time of less than 0.5 s per gesture. This system is highly beneficial for people with congenital or early-stage hearing loss and visually impaired people who are confined to beds due to illness.