Knowledge Collector: Towards Safer Operations of Near-Miss Incidents with Multimodal Data Analysis

This paper presents a practical case of AI in the gas industry, where AI systems can assist onsite and offsite workers in making near-miss incident reports effectively by using their voice. To facilitate a proof-of-concept for this, we extend HPGIncident, a dataset for high-pressure gas incidents introduced by [1], by simulating audio in a manufacturing context with the belief that the fine-tuned models can adapt well in a noisy environment. Experimental results on the three tasks, including ASR, NER, and cause-effect extraction, show that the system can be beneficial for effectively making near-miss reports. Moreover, we also introduce a prototype called Knowledge Collector, which allows users to input incidents verbally, receive extracted results, and manage reports for future safety operations. Feedback from prospective clients suggests the prototype has strong potential for business usage. The demonstration video for this prototype is at https://youtu.be/agqcFzzLmp0.