Visual-triggered contextual guidance for lithium battery disassembly: a multi-modal event knowledge graph approach

Lithium battery disassembly is a complex task that presents numerous challenges, including the wide variety of battery types, intricate manufacturing processes, and the absence of standardised procedures. These challenges pose significant obstacles in achieving efficient and accurate disassembly operations. Consequently, there is a clear need for guidance and support throughout the disassembly process. Conventional strategies for providing disassembly guidance often struggle to adapt to the dynamic changes that occur during the process. Therefore, it is essential to develop an approach that offers flexibility, efficiency, and adaptability to real-time visual cues in order to effectively address these challenges. This paper introduces a novel real-time visual guidance scheme that utilises a sophisticated multi-modal event knowledge graph. By leveraging outputs from computer vision models, the proposed scheme monitors the disassembly process in real-time and provides visually triggered, context-sensitive guidance through the multi-modal event knowledge graph. Additionally, this paper presents an auxiliary training method for visual state detection, guided by the large-scale visual model known as the Segment Anything Model (SAM), which helps mitigate the costs associated with data annotation during model development. The efficacy of the proposed framework is validated through experimental evaluations, demonstrating its potential in enhancing disassembly efficiency.