Proposal for an Automatic Recall Assistance System Using Face Recognition, Voice Recognition, and Conversational Content

We experience various things in our daily life, and our brain stores the information obtained through these experiences, but it is difficult to keep it for a long period of time. Therefore, we propose a recall assistance system, which uses lifelogs. Related studies in memory recall have examined the usefulness of using only images captured by cameras or combining images with other elements for recall. However, there is an issue that past information cannot be easily recalled. The purpose of this research is to develop a system that can extract and present information about memories and past events based on multiple pieces of information to the user in order to realize a recall assistance system that utilizes lifelogs. We propose a system that facilitates the recall of past memories by pointing a camera at the person with whom the user is conversing. We conducted experiments in which we used face recognition, voice recognition, and language processing to identify the three types of information needed to assist recall: face, voice, and conversational content, and used this information to extract past events from a set of data. The results showed that the combination of the three types of information can extract highly relevant past events. In addition, the addition of various information other than the information used in this study is expected to increase the versatility and reproducibility of the method.