A Time-Surface Enhancement Model for Event-based Spatiotemporal Feature Extraction

Event-based cameras provide a unique way of visual perception through the event-driven characteristics and representation of sparse spatiotemporal data. Meanwhile, as brain-inspired models, Spiking Neural Networks (SNNs) have asynchronous event processing characteristics and are able to process event data naturally. However, various feature extraction methods currently in SNNs have not fully utilized the temporal information in the output of bionic visual sensors, making it difficult to better maintain Address Event Representations (AER). In this paper, we proposed a method to enhance time surfaces by extracting hidden temporal information in the time surface and multi-scale dilated time surface. The method of extracting hidden temporal information (EHTI) is to enhance the time surface by exploiting the information of time surface changes between the current event and other events in the spatiotemporal neighborhood, allowing the event’s temporal features to be fully utilized. In addition, we adopted the multi-scale dilated time surface (MDTS) method to solve the problem of limited event receptive fields by using dilated time surfaces for multi-scale time surface feature extraction, which can better capture the spatiotemporal connections between events in further neighborhoods. Experimental results on various event-based datasets (i.e., N-MNIST, MNIST-DVS, DVS128 Gesture, and DailyAction-DVS) show that our method is able to extract richer features and outperforms other methods on classification tasks.