Poster Abstract: Representation Learning from Multimodal Sensor Data with Maximally Correlated Autoencoders

With the development of sensing technology, multiple sensors are widely used in Internet of Things (IoT) devices. A key challenge is to learn feature representations from multimodal sensor data to combine the information of different sensors. Although progress has been made by previous works, the correlation between different sensors is still not well exploited, which may limit the performance of representation learning. To address this problem, we propose a deep learning approach to learn representations from multimodal sensor data with maximally correlated autoencoders (MCA). It can efficiently capture high dependence between different modalities at different feature levels. The learned representations are further used for the recognition task. Experimental results on the real-world RGB- D dataset demonstrate the high effectiveness of MCA.