Joint Shared-and-Specific Information for Deep Multi-View Clustering

Multi-view data describes an image sample with different modalities of features, thus provides a more comprehensive description of data. Its three basic characteristics, i.e ., consensus, complementary and redundancy, determine its performances in computer vision tasks. In this paper, we effectively exploit the above three characteristics to propose a deep learning scheme with joint shared-and-specific information (JSSI) for multi-view clustering. Aiming at facilitating the consensus, JSSI extracts shared information of multi-view data via an adversarial similarity constraint, which is realized by classification and discrimination interactions. Aiming at reducing the redundancy, JSSI separate out view-specific features and prevent them from interfering with the shared features via a difference constraint. Aiming at ensuring the complementary, JSSI aligns the shared features and then concatenates them with the specific features. We examine the effectiveness of JSSI with multi-view clustering on real-world datasets, such as faces and indoor scenes. Extensive experiments and comparisons show that JSSI outperforms other state-of-the-art methods in most of these datasets.