Complementarity-Aware Local–Global Feature Fusion Network for Building Extraction in Remote Sensing Images

Building extraction is a challenging research direction in remote sensing image (RSI) interpretation. Due to the fact that a building has not only its own local structures but also similar architectural styles with other buildings located in a global area (e.g., street or community), fusing local and global features becomes a promising way to improve performance of building extraction. Focused on this, we propose a new complementarity-aware local-global feature fusion network (CLGFF-Net) by integrating a convolutional branch and a Transformer branch. The two branches respectively capture local patterns and global long-range dependencies of RSIs, thereby leading to highly complementary features. To dig out the implicit complementary information for fusion, we develop a complementarity-aware fusion module (CFM) which separates shared features (SFs) and distinct features (DFs) between two branches, by building a commonalities analysis path and two difference analysis paths. Meanwhile, to make sure the similarity of SFs and dissimilarity of DFs, a triplet loss function is designed to enforce the distances between SFs to be near and DFs to be far. By this way, complementary information can be explicitly included in DFs and is adaptively exchanged between two branches for fusion. Besides, since multilayer features in each branch generally convey different-level semantic information, a multi-layer fusion scheme (MLFS) is designed to fuse them by introducing cross-layer connections and gate mechanism. By coupling CFMs with MLFS, the abilities in characterizing local and global context information, as well as different-level semantic information, can be fully exploited for better mapping of complicated building objects. Experimental results demonstrate the effectiveness of our proposed method.