Remote sensing (RS) change detection is a technique for identifying changes in the ground surface by comparing RS images from different periods. Although such tasks have been developed for a long time and some methods have been proposed to enhance the features of real changes in the ground objects, they still face the perception ambiguity caused by the heterogeneity of the ground objects in complex RS environments: 1) insufficient processing of nonstationary changes between dual-temporal image features and 2) high spatial heterogeneity leads to difficulties in structural identification. In order to solve the interference of these problems on the downstream tasks of change detection, this article proposes geo-spatial structural refinement network (GSSR-Net) for RS change detection. First, we introduce a DualTime Mamba structure with an omnidirectional scanning path, adjust its input matrix between dual-temporal image features in the deep scale, and allow the model to fully consider the spatiotemporal dependence and image structure information of the previous and next time points. In addition, this article designs a land-cover feature extraction (LCFE) method to improve the perception ability of the ground object target structure. Specifically, this method refines the image edge by separating high frequencies, adjusts the contour structure information of the dual-phase image by separating low frequencies, and then further models the relationship between pixels by combining feature structures and spatial offset mechanisms. Our experimental results on three datasets demonstrate the superiority of GSSR-Net. The network code address is https://github.com/SparrowTought/GSSR-Net.