Fire and smoke in enclosed building environments cause severe visual degradation that impairs the effectiveness of intelligent monitoring and emergency management systems. Dense and dynamically diffused smoke leads to strong non-uniform occlusion, structural corrosion, and multi-scale interference, making reliable perception in building fire emergencies highly challenging. To address these challenges, we propose a structure-detail aware network (SDANet), composed of three novel architectural blocks. First, the smoke feature adaptive extraction (SFAE) module constructs channel-specific spatial attention maps to achieve joint channel–spatial modeling, enhancing smoke representation in highly non-uniform regions. Second, the multi-resolution frequency domain enhancement (MFDE) module preserves high-frequency edge and texture information during downsampling and reinforces global semantic consistency. Third, the cross-scale non-uniform perception fusion (CNPF) module allocates adaptive fusion weights to shallow and deep features based on regional context, alleviating semantic conflicts between local occlusion and global diffusion. To validate the proposed method, we construct a novel enclosed building spaces fire smoke image restoration dataset for training and evaluation. Experimental results show that SDANet outperforms seven state-of-the-art methods, with PSNR improvements between 4.57 and 14.79 dB, and SSIM gains between 0.07 and 0.46. Its stability, smoke removal quality, efficiency, and data dependency prove its capabilities in structure and detail recovery and cross-scene robustness. An explainability analysis using SHAP quantitatively attributes each module's contribution, enhancing the network's interpretability and theoretical completeness.