PG‐MVSNet: Plane‐Guided Multi‐View Stereo Network With Dual‐Depth Estimation

ABSTRACT Recent learning‐based multi‐view stereo (MVS) methods employ a coarse‐to‐fine strategy to efficiently estimate high‐resolution depth maps. However, achieving complete 3D reconstructions remains a significant challenge due to unreliable feature matching and cost regularization in low‐textured areas. As these regions often exhibit strong planarity, we introduce PG‐MVSNet, a plane‐guided dual‐depth estimation network designed to predict more reliable depth in low‐textured areas. Our network comprises two complementary branches: a conventional coarse‐to‐fine MVS branch for depth estimation in richly textured regions and a plane‐guided enhancement branch for low‐textured areas. In the enhancement branch, plane priors are first constructed by leveraging high‐confidence depth points, which are derived through a non‐parametric support point selection method. Subsequently, erroneous planes are filtered using a normal‐based plane filter, after which a plane‐guided depth map is generated and refined. Finally, the estimated dual depths are filtered by a selective filtering mechanism, resulting in a complete 3D model while maintaining high reconstruction accuracy. As a single accurate depth estimate per pixel from the dual‐depth predictions is sufficient for generating a complete reconstruction model, we leverage this insight to propose a novel focus loss for efficient network training. Our method achieves competitive overall performance on the DTU, Tanks and Temples, and ETH3D datasets. Specifically, on the Tanks and Temples dataset, our method achieves a top‐tier F ‐score of 66.52 on the intermediate set and 41.86 on the advance set, highlighting the effectiveness of the proposed method in reconstructing high‐quality 3D models.