PG‐VTON: Front‐And‐Back Garment Guided Panoramic Gaussian Virtual Try‐On With Diffusion Modeling

ABSTRACT Virtual try‐on (VTON) technology enables the rapid creation of realistic try‐on experiences, which makes it highly valuable for the metaverse and e‐commerce. However, 2D VTON methods struggle to convey depth and immersion, while existing 3D methods require multi‐view garment images and face challenges in generating high‐fidelity garment textures. To address the aforementioned limitations, this paper proposes a panoramic Gaussian VTON framework guided solely by front‐and‐back garment information, named PG‐VTON, which uses an adapted local controllable diffusion model for generating virtual dressing effects in specific regions. Specifically, PG‐VTON adopts a coarse‐to‐fine architecture consisting of two stages. The coarse editing stage employs a local controllable diffusion model with a score distillation sampling (SDS) loss to generate coarse garment geometries with high‐level semantics. Meanwhile, the refinement stage applies the same diffusion model with a photometric loss not only to enhance garment details and reduce artifacts but also to correct unwanted noise and distortions introduced during the coarse stage, thereby effectively enhancing realism. To improve training efficiency, we further introduce a dynamic noise scheduling (DNS) strategy, which ensures stable training and high‐fidelity results. Experimental results demonstrate the superiority of our method, which achieves geometrically consistent and highly realistic 3D virtual try‐on generation.