Loading 3D model...
Overview Video
Abstract
To enable large-scale reuse of real-world 3D assets-where garments and characters rarely share skeletons, templates, or dense correspondences-we present a fully automated virtual try-on system that dresses complex, multi-layer garments onto diverse, arbitrarily posed humanoids. Our key idea is to use SMPL as an intermediate proxy and decompose clothing-to-body transfer into two correspondence tasks with distinct challenges: (1) clothing-to-SMPL (partial-to-complete alignment) and (2) body-to-SMPL (large pose/shape variation and stylization). We address clothing-to-SMPL using a geometry-driven correspondence model, and introduce a diffusion-based body-to-SMPL correspondence approach that leverages multi-view consistent appearance features together with a pretrained 2D foundation model. Using these correspondences, we register SMPL/SMPL+D (Displacement) to the garment and target body and then perform simulator-driven fitting by transferring the garment along a smooth SMPL-to-SMPL+D transition, producing physically plausible draping on the target. Our system handles complex garment topology (including non-manifold meshes) and generalizes to a wide range of humanoid characters (e.g., humans, robots, cartoons, and creatures) while remaining computationally practical. Upon draping, our system also supports fast customization of clothing size. We show that our system
can produce high-quality 3D clothing fittings without any human labor,
even when 2D clothing sewing patterns are not available.
Method Overview
Our system is a fully automated virtual try-on system that dresses arbitrary 3D garments onto arbitrarily posed humanoid characters without requiring a dressed source avatar, skeleton rigs, or predefined correspondences. Given a rest-posed garment mesh and a target humanoid body, we use SMPL as an intermediate proxy and decompose the dressing task into correspondence prediction, registration, and simulation-based fitting.
First, we estimate dense correspondences in two complementary branches. For clothing-to-body alignment, we predict clothing-to-SMPL correspondences using a DiffusionNet-based surface network that learns partial-to-complete mappings in SMPL UV space, enabling robust handling of complex, multilayer, and non-manifold garments. In parallel, we predict body-to-SMPL correspondences using a diffusion-based correspondence predictor that leverages multi-view consistent diffusion features and semantic cues from a pretrained vision foundation model, allowing robust alignment across large pose variation and stylized body shapes.
Next, we perform two registration steps guided by these correspondences. We first register SMPL inside the garment to obtain a clothing-aligned proxy, and then register an extended SMPL+D model to the target body using additional per-vertex displacements and anisotropic scaling to accommodate extreme proportions.
Finally, we interpolate shape and pose between the registered SMPL and SMPL+D representations to construct a smooth transition sequence, which drives a neural cloth simulator to transfer the garment onto the target body. This staged transition avoids unstable collisions and produces physically plausible draping while preserving garment structure. The resulting pipeline generalizes to unseen garments and bodies and supports automatic, high-quality 3D virtual try-on without human intervention.
Interactive 3D Demo
Drag to rotate • Scroll to zoom
Results Gallery
Drag to rotate • Scroll to zoom
Animation Gallery
Amy
Jody
Mannequin
Mutant
Ortiz
Xbot