LUIVITON: Learned Universal Interoperable Virtual Try-ON

LUIVITON transfers complex real-world garments onto diverse humanoid characters through learned correspondences, SMPL-based registration, and simulation-guided fitting. The system supports varied garment topology and stylized target bodies without requiring shared rigs or manual correspondences.

Overview Video

LUIVITON Project Video

Abstract

To enable large-scale reuse of real-world 3D assets-where garments and characters rarely share skeletons, templates, or dense correspondences-we present a fully automated virtual try-on system that dresses complex, multi-layer garments onto diverse, arbitrarily posed humanoids. Our key idea is to use SMPL as an intermediate proxy and decompose clothing-to-body transfer into two correspondence tasks with distinct challenges: (1) clothing-to-SMPL (partial-to-complete alignment) and (2) body-to-SMPL (large pose/shape variation and stylization). We address clothing-to-SMPL using a geometry-driven correspondence model, and introduce a diffusion-based body-to-SMPL correspondence approach that leverages multi-view consistent appearance features together with a pretrained 2D foundation model. Using these correspondences, we register SMPL/SMPL+D (Displacement) to the garment and target body and then perform simulator-driven fitting by transferring the garment along a smooth SMPL-to-SMPL+D transition, producing physically plausible draping on the target. Our system handles complex garment topology (including non-manifold meshes) and generalizes to a wide range of humanoid characters (e.g., humans, robots, cartoons, and creatures) while remaining computationally practical. Upon draping, our system also supports fast customization of clothing size. We show that our system can produce high-quality 3D clothing fittings without any human labor, even when 2D clothing sewing patterns are not available.

Method Overview

Our system is a fully automated virtual try-on system that dresses arbitrary 3D garments onto arbitrarily posed humanoid characters without requiring a dressed source avatar, skeleton rigs, or predefined correspondences. Given a rest-posed garment mesh and a target humanoid body, we use SMPL as an intermediate proxy and decompose the dressing task into correspondence prediction, registration, and simulation-based fitting.

First, we estimate dense correspondences in two complementary branches. For clothing-to-body alignment, we predict clothing-to-SMPL correspondences using a DiffusionNet-based surface network that learns partial-to-complete mappings in SMPL UV space, enabling robust handling of complex, multilayer, and non-manifold garments. In parallel, we predict body-to-SMPL correspondences using a diffusion-based correspondence predictor that leverages multi-view consistent diffusion features and semantic cues from a pretrained vision foundation model, allowing robust alignment across large pose variation and stylized body shapes.

Next, we perform two registration steps guided by these correspondences. We first register SMPL inside the garment to obtain a clothing-aligned proxy, and then register an extended SMPL+D model to the target body using additional per-vertex displacements and anisotropic scaling to accommodate extreme proportions.

Finally, we interpolate shape and pose between the registered SMPL and SMPL+D representations to construct a smooth transition sequence, which drives a neural cloth simulator to transfer the garment onto the target body. This staged transition avoids unstable collisions and produces physically plausible draping while preserving garment structure. The resulting pipeline generalizes to unseen garments and bodies and supports automatic, high-quality 3D virtual try-on without human intervention.

3D Viewer of our results

Drag to rotate • Scroll to zoom

Loading 3D model...

Comparisons with Expert

We compare LUIVITON outputs with expert artist results on three representative characters. Each row shows the target body, two garment inputs, our fitted results, and the corresponding artist-created references.

body

input garment

ours

expert

input garment

ours

expert

Results Gallery

Drag to rotate • Scroll to zoom

Size Adjustment

Users can modify the garment scale and view different fitting styles, such as tighter or looser clothing.

Animation Gallery

Our system produces a fitted garment mesh in the initial pose of the target character. Once the garment is fitted to the initial posed character, it can serve as a clean starting point for downstream animation or simulation.

Amy

Jody

Mannequin

Mutant

Ortiz

Xbot

Citation

@misc{cao2026luivitonlearneduniversalinteroperable,
      title={LUIVITON: Learned Universal Interoperable VIrtual Try-ON}, 
      author={Cong Cao and Xianhang Cheng and Jingyuan Liu and Yujian Zheng and Zhenhui Lin and Ren Li and Meriem Chkir and Hao Li},
      year={2026},
      eprint={2509.05030},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.05030}, 
}