High-Fidelity Dynamic Human Synthesis via UV-Guided NeRF with Sparse Views

In the field of dynamic human synthesis, some recent works try to decompose a non-rigidly deforming scene into a canonical neural radiance field and use a set of deformation fields for mapping observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images. Due to the highly under-constrained optimization cased by deformation field without prior and the insufficient of surface appearance information cased by sparse views, the rendering result exists obvious appearance artifacts. In this paper, to address the problem of artifacts, we present a novel method called UV-guided Neural Radiance Fields (UVNeRF), consisting of three modules: Canonical Space Mapping Module (CSMM), Texture Space Mapping Module (TSMM), UV-guided Neural Rendering Module (UVNRM). CSMM map observation-space points to the canonical space based 3D human skeletons which can regularize learning of the deformation field. TSMM map canonical-space points to the texture space for obtaining a rough human surface representation on the UV space as the extra information. UVNRM render the image result using the outputs of CSMM and TSMM. The experimental studies on Human3.6M and ZJU-MoCap dataset show that our approach gains noteworthy enhancements comparing recent dynamic human synthesis methods.