Text-Guided 3D Object Generation via Disentangled Shape and Appearance Score Distillation Sampling

Previous text-guided 3D generative models primarily utilize CLIP-based semantic constraints or employ score distillation sampling from pre-trained 2D diffusion models to provide prior knowledge for neural radiance field optimization. However, these methods lack incorporation of 3D prior knowledge, and are prone to generate poorly structured 3D objects, particularly when the text prompts are not specific. In this paper, we exploit 3D shape prior information from text-guided shape generative models and propose a novel Double Score Distillation Sampling method (Double SDS). Compared to the previous methods that solely use 2D diffusion models in color space, our proposed method leverages both the text-to-shape and text-to-image diffusion models to optimize the disentangled color and density of the neural radiance field, respectively. Additionally, we employ the Low-Rank Adaptation method to fine-tune the pre-trained diffusion models, aiming to enhance the similarity between the generated 3D objects and the target 3D object datasets. Experimental results demonstrate that our proposed method can generate 3D objects with higher visual quality and better geometric structure compared to previous methods.