DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mapping

Simple item page
dc.contributor.authorCai, Zeyuen_US
dc.contributor.authorWang, Duotunen_US
dc.contributor.authorLiang, Yixunen_US
dc.contributor.authorShao, Zhijingen_US
dc.contributor.authorChen, Ying-Congen_US
dc.contributor.authorZhan, Xiaohangen_US
dc.contributor.authorWang, Zeyuen_US
dc.contributor.editorChen, Renjieen_US
dc.contributor.editorRitschel, Tobiasen_US
dc.contributor.editorWhiting, Emilyen_US
dc.date.accessioned2024-10-13T18:05:13Z
dc.date.available2024-10-13T18:05:13Z
dc.date.issued2024
dc.description.abstractScore Distillation Sampling (SDS) has emerged as a prevalent technique for text-to-3D generation, enabling 3D content creation by distilling view-dependent information from text-to-2D guidance. However, they frequently exhibit shortcomings such as over-saturated color and excess smoothness. In this paper, we conduct a thorough analysis of SDS and refine its formulation, finding that the core design is to model the distribution of rendered images. Following this insight, we introduce a novel strategy called Variational Distribution Mapping (VDM), which expedites the distribution modeling process by regarding the rendered images as instances of degradation from diffusion-based generation. This special design enables the efficient training of variational distribution by skipping the calculations of the Jacobians in the diffusion U-Net. We also introduce timestep-dependent Distribution Coefficient Annealing (DCA) to further improve distilling precision. Leveraging VDM and DCA, we use Gaussian Splatting as the 3D representation and build a text-to-3D generation framework. Extensive experiments and evaluations demonstrate the capability of VDM and DCA to generate high-fidelity and realistic assets with optimization efficiency.en_US
dc.description.sectionheaders3D Modeling and Editing
dc.description.seriesinformationPacific Graphics Conference Papers and Posters
dc.identifier.doi10.2312/pg.20241311
dc.identifier.isbn978-3-03868-250-9
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.2312/pg.20241311
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/pg20241311
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Image manipulation; Shape modeling
dc.subjectComputing methodologies → Image manipulation
dc.subjectShape modeling
dc.titleDreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mappingen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
pg20241311.pdf
Size:
4.72 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
dreammapping_demo.mp4
Size:
124.67 MB
Format:
Video MP4
Download
Thumbnail Image
Name:
_pg2024_separate_supplemental_materials.pdf
Size:
2.86 MB
Format:
Adobe Portable Document Format
Download
Collections
PG2024 Conference Papers and Posters