TSDN: Transport-based Stylization for Dynamic NeRF
| dc.contributor.author | Gong, Yuning | en_US |
| dc.contributor.author | Song, Mingqing | en_US |
| dc.contributor.author | Ren, Xiaohua | en_US |
| dc.contributor.author | Liao, Yuanjun | en_US |
| dc.contributor.author | Zhang, Yanci | en_US |
| dc.contributor.editor | Chen, Renjie | en_US |
| dc.contributor.editor | Ritschel, Tobias | en_US |
| dc.contributor.editor | Whiting, Emily | en_US |
| dc.date.accessioned | 2024-10-13T18:04:35Z | |
| dc.date.available | 2024-10-13T18:04:35Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | While previous Neural Radiance Fields (NeRF) stylization methods achieve visually appealing results on transferring color style for static NeRF scenes, they lack the ability to stylize dynamic NeRF scenes with geometrically stylized features (like brushstrokes or feature elements from artists' works), which is also important for style transfer. However, directly stylizing each frame of dynamic NeRF independently with geometrically stylized features would lead to flickering results due to bad feature alignment. To overcome these problems, in this paper, we propose Transport-based Stylization for Dynamic NeRF (TSDN), a new dynamic NeRF stylization method that is able to stylize geometric features and align them with the motion in the scene. TSDN utilizes stylization guiding velocity fields to advect dynamic NeRF to get stylized results and then transfers these velocity fields between frames to maintain feature alignment. Also, to deal with the noisy stylized results due to the ambiguity of the deformation field, we propose a feature advection scheme and a novel regularization function specified for dynamic NeRF. The experiment results show that our method has the ability to stylize dynamic scenes with detailed geometrically stylized features from videos or multi-view image inputs, while preserving the original color style if desired. This capability is not present in previous video stylization methods. | en_US |
| dc.description.sectionheaders | Image Processing and Filtering I | |
| dc.description.seriesinformation | Pacific Graphics Conference Papers and Posters | |
| dc.identifier.doi | 10.2312/pg.20241300 | |
| dc.identifier.isbn | 978-3-03868-250-9 | |
| dc.identifier.pages | 12 pages | |
| dc.identifier.uri | https://doi.org/10.2312/pg.20241300 | |
| dc.identifier.uri | https://diglib.eg.org/handle/10.2312/pg20241300 | |
| dc.publisher | The Eurographics Association | en_US |
| dc.rights | Attribution 4.0 International License | |
| dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | |
| dc.subject | CCS Concepts: Computing methodologies → Image processing; Volumetric models; Shape representations | |
| dc.subject | Computing methodologies → Image processing | |
| dc.subject | Volumetric models | |
| dc.subject | Shape representations | |
| dc.title | TSDN: Transport-based Stylization for Dynamic NeRF | en_US |