DepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenes

Simple item page
dc.contributor.authorManus, Raphaelen_US
dc.contributor.authorChristie, Marcen_US
dc.contributor.authorBoivin, Samuelen_US
dc.contributor.authorGuehl, Pascalen_US
dc.contributor.editorCatalano, Chiara Evaen_US
dc.contributor.editorParakkat, Amal Deven_US
dc.date.accessioned2025-05-09T09:43:24Z
dc.date.available2025-05-09T09:43:24Z
dc.date.issued2025
dc.description.abstractWe present DepthLight, a method to estimate spatial lighting for photorealistic Visual Effects (VFX) using a single image as input. Previous techniques rely either on estimated or captured light representations that fail to account for localized lighting effects, or use simplified lights that do not fully capture the complexity of the illumination process. DepthLight addresses these limitations by using a single LDR image with a limited field of view (LFOV) as an input to compute an emissive texture mesh around the image (a mesh which generates spatial lighting in the scene), producing a simple and lightweight 3D representation for photorealistic object relighting. First, an LDR panorama is generated around the input image using a photorealistic diffusion-based inpainting technique, conditioned on the input image. An LDR to HDR network then reconstructs the full HDR panorama, while an off-the-shelf depth estimation technique generates a mesh representation to finally build a 3D emissive mesh. This emissive mesh approximates the bidirectional light interactions between the scene and the virtual objects that is used to relight virtual objects placed in the scene. We also exploit this mesh to cast shadows from the virtual objects on the emissive mesh, and add these shadows to the original LDR image. This flexible pipeline can be easily integrated into different VFX production workflows. In our experiments, DepthLight shows that virtual objects are seamlessly integrated into real scenes with a visually plausible estimation of the lighting.We compared our results to the ground truth lighting using Unreal Engine, as well as to state-of-the-art approaches that use pure HDRi lighting techniques (see Figure 1). Finally, we validated our approach conducting a user evaluation over 52 participants as well as a comparison to existing techniques.en_US
dc.description.sectionheadersImmersive Rendering
dc.description.seriesinformationACM/EG Expressive Symposium - WICED: Eurographics Workshop on Intelligent Cinematography and Editing
dc.identifier.doi10.2312/exw.20251060
dc.identifier.isbn978-3-03868-271-4
dc.identifier.pages11 pages
dc.identifier.urihttps://doi.org/10.2312/exw.20251060
dc.identifier.urihttps://diglib.eg.org/handle/10.2312/exw20251060
dc.publisherThe Eurographics Associationen_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Computational photography; Image manipulation; Mixed / augmented reality; Computer graphics
dc.subjectComputing methodologies → Computational photography
dc.subjectImage manipulation
dc.subjectMixed / augmented reality
dc.subjectComputer graphics
dc.titleDepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenesen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
exw20251060.pdf
Size:
18.9 MB
Format:
Adobe Portable Document Format
Download
Collections
Expressive + WICED 2025