DepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenes
Loading...
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
We present DepthLight, a method to estimate spatial lighting for photorealistic Visual Effects (VFX) using a single image as input. Previous techniques rely either on estimated or captured light representations that fail to account for localized lighting effects, or use simplified lights that do not fully capture the complexity of the illumination process. DepthLight addresses these limitations by using a single LDR image with a limited field of view (LFOV) as an input to compute an emissive texture mesh around the image (a mesh which generates spatial lighting in the scene), producing a simple and lightweight 3D representation for photorealistic object relighting. First, an LDR panorama is generated around the input image using a photorealistic diffusion-based inpainting technique, conditioned on the input image. An LDR to HDR network then reconstructs the full HDR panorama, while an off-the-shelf depth estimation technique generates a mesh representation to finally build a 3D emissive mesh. This emissive mesh approximates the bidirectional light interactions between the scene and the virtual objects that is used to relight virtual objects placed in the scene. We also exploit this mesh to cast shadows from the virtual objects on the emissive mesh, and add these shadows to the original LDR image. This flexible pipeline can be easily integrated into different VFX production workflows. In our experiments, DepthLight shows that virtual objects are seamlessly integrated into real scenes with a visually plausible estimation of the lighting.We compared our results to the ground truth lighting using Unreal Engine, as well as to state-of-the-art approaches that use pure HDRi lighting techniques (see Figure 1). Finally, we validated our approach conducting a user evaluation over 52 participants as well as a comparison to existing techniques.
Description
CCS Concepts: Computing methodologies → Computational photography; Image manipulation; Mixed / augmented reality; Computer graphics
@inproceedings{10.2312:exw.20251060,
booktitle = {ACM/EG Expressive Symposium - WICED: Eurographics Workshop on Intelligent Cinematography and Editing},
editor = {Catalano, Chiara Eva and Parakkat, Amal Dev},
title = {{DepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenes}},
author = {Manus, Raphael and Christie, Marc and Boivin, Samuel and Guehl, Pascal},
year = {2025},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-271-4},
DOI = {10.2312/exw.20251060}
}