Archive translation. Research status, specifications and availability reflect the original publication date in 2021. Verify current implementations before production use.
In brief: Zian Wang, Jonah Philion, Sanja Fidler and Jan Kautz of NVIDIA and the University of Toronto presented a neural method for inserting and lighting 3D objects inside 2D images.
Method: Inspired by volume-rendering techniques, the researchers developed a volumetric spherical-Gaussian lighting representation. Rays from the 3D object were parameterised on a voxel grid. The method was designed to produce physically plausible lighting without requiring captured HDR environment data.
The team stated that reflections on inserted objects adapted continuously to the surrounding environment. This describes the researchers’ reported result and was not independently benchmarked here.
The paper and project material were published on the NVIDIA research project page.
Related Digital Production research coverage includes video-to-video synthesis and water simulation.
Evaluate training data, failure cases, colour consistency and temporal stability before production use.
