We present Lighting in Motion (LiMo), a diffusion-based approach to spatiotemporal lighting estimation. LiMo targets both realistic high-frequency detail prediction and accurate illuminance estimation. To account for both, we propose generating a set of mirrored and diffuse spheres at different exposures, based on their 3D positions in the input. Making use of diffusion priors, we fine-tune powerful existing diffusion models on a large-scale customized dataset of indoor and outdoor scenes, paired with spatiotemporal light probes. For accurate spatial conditioning, we demonstrate that depth alone is insufficient and we introduce a new geometric condition to provide the relative position of the scene to the target 3D position. Finally, we combine diffuse and mirror predictions at different exposures into a single HDRI map leveraging differentiable rendering. We thoroughly evaluate our method and design choices to establish LiMo as state-of-the-art for both spatial control and prediction accuracy.

翻译：本文提出动态光照（LiMo），一种基于扩散模型的时空光照估计方法。LiMo旨在同时实现真实的高频细节预测与精确的照度估计。为此，我们提出根据输入场景中三维位置生成一组不同曝光下的镜面与漫反射球体。利用扩散先验，我们在大规模定制化的室内外场景数据集上对现有强效扩散模型进行微调，该数据集包含配对的时空光照探针。为实现精确的空间条件控制，我们论证了仅依赖深度信息不足，并引入一种新的几何条件以提供场景相对于目标三维位置的相对方位。最后，我们通过可微分渲染技术将不同曝光下的漫反射与镜面预测融合为单一高动态范围图像（HDRI）图。我们通过全面评估方法及设计选择，确立LiMo在空间控制与预测精度方面均达到业界领先水平。