In this paper, we propose an approach to perform novel view synthesis and depth estimation via dense 3D reconstruction from a single image. Our NeMI unifies Neural radiance fields (NeRF) with Multiplane Images (MPI). Specifically, our NeMI is a general two-dimensional and image-conditioned extension of NeRF, and a continuous depth generalization of MPI. Given a single image as input, our method predicts a 4-channel image (RGB and volume density) at arbitrary depth values to jointly reconstruct the camera frustum and fill in occluded contents. The reconstructed and inpainted frustum can then be easily rendered into novel RGB or depth views using differentiable rendering. Extensive experiments on RealEstate10K, KITTI and Flowers Light Fields show that our NeMI outperforms state-of-the-art by a large margin in novel view synthesis. We also achieve competitive results in depth estimation on iBims-1 and NYU-v2 without annotated depth supervision. Project page available at https://vincentfung13.github.io/projects/nemi/

翻译：在本文中,我们提出一种方法,通过从一个图像中进行密度3D重建来进行新颖的视图合成和深度估计。我们的NeMI用多平面图像(MPI)将神经分光场(NeRF)统一成新的RGB或深度视图。具体地说,我们的NeMI是NeRF一般的二维和有图像条件的扩展,并且对MPI进行持续的深度概括。鉴于一个作为投入的单一图像,我们的方法预测了一种4通道图像(RGB和体积密度),其任意深度值,以联合重建相机的横梁和填充隐蔽的内容。然后,利用不同的图像,将重建的和有油漆的断裂状的RGB或深度视图很容易转换成新的RGB或深度视图。关于RealEstate10K、KITTI和花岗光场的广泛实验表明,我们的NEMMI以新的合成大边距优于最新状态。我们还在iBims-1和NYU-V2上取得了竞争性的深度估计结果,而没有附加的深度监督。在http://vincentfung13.githubub/nimub/ must/ must上的项目页面上可以查阅。