Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image. Even though recent work on GANs enables synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches. We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion. We propose a novel latent space for image blending which is better at preserving detail and encoding spatial information, and propose a new GAN-embedding algorithm which is able to slightly modify images to conform to a common segmentation mask. Our novel representation enables the transfer of the visual properties from multiple reference images including specific details such as moles and wrinkles, and because we do image blending in a latent-space we are able to synthesize images that are coherent. Our approach avoids blending artifacts present in other approaches and finds a globally consistent image. Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 percent of the time.
翻译:由于照明、几何和部分封隔关系复杂,造成图像不同部分的相联性,多图像的不缝混合特征极具挑战性。尽管最近关于GANs的工作能够将现实的头发或面孔合成,但是仍然难以将它们合并成单一、连贯和可信的图像,而不是不相交的图像补补补。我们提出了一个图像混合的新解决方案,特别是基于GAN-inversion的发型传换问题。我们提出了图像混合的新颖的潜在空间,这能更好地保存细节和编码空间信息,并提出了新的GAN组装算法,能够略微修改图像以符合共同的分割面罩。我们的新表述使得图像从多个参考图像中转换成视觉属性,包括摩尔和皱纹等具体细节,因为我们在潜层空间进行图像混合,我们可以合成一致的图像。我们的方法避免将其他方法中的文物混合,并找到一个全球一致的图像。我们的结果显示,在用户研究中,对当前艺术状态有显著改进,用户更喜欢混合95 %的解决方案。