Image inpainting involves filling missing areas of a corrupted image. Despite impressive results have been achieved recently, restoring images with both vivid textures and reasonable structures remains a significant challenge. Previous methods have primarily addressed regular textures while disregarding holistic structures due to the limited receptive fields of Convolutional Neural Networks (CNNs). To this end, we study learning a Zero-initialized residual addition based Incremental Transformer on Structural priors (ZITS++), an improved model upon our conference work, ZITS~\cite{dong2022incremental}. Specifically, given one corrupt image, we present the Transformer Structure Restorer (TSR) module to restore holistic structural priors at low image resolution, which are further upsampled by Simple Structure Upsampler (SSU) module to higher image resolution. To recover image texture details, we use the Fourier CNN Texture Restoration (FTR) module, which is strengthened by Fourier and large-kernel attention convolutions. Furthermore, to enhance the FTR, the upsampled structural priors from TSR are further processed by Structure Feature Encoder (SFE) and optimized with the Zero-initialized Residual Addition (ZeroRA) incrementally. Besides, a new masking positional encoding is proposed to encode the large irregular masks. Compared with ZITS, ZITS++ improves the FTR's stability and inpainting ability with several techniques. More importantly, we comprehensively explore the effects of various image priors for inpainting and investigate how to utilize them to address high-resolution image inpainting with extensive experiments. This investigation is orthogonal to most inpainting approaches and can thus significantly benefit the community. Codes and models will be released in https://github.com/DQiaole/ZITS_inpainting.

翻译：尽管最近取得了令人印象深刻的成果,但以生动的纹理和合理的结构恢复图像仍然是一项重大挑战。以前的方法主要解决了常规质地,而忽视了整体结构,因为革命神经网络(CNNs)的可接受领域有限。为此,我们研究如何在结构前科上学习一个零初始剩余添加基于结构前科的递增变异变异器(ZITS++),这是我们会议工作改进的模型,ZITS ⁇ cite{tlo2022increment}。具体来说,鉴于一个腐败的图像,我们展示了变异结构恢复机(TSR)模块,以低图像分辨率恢复整体结构前科,而由于简单的结构上层神经网络(SSUpsampler)模块进一步更新了整体结构结构结构结构结构。为了恢复图像细节细节,我们使用FourierCNN Texturereture Refure Reformation(FIT+TR)模块,该模块的强化了FITRTR, 其上版结构前版结构更新了OILMLS-SBIal 和SIMBIDRIDRIL 的升级能力,因此将进一步在SFSFSFSDRDSBSDRDRBDRBSBSBSBSBSBDBS 上进行进一步升级的升级的升级的升级和升级。