• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

Face inpainting via Learnable Structure Knowledge of Fusion Network

Vol. 16, No. 3, March 31, 2022
10.3837/tiis.2022.03.007, Download Paper (Free):

Abstract

With the development of deep learning, face inpainting has been significantly enhanced in the past few years. Although image inpainting framework integrated with generative adversarial network or attention mechanism enhanced the semantic understanding among facial components, the issues of reconstruction on corrupted regions are still worthy to explore, such as blurred edge structure, excessive smoothness, unreasonable semantic understanding and visual artifacts, etc. To address these issues, we propose a Learnable Structure Knowledge of Fusion Network (LSK-FNet), which learns a prior knowledge by edge generation network for image inpainting. The architecture involves two steps: Firstly, structure information obtained by edge generation network is used as the prior knowledge for face inpainting network. Secondly, both the generated prior knowledge and the incomplete image are fed into the face inpainting network together to get the fusion information. To improve the accuracy of inpainting, both of gated convolution and region normalization are applied in our proposed model. We evaluate our LSK-FNet qualitatively and quantitatively on the CelebA-HQ dataset. The experimental results demonstrate that the edge structure and details of facial images can be improved by using LSK-FNet. Our model surpasses the compared models on L1, PSNR and SSIM metrics. When the masked region is less than 20%, L1 loss reduce by more than 4.3%.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
Y. Yang, S. Liu, B. Xing, K. Li, "Face inpainting via Learnable Structure Knowledge of Fusion Network," KSII Transactions on Internet and Information Systems, vol. 16, no. 3, pp. 877-893, 2022. DOI: 10.3837/tiis.2022.03.007.

[ACM Style]
You Yang, Sixun Liu, Bin Xing, and Kesen Li. 2022. Face inpainting via Learnable Structure Knowledge of Fusion Network. KSII Transactions on Internet and Information Systems, 16, 3, (2022), 877-893. DOI: 10.3837/tiis.2022.03.007.

[BibTeX Style]
@article{tiis:25521, title="Face inpainting via Learnable Structure Knowledge of Fusion Network", author="You Yang and Sixun Liu and Bin Xing and Kesen Li and ", journal="KSII Transactions on Internet and Information Systems", DOI={10.3837/tiis.2022.03.007}, volume={16}, number={3}, year="2022", month={March}, pages={877-893}}