VIRTUAL RESTORATION OF MISSING PAINT LOSS OF MURAL BASED ON GENERATIVE ADVERSARIAL NETWORK

• Published in: International archives of the photogrammetry, remote sensing and spatial information sciences. remote sensing and spatial information sciences., 2021-09, Vol.XLVI-M-1-2021, p.807-811
• Main Authors: Wang, Q., Hou, M., Lyu, S.
• Format: Article
• Language: English
• Subjects: Algorithms; Color; Color fading; Colour; Cultural heritage; Cultural resources; Generative adversarial networks; Image restoration; Manpower; Matching; Murals; Repair; Restoration; Semantics; Training
• ISSN: 2194-9034; 1682-1750; 2194-9034
• DOI: 10.5194/isprs-archives-XLVI-M-1-2021-807-2021

Mural painting is one of the important cultural heritage reflecting the historical migration of the nation. In order to inherit these precious historical and cultural heritage, how to non - destructively and digitally protect and restore the existing murals has become an urgent task. The use of computer - assisted restoration of murals can not only save manpower and material resources, but also avoid secondary damage to the murals.However, most of the existing computer-assisted mural restoration algorithms use similar blocks with priority calculations and matching blocks in adjacent areas to guide mural restoration. There are some problems such as incoherent overall semantic structure, unnatural detail texture and inability to effectively repair large area missing remain to be solved. Aiming at the problems existing in the restoration of large area diseases such as paint loss and color fading in murals, we constructed a fine image restoration network model which based on generative adversarial network. A multi-scale dense matching repair network based on a generative adversarial network is constructed. First, the dense combination of dilated convolutions is used to improve the repair effect of detailed textures, Then, mean absolute Error, (Visual Geometry Group, VGG) feature matching, auto-guided regression, and geometric alignment are used as the loss function to guide the training of the generative network. Second, the discriminator with local and global branches is used to train the discriminant network, so that the repaired image is in the local and global content. Experiments were performed on the three mural data sets one by one. The results show that the network model can effectively restore the lines and faces in the murals. The images restored are not only coherent in semantic details, but also natural in color, which is conducive to the appreciation and display of murals. Thus, as one of the important directions of cultural heritage digital protection，the use of generative adversarial network in the digital restoration of ancient murals have been proved to be effective. It not only provides a reference for the true restoration of the murals but also means a lot to the preservation of murals.