A Character Attention Generative Adversarial Network for Degraded Historical Document Restoration

Despite of recent breakthroughs in the accuracy of single character recognition using the deeper convolution neural networks, one of the remaining problems is that OCRs almost fail to recognize character patterns when they are severely degraded, especially those of the historical documents. Another problem to recognize characters in historical documents is the lack of sufficient training patterns because of the heavy cost for annotation. This paper proposes a character attention generative adversarial network named CAGAN for restoring heavily degraded character patterns in historical documents so that OCRs improve their accuracy and even help archeologists to decode them. The network is based on the U-Net like architecture [1] with skip connections, and it is trained by the proposed loss function including the common adversarial loss (global loss) and the hierarchical character attentive loss (local loss). We made an experiment on 118 categories of most common Japanese Kanji characters, collected from severely damaged historical documents called Heijokyo mokkan written during the Nara period in Japan. The experiment shows that our method restores the shapes of characters and improves the recognition rate significantly, which is helpful for archeologists to decode damaged character patterns.