Layout similarity based key information extraction framework for structural images

Image key information extraction is an important technique in automatic digital image recognizers. To deal with complex image layout, vague image semantics and image defects, we propose a layout similarity-based model to extract image key information under certain image distortion situations. Our model uses perceptual hash algorithm (PHA) to detect visually similar images and uses a template matching based algorithm to solve distortion problems when PHA cannot find a match. The framework is tested on a dataset with more than 17,000 images and we demonstrate that our model can reach 99.5% accuracy on the original dataset, significantly outperforming Chargrid model and a parity-based model. We also compare the performance of different models on images with artificially overlaid boxes, which is commonly seen on screenshots. Results show that our model still performs much better than the parity-based model.