General Category Network: Handwritten Mathematical Expression Recognition with Coarse-Grained Recognition Task

Handwritten Mathematical Expression Recognition (HMER) is an important task in pattern recognition. It is a challenging task due to symbols resembling each other in appearance("z/2", "B/β") and the complex mathematical syntax. The encoder-decoder architecture has been widely used in recent HMER methods. Several works introduce HMER-related tasks that enhance the performance of HMER. We propose the General Category Recognition Task (GCRT) and design the General Category Network(GCN) to perform HMER and GCRT in parallel. GCRT alleviates the symbol confusion problem and also helps the model generate results conforming to mathematical syntax. Compared with SOTA methods, the experimental results show that Expression Recognition Rates(ExpRate) of our method are increased by 1.12%, 2.45% and 1.84% on CROHME 2014, 2016 and 2019 test set respectively. In addition, current works explore the effect of a single auxiliary task on HMER. We investigated the effect of multiple tasks on HMER. We performed many experiments and drew multiple valuable conclusions.We will publish all codes in the future.