A Multi-level Synthesis Strategy for Online Handwritten Chemical Equation Recognition

Handwritten chemical equation recognition is an appealing task, but its development is hampered by the lack of publicly available datasets. To this end, we propose a multi-level synthesis strategy to synthesize the corresponding handwritten equations from LaTeX expressions and regard the chemical equation recognition as an image-to-markup task. In particular, our approach first decomposes the LaTeX expression into a symbol layout tree (SLT) and obtains different multi-level components in stages by traversing the SLT. Then, online isolated symbols are placed in appropriate locations consistent with handwritten habits through a baseline-based layout strategy. Furthermore, expression patterns are enhanced at the local, component, and global levels to increase the diversity of synthesized data. It is worth noting that our synthesis strategy is theoretically applicable to any LaTeX-based expression. We also collected a real dataset containing 1595 handwritten chemical equations, and the experimental results confirm that our proposed method can effectively improve the performance of handwritten chemical equation recognition systems. The dataset we generated will be released.