A General Survey on Attention Mechanisms in Deep Learning

Attention is an important mechanism that can be employed for a variety of\ndeep learning models across many different domains and tasks. This survey\nprovides an overview of the most important attention mechanisms proposed in the\nliterature. The various attention mechanisms are explained by means of a\nframework consisting of a general attention model, uniform notation, and a\ncomprehensive taxonomy of attention mechanisms. Furthermore, the various\nmeasures for evaluating attention models are reviewed, and methods to\ncharacterize the structure of attention models based on the proposed framework\nare discussed. Last, future work in the field of attention models is\nconsidered.\n