MTTLADE: A multi-task transfer learning-based method for adverse drug events extraction

Extracting mentions of Adverse Drug Events (ADEs) and the potential relationships among them from clinical textual data remains challenging tasks due to the following issues: (1) many ADEs mentions have multiple relations, also known as the multi-head issue, and (2) many ADEs relations contain discontinuous mentions. To deal with these problems, in this paper, we propose a Multi-Task Transfer Learning-based method for ADEs extraction, called MTTLADE. Firstly, the MTTLADE system converts the ADEs extraction task to a dual-task sequence labelling which includes ADEs source mention extraction (ADE-SE) and ADEs attribute-relation extraction (ADE-Att-RE) tasks. The ADE-SE task aims at extracting the source mentions that are likely related to at least one relation, while the ADE-Att-RE task consists in linking the previously identified source mentions to their target attributes and relation types by adopting a unified sequence labelling. Then, it uses the multi-task transfer learning (MTTL) based approach to process the two proposed tasks simultaneously. The MTTL adopts a shared representation obtained from the pre-trained language model learned through transformer architecture and ends up with task-specific fine-tuning. This allows the MTTLADE system to yield more generalized representation across the tasks. Finally, MTTLADE produces sequences for each task from the generated model so as to extract ADEs mentions and relations. Experimental evaluations conducted on two datasets provided by the TAC 2017 and n2c2 2018 shared tasks show the effectiveness and generalizability of MTTLADE. The proposed MTTLADE system significantly outperforms the state-of-the-art ones on both datasets. The results also show that combining transfer and multi-task learning makes MTTLADE more effective for solving the multi-head issue and extracting intricate ADEs.