Clinical Trial Information Extraction with BERT

Abstract Background Clinical trials are the gateway to new treatments. Clinical trial designs are documented in unstructured text and insights from natural language processing (NLP) of the documents can be useful in new trial design. Here we identify entity and relation types relevant to clinical trial design and propose a framework called CT-BERT to extract entities and relations from the clinical trial documents. Methods CT-BERT uses BERT models for named entity recognition (NER) and a hybrid approach for relation extraction including rule-based and machine learning-based models. We trained NER models to extract eligibility criteria entities by fine-tuning a set of pre-trained BERT models, including BERT, BioBERT, ClinicalBERT and BlueBERT. We then developed a rule-based relation extraction model to extract relations based on the NER results. For comparison, we also trained machine learning models using BERT to classify entity pairs into relation types. We compared the performance of CT-BERT with recent baseline methods including attention-based BiLSTM (Att-BiLSTM) and Criteria2Query. Results The NER training result shows that BioBERT large fine-tuned on clinical trials data achieves the best F1 score and is thus selected as the optimal NER model for CT-BERT. The result on a benchmark dataset showed that CT-BERT NER outperforms Att-BiLSTM and Criteria2Query. CT-BERT NER achieves 0.844 F1, while Att-BiLSTM and Criteria2Query achieve 0.802 F1 and 0.804 F1, respectively. For relation extraction, the CT-BERT rule-based model achieves 0.870 F1, while the equivalent model in Criteria2Query only achieves 0.793 F1. The CT-BERT rule-based model also outperforms the machine learning models trained on a curated relation data set. This suggests the advantage of the rule-based method in relation extraction. Conclusions The NER and relation extraction results demonstrate the effectiveness of CT-BERT for clinical trial information extraction. These results form the foundation for advancing clinical trials NLP towards AI-enhanced clinical trial design.