An attentive joint model with transformer-based weighted graph convolutional network for extracting adverse drug event relation

• Published in: Journal of biomedical informatics 2022-01, Vol.125, p.103968-103968, Article 103968
• Main Authors: El-allaly, Ed-drissiya, Sarrouti, Mourad, En-Nahnahi, Noureddine, Ouatik El Alaoui, Said
• Format: Article
• Language: English
• Subjects: Adverse drug events; Drug-Related Side Effects and Adverse Reactions; Humans; Joint learning; Natural language processing; Relation extraction; Transfer learning; Weighted graph convolutional network
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2021.103968

[Display omitted] •We develop ADERel, a neural joint system for extracting ADE relations.•It jointly learns N-level sequence labelling to deal with complex relations.•It integrates a transformed-based weighted GCN as a shared representation.•A multi-head attention is applied for exchanging boundary knowledge across levels.•We study the effectiveness and generalizability of ADERel on two datasets. Adverse drug event (ADE) relation extraction is a crucial task for drug safety surveillance which aims to discover potential relations between ADE mentions from unstructured medical texts. To date, the graph convolutional networks (GCN) have been the state-of-the-art solutions for improving the ability of relation extraction task. However, there are many challenging issues that should be addressed. Among these, the syntactic information is not fully exploited by GCN-based methods, especially the diversified dependency edges. Still, these methods fail to effectively extract complex relations that include nested, discontinuous and overlapping mentions. Besides, the task is primarily regarded as a classification problem where each candidate relation is treated independently which neglects the interaction between other relations. To deal with these issues, in this paper, we propose an attentive joint model with transformer-based weighted GCN for extracting ADE Relations, called ADERel. Firstly, the ADERel system formulates the ADE relation extraction task as an N-level sequence labelling so as to model the complex relations in different levels and capture greater interaction between relations. Then, it exploits our neural joint model to process the N-level sequences jointly. The joint model leverages the contextual and structural information by adopting a shared representation that combines a bidirectional encoder representation from transformers (BERT) and our proposed weighted GCN (WGCN). The latter assigns a score to each dependency edge within a sentence so as to capture rich syntactic features and determine the most influential edges for extracting ADE relations. Finally, the system employs a multi-head attention to exchange boundary knowledge across levels. We evaluate ADERel on two benchmark datasets from TAC 2017 and n2c2 2018 shared tasks. The experimental results show that ADERel is superior in performance compared with several state-of-the-art methods. The results also demonstrate that incorporating a transformer model with WGCN makes the proposed system mo