LERCause: Deep learning approaches for causal sentence identification from nuclear safety reports

• Published in: PloS one 2024-08, Vol.19 (8), p.e0308155
• Main Authors: Kim, Jinmo, Kim, Jenna, Lee, Aejin, Kim, Jinseok, Diesner, Jana
• Format: Article
• Language: English
• Subjects: Analysis; Artificial neural networks; Automation; Biology and Life Sciences; Causality; Classification; Computational linguistics; Computer and Information Sciences; Datasets; Deep Learning; Heuristic methods; Human error; Humans; Identification; Keywords; Labeling; Labels; Language; Language processing; Learning algorithms; Machine learning; Natural language interfaces; Natural Language Processing; Neural networks; Neural Networks, Computer; Nuclear accidents; Nuclear accidents & safety; Nuclear energy; Nuclear engineering; Nuclear power plants; Nuclear safety; Physical Sciences; Safety; Safety research; Social Sciences; Support Vector Machine; Support vector machines
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0308155

Identifying causal sentences from nuclear incident reports is essential for advancing nuclear safety research and applications. Nonetheless, accurately locating and labeling causal sentences in text data is challenging, and might benefit from the usage of automated techniques. In this paper, we introduce LERCause, a labeled dataset combined with labeling methods meant to serve as a foundation for the classification of causal sentences in the domain of nuclear safety. We used three BERT models (BERT, BioBERT, and SciBERT) to 10,608 annotated sentences from the Licensee Event Report (LER) corpus for predicting sentence labels (Causal vs. non-Causal). We also used a keyword-based heuristic strategy, three standard machine learning methods (Logistic Regression, Gradient Boosting, and Support Vector Machine), and a deep learning approach (Convolutional Neural Network; CNN) for comparison. We found that the BERT-centric models outperformed all other tested models in terms of all evaluation metrics (accuracy, precision, recall, and F1 score). BioBERT resulted in the highest overall F1 score of 94.49% from the ten-fold cross-validation. Our dataset and coding framework can provide a robust baseline for assessing and comparing new causal sentences extraction techniques. As far as we know, our research breaks new ground by leveraging BERT-centric models for causal sentence classification in the nuclear safety domain and by openly distributing labeled data and code to enable reproducibility in subsequent research.