Topic Categorisation of Statements in Suicide Notes with Integrated Rules and Machine Learning

• Published in: Biomedical Informatics Insights 2012-01, Vol.2012 (Suppl. 1), p.115-124
• Main Authors: Kovačević, Aleksandar, Dehghan, Azad, Keane, John A., Nenadic, Goran
• Format: Article
• Language: English
• Subjects: Automated; Categories; Intermetallics; Machine learning; Mathematical models; Original Research; Recall; Silicides; Texts
• ISSN: 1178-2226; 1178-2226
• DOI: 10.4137/BII.S8978

We describe and evaluate an automated approach used as part of the i2b2 2011 challenge to identify and categorise statements in suicide notes into one of 15 topics, including Love, Guilt, Thankfulness, Hopelessness and Instructions. The approach combines a set of lexico-syntactic rules with a set of models derived by machine learning from a training dataset. The machine learning models rely on named entities, lexical, lexico-semantic and presentation features, as well as the rules that are applicable to a given statement. On a testing set of 300 suicide notes, the approach showed the overall best micro F-measure of up to 53.36%. The best precision achieved was 67.17% when only rules are used, whereas best recall of 50.57% was with integrated rules and machine learning. While some topics (eg, Sorrow, Anger, Blame) prove challenging, the performance for relatively frequent (eg, Love) and well-scoped categories (eg, Thankfulness) was comparatively higher (precision between 68% and 79%), suggesting that automated text mining approaches can be effective in topic categorisation of suicide notes.