Evaluating the Validity of Computerized Content Analysis Programs for Identification of Emotional Expression in Cancer Narratives

• Published in: Psychological assessment 2009-03, Vol.21 (1), p.79-88
• Main Authors: Bantum, Erin O'Carroll, Owen, Jason E
• Format: Article
• Language: English
• Subjects: Adaptation, Psychological; Anxiety - diagnosis; Anxiety - etiology; Anxiety - therapy; Automatic Data Processing - methods; Automatic Data Processing - statistics & numerical data; Behavior modification; Biological and medical sciences; Breast cancer; Breast Neoplasms; Breast Neoplasms - complications; Breast Neoplasms - psychology; Cancer; Coding; Computational Linguistics; Computer Software; Content Analysis; Correlation; Depression - diagnosis; Depression - etiology; Depression - therapy; Discriminant Analysis; Emotional Response; Emotions; Evaluators; Expressed Emotion; Female; Females; Fundamental and applied biological sciences. Psychology; Human; Humans; Identification; Internet; Intervention; Linguistics; Medical diagnosis; Middle Aged; Narratives; Psychiatric Status Rating Scales - statistics & numerical data; Psycholinguistics - statistics & numerical data; Psychological tests; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Psychometrics; Psychometrics. Statistics. Methodology; Quality of Life; Reproducibility of Results; Self Disclosure; Sensitivity and Specificity; Signal Detection, Psychological; Social Support; Stress, Psychological - diagnosis; Stress, Psychological - etiology; Stress, Psychological - therapy; Studies; Test Validity; Validity; Womens health
• ISSN: 1040-3590; 1939-134X
• DOI: 10.1037/a0014643

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer ( n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry and Word Count [LIWC] and Psychiatric Content Analysis and Diagnosis [PCAD]). Although the computerized coding methods captured most of the emotion identified by raters (LIWC sensitivity = .88; PCAD sensitivity = .83), both over-identified emotional expression (LIWC positive predictive value = .31; PCAD positive predictive value = .19). Correlational analyses suggested better convergent and discriminant validity for LIWC. The results highlight previously unrecognized deficiencies in commonly used computerized content-analysis programs and suggest potential modifications to both programs that could improve overall accuracy of automated identification of emotional expression. Although the authors recognize these limitations, they conclude that LIWC is superior to PCAD for rapid identification of emotional expression in text.