Machine learning judged neutral facial expressions as key factors for a “good therapist” within the first five minutes: An experiment to simulate online video counselling

• Published in: PEC innovation 2024-12, Vol.4, p.100302, Article 100302
• Main Authors: Yokoyama, Satoshi, Shikano, Asuna, Chiba, Hiroki, Murakami, Takeshi, Kawamorita, Takushi, Murayama, Takayuki, Ito, Daisuke, Ichikura, Kanako
• Format: Article
• Language: English
• Subjects: Facial expression; Full length article; Impression of therapists; Machine learning; Online video counselling
• ISSN: 2772-6282; 2772-6282
• DOI: 10.1016/j.pecinn.2024.100302

Machine learning models were employed to discern patients' impressions from the therapists' facial expressions during a virtual online video counselling session. Eight therapists simulated an online video counselling session for the same patient. The facial emotions of the therapists were extracted from the session videos; we then utilized a random forest model to determine the therapist's impression as perceived by the patients. The therapists' neutral facial expressions were important controlling factors for patients' impressions. A predictive model with three neutral facial features achieved an accuracy of 83% in identifying patients' impressions. Neutral facial expressions may contribute to patient impressions in an online video counselling environment with spatiotemporal disconnection. Expression recognition techniques were applied innovatively to an online counselling setting where therapists' expressions are limited. Our findings have the potential to enhance psychiatric clinical practice using Information and Communication Technology. •Little is known about the impact of therapists' facial expressions on patients in a limited and time-lagged video space.•Therapists' neutral facial expressions are important factors determining impressions.•Therapists might need to pay attention to neutral facial expressions.