Deep learning for depression recognition with audiovisual cues: A review

• Published in: Information fusion 2022-04, Vol.80, p.56-86
• Main Authors: He, Lang, Niu, Mingyue, Tiwari, Prayag, Marttinen, Pekka, Su, Rui, Jiang, Jiewei, Guo, Chenguang, Wang, Hongyu, Ding, Songtao, Wang, Zhongmin, Pan, Xiaoying, Dang, Wei
• Format: Article
• Language: English
• Subjects: Affective computing; Automatic depression estimation; Deep learning; Depression; Review
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2021.10.012

With the acceleration of the pace of work and life, people are facing more and more pressure, which increases the probability of suffering from depression. However, many patients may fail to get a timely diagnosis due to the serious imbalance in the doctor–patient ratio in the world. A promising development is that physiological and psychological studies have found some differences in speech and facial expression between patients with depression and healthy individuals. Consequently, to improve current medical care, Deep Learning (DL) has been used to extract a representation of depression cues from audio and video for automatic depression detection. To classify and summarize such research, we introduce the databases and describe objective markers for automatic depression estimation. We also review the DL methods for automatic detection of depression to extract a representation of depression from audio and video. Lastly, we discuss challenges and promising directions related to the automatic diagnoses of depression using DL. •First review on depression recognition from audiovisual cues that is dealing with both single modal and multi-modal analysis.•The review focuses on deep learning that is adopting in depression recognition task with audiovisual cues.•It covers the state-of-the-art methods for depression recognition in details.