Body communicative cue extraction for conversational analysis

• Main Authors: Marcos-Ramiro, Alvaro, Pizarro-Perez, Daniel, Marron-Romera, Marta, Nguyen, Laurent, Gatica-Perez, Daniel
• Format: Conference Proceeding
• Language: English
• Subjects: Cameras; Face; Image color analysis; Joints; Skin; Torso; Training
• DOI: 10.1109/FG.2013.6553741

Nonverbal communication plays an important role in many aspects of our lives, such as in job interviews, where vis-α-vis conversations take place. This paper proposes a method to automatically detect body communicative cues by using video sequences of the upper body of individuals in a conversational context. To our knowledge, our work brings novelty by explicitly addressing the recognition of visual activity in a seated, conversational setting from monocular video, compared to most existing work in video-based motion capture, which targets full-body with lower limb activities. We first detect the person hands in the sequence by searching for the higher speed parts along the whole video. Then, aided by training a set of typical conversational movements, we infer the approximate 3D upper body pose, that we transfer to a low-dimensionality space in order to perform action recognition. We test our system in the context of job interviews, with several new databases that we make publicly available.