Facial Affect Sensing for T-learning

• Main Authors: Hupont, I, Abadia, D, Baldassarri, S, Cerezo, E, Del-Hoyo, R
• Format: Conference Proceeding
• Language: English
• Subjects: Affective Computing; Artificial intelligence; Data mining; Emotion recognition; emotional intelligence; Face recognition; facial expression classification; HDTV; intelligent tutoring systems; interactive digital television
• ISSN: 1082-3409; 2375-0197
• DOI: 10.1109/ICTAI.2010.109

Interactive Digital TV has arisen new forms of interaction not traditionally associated with this medium. One of its applications is the so-called "t-learning" or TV-based interactive learning. To date, most existing t-learning applications confine themselves to edutainment more than to more complex and pedagogical forms of learning, due to the technological constraints imposed by set-top boxes. This paper overcomes those technological barriers to propose the first t-learning affective aware tutoring tool. The tool allows to capture by means of a camera the facial expressions of the student while performing evaluations (tests) through a broadcasted interactive t-learning application at home. It integrates a novel Artificial Intelligence method for facial affect recognition, where facial expressions are evaluated with a psychological 2-dimensional continuous affective approach. Thanks to it, the t-learning tool is able to automatically extract emotional information from the learner, which is further presented in a simple and efficient way to the distance tutor so that he can be aware of student's encountered difficulties and emotional progression during the learning process.