Signing avatars: making education more inclusive

• Published in: Universal access in the information society 2017-08, Vol.16 (3), p.793-808
• Main Authors: De Martino, José Mario, Silva, Ivani Rodrigues, Bolognini, Carmen Zink, Costa, Paula Dornhofer Paro, Kumada, Kate Mamhy Oliveira, Coradine, Luis Cláudius, Brito, Patrick Henrique da Silva, do Amaral, Wanessa Machado, Benetti, Ângelo Brandão, Poeta, Enzo Telles, Angare, Leandro Martin Guertzenstein, Ferreira, Carolina Monteiro, De Conti, Davi Faria
• Format: Article
• Language: English
• Subjects: Accessibility; Avatars; Bilingual education; Brazilian Portuguese; Brazilian Sign Language; Children; Computer Communication Networks; Computer Science; Computers and Society; Corpus linguistics; Deafness; Education; English as a second language; Handicapped accessibility; Hearing loss; Inclusive education; Information Storage and Retrieval; Information Systems Applications (incl.Internet); Inhabitants; Intelligibility; IT in Business; Long Paper; Machine translation; Motion capture; Portuguese as a second language; Students with disabilities; Teaching methods; User Interfaces and Human Computer Interaction
• ISSN: 1615-5289; 1615-5297
• DOI: 10.1007/s10209-016-0504-x

In Brazil, there are approximately 9.7 million inhabitants who are deaf or hard of hearing. Moreover, about 30% of the Brazilian deaf community is illiterate in Brazilian Portuguese due to difficulties to offer deaf children an inclusive environment based on bilingual education. Currently, the prevailing teaching practice depends heavily on verbal language and on written material, making the inclusion of the deaf a challenging task. This paper presents the author’s approach for tackling this problem and improving deaf students’ accessibility to written material in order to help them master Brazilian Portuguese as a second language. We describe an ongoing project aimed at developing an automatic Brazilian Portuguese-to-Libras translation system that presents the translated content via an animated virtual human, or avatar. The paper describes the methodology adopted to compile a source language corpus having the deaf student needs in central focus. It also describes the construction of a parallel Brazilian Portuguese/Brazilian Sign Language (Libras) corpus based on motion capture technology. The envisioned translation architecture includes the definition of an Intermediate Language to drive the signing avatar. The results of a preliminary assessment of signs intelligibility highlight the application potential.