PrAACT: Predictive Augmentative and Alternative Communication with Transformers

• Published in: Expert systems with applications 2024-04, Vol.240, p.122417, Article 122417
• Main Authors: Pereira, Jayr Alencar, Pereira, Jaylton Alencar, Zanchettin, Cleber, do Nascimento Fidalgo, Robson
• Format: Article
• Language: English
• Subjects: Augmentative and Alternative Communication; Communication card prediction; Language modeling
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2023.122417

Augmentative and Alternative Communication (AAC) systems assist individuals with complex communication needs, allowing them to construct sentences by selecting communication cards. The facilitation of sentence construction primarily entails enabling efficient communication card selection. An approach to this involves using language models for communication card prediction. Nonetheless, the heterogeneity of the AAC population poses unique challenges due to diverse vocabulary needs, suggesting one-size-fits-all approaches may be insufficient. This study introduces PrAACT, a method that leverages large, transformer-based language models, such as BERT, for communication card prediction. This method focuses on easy adaptability to incorporate user-specific vocabularies into the model. The method involves adapting a text corpus to the AAC domain, fine-tuning a transformer-based language model, and replacing the language model weights with an encoded representation of the user’s vocabulary. We conducted an assessment of PrAACT under both zero-shot and few-shot scenarios. The performance of the models produced using PrAACT was superior to those pre-trained for the task. In addition, the main advantage of PrAACT is that it allows to quickly adapt a transformer-based language model to communication card prediction according to the user’s vocabulary. •Developed PrAACT, a new method for communication card prediction in AAC systems.•PrAACT enhances message authoring by accommodating diverse vocabularies.•PrAACT models outperform pre-trained in zero-shot and few-shot settings.•Demonstrated the potential of LLMs to improve communication in AAC system.