Prediction Models of Collaborative Behaviors in Dyadic Interactions: An Application for Inclusive Teamwork Training in Virtual Environments

• Published in: Signals (Basel) 2024-06, Vol.5 (2), p.382-401
• Main Authors: Ashwaq Zaini Amat, Plunk, Abigale, Adiani, Deeksha, D Mitchell Wilkes, Sarkar, Nilanjan
• Format: Article
• Language: English
• Subjects: Accuracy; Autism; Behavior; Classification; Collaboration; collaborative virtual environment; cross-neurotype collaboration; Design; Feedback; Furniture; human behavior recognition; human–computer interaction; Machine learning; Neural networks; probabilistic modeling; Skills; Support vector machines; Teamwork; Work environment
• ISSN: 2624-6120; 2624-6120
• DOI: 10.3390/signals5020019

Collaborative virtual environment (CVE)-based teamwork training offers a promising avenue for inclusive teamwork training. The incorporation of a feedback mechanism within virtual training environments can enhance the training experience by scaffolding learning and promoting active collaboration. However, an effective feedback mechanism requires a robust prediction model of collaborative behaviors. This paper presents a novel approach using hidden Markov models (HMMs) to predict human behavior in collaborative interactions based on multimodal signals collected from a CVE-based teamwork training simulator. The HMM was trained using k-fold cross-validation, achieving an accuracy of 97.77%. The HMM was evaluated against expert-labeled data and compared against a rule-based prediction model, demonstrating the superior predictive capabilities of the HMM, with the HMM achieving 90.59% accuracy compared to 76.53% for the rule-based model. These results highlight the potential of HMMs to predict collaborative behaviors that could be used in a feedback mechanism to enhance teamwork training experiences despite the complexity of these behaviors. This research contributes to advancing inclusive and supportive virtual learning environments, bridging gaps in cross-neurotype collaborations.