A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform

During the COVID-19 pandemic, the corporate online training sector has increased exponentially and online course providers had to implement innovative solutions to be more efficient and provide a satisfactory service. This paper considers a real case study in implementing a chatbot, which answers fr...

Full description

Saved in:
Bibliographic Details
Published in:Future internet 2022-02, Vol.14 (2), p.62
Main Authors: Zubani, Matteo, Sigalini, Luca, Serina, Ivan, Putelli, Luca, Gerevini, Alfonso E., Chiari, Mattia
Format: Article
Language:English
Subjects:
Chatbots
Cloud computing
cloud-based services
Coronaviruses
COVID-19
Distance learning
Internet
machine learning
natural language understanding
Occupational safety
Online instruction
Performance evaluation
Recognition
Response time
Sentences
Students
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:During the COVID-19 pandemic, the corporate online training sector has increased exponentially and online course providers had to implement innovative solutions to be more efficient and provide a satisfactory service. This paper considers a real case study in implementing a chatbot, which answers frequently asked questions from learners on an Italian e-learning platform that provides workplace safety courses to several business customers. Having to respond quickly to the increase in the courses activated, the company decided to develop a chatbot using a cloud-based service currently available on the market. These services are based on Natural Language Understanding (NLU) engines, which deal with identifying information such as entities and intentions from the sentences provided as input. To integrate a chatbot in an e-learning platform, we studied the performance of the intent recognition task of the major NLU platforms available on the market with an in-depth comparison, using an Italian dataset provided by the owner of the e-learning platform. We focused on intent recognition, carried out several experiments and evaluated performance in terms of F-score, error rate, response time, and robustness of all the services selected. The chatbot is currently in production, therefore we present a description of the system implemented and its results on the original users’ requests.
ISSN:1999-5903
1999-5903
DOI:10.3390/fi14020062