Uncertainty-Aware Mean Opinion Score Prediction

Mean Opinion Score (MOS) prediction has made significant progress in specific domains. However, the unstable performance of MOS prediction models across diverse samples presents ongoing challenges in the practical application of these systems. In this paper, we point out that the absence of uncertai...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-08
Main Authors: Wang, Hui, Zhao, Shiwan, Zhou, Jiaming, Zheng, Xiguang, Sun, Haoqin, Wang, Xuechen, Qin, Yong
Format: Article
Language:English
Subjects:
Performance prediction
Prediction models
Uncertainty analysis
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Mean Opinion Score (MOS) prediction has made significant progress in specific domains. However, the unstable performance of MOS prediction models across diverse samples presents ongoing challenges in the practical application of these systems. In this paper, we point out that the absence of uncertainty modeling is a significant limitation hindering MOS prediction systems from applying to the real and open world. We analyze the sources of uncertainty in the MOS prediction task and propose to establish an uncertainty-aware MOS prediction system that models aleatory uncertainty and epistemic uncertainty by heteroscedastic regression and Monte Carlo dropout separately. The experimental results show that the system captures uncertainty well and is capable of performing selective prediction and out-of-domain detection. Such capabilities significantly enhance the practical utility of MOS systems in diverse real and open-world environments.
ISSN:2331-8422