Devil in the details: Delving into accurate quality scoring for DensePose

• Published in: Pattern recognition 2024-04, Vol.148, p.110197, Article 110197
• Main Authors: Sun, Junyao, Liu, Qiong
• Format: Article
• Language: English
• Subjects: Dense human pose estimation; Instance-aware feature; Quality estimation
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2023.110197

How to score the quality of the network output is an essential but long-neglected problem in DensePose, which dramatically limits the potential of the existing methods. To fill the blank in the quality estimation of DensePose, we conduct rigorous experiments to clarify the key factors that accurately reflect the quality of DensePose results. We find that the accurate results already exist in the candidate pool but are mistakenly removed due to the inappropriate quality scores. To solve this problem, we proposed DensePose Scoring RCNN (DS RCNN), a simple and comprehensive quality estimation framework to learn the calibrated quality score and select high-quality results from the pool. DS RCNN introduces a quality scoring module (QSM) and a quality perception module (QPM) into the existing high-performance pipeline. The QSM scores the quality of DensePose results by fusing diverse quality information, and the QPM enhances the ability of quality perception by extracting instance-aware quality features guided by the predicted IUV maps. Benefiting from the superiority of QSM and QPM, DS RCNN outperforms baselines by up to 4.8 AP on the DensePose-COCO dataset. •Analyze and reveal how using quality information to benefit from the DensePose task.•Propose DS RCNN to learn quality scores with quality scoring and perception modules.•Validate the effectiveness and generalization of DS RCNN on three DensePose datasets.