Monte Carlo DropBlock for modeling uncertainty in object detection

• Published in: Pattern recognition 2024-02, Vol.146, p.110003, Article 110003
• Main Authors: Yelleni, Sai Harsha, Kumari, Deepshikha, P.K., Srijith, C., Krishna Mohan
• Format: Article
• Language: English
• Subjects: Bayesian deep learning; DropBlock; Instance segmentation; Monte Carlo method; Object detection; Uncertainty estimation
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2023.110003

With the advancements made in deep learning, computer vision problems have seen a great improvement in performance. However, in many real-world applications such as autonomous driving vehicles, the risk associated with incorrect predictions of objects or segmentation of images is very high. Standard deep learning models for object detection and segmentation such as YOLO models are often overconfident in their predictions and do not take into account the uncertainty in predictions on out-of-distribution data. In this work, we propose an efficient and effective approach, Monte-Carlo DropBlock (MC-DropBlock), to model uncertainty in YOLO and convolutional vision Transformers for object detection. The proposed approach applies drop-block during training time and testing time on the convolutional layer of the deep learning models such as YOLO and convolutional transformer. We theoretically show that this leads to a Bayesian convolutional neural network capable of capturing the epistemic uncertainty in the model. Additionally, we capture the aleatoric uncertainty in the data using a Gaussian likelihood. We demonstrate the effectiveness of the proposed approach on modeling uncertainty in object detection and segmentation tasks using out-of-distribution experiments. Experimental results show that MC-DropBlock improves the generalization, calibration, and uncertainty modeling capabilities of YOLO models and convolutional Transformer models for object detection and segmentation. •We propose Monte-Carlo DropBlock to model epistemic uncertainty in complex CNNs such as YOLO models for object detection and segmentation.•We theoretically show that applying MC-DropBlock on a CNN is equivalent to performing variational inference with a specific variational distribution on a Bayesian CNN.•We show quantitatively and qualitatively that MC-DropBlock significantly improves generalization, calibration and uncertainty modeling capabilities of YOLO models for object detection and segmentation.