Multidimensional Evaluation Methods for Deep Learning Models in Target Detection for SAR Images

• Published in: Remote sensing (Basel, Switzerland) Switzerland), 2024-03, Vol.16 (6), p.1097
• Main Authors: Wang, Pengcheng, Liu, Huanyu, Zhou, Xinrui, Xue, Zhijun, Ni, Liang, Han, Qi, Li, Junbao
• Format: Article
• Language: English
• Subjects: Accuracy; Adaptability; Algorithms; Artificial intelligence; Artificial satellites in remote sensing; Data processing; Datasets; Deep learning; Energy consumption; evaluation metrics; Experiments; Methods; multidimensional evaluation; Multidimensional methods; Object recognition; Optimization; Performance evaluation; Radar imaging; Remote sensing; Researchers; SAR images; Synthetic aperture radar; Target detection; Task complexity; Training; Ultrasonic imaging
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16061097

As artificial intelligence technology advances, the application of object detection technology in the field of SAR (synthetic aperture radar) imagery is becoming increasingly widespread. However, it also faces challenges such as resource limitations in spaceborne environments and significant uncertainty in the intensity of interference in application scenarios. These factors make the performance evaluation of object detection key to ensuring the smooth execution of tasks. In the face of such complex and harsh application scenarios, methods that rely on single-dimensional evaluation to assess models have had their limitations highlighted. Therefore, this paper proposes a multi-dimensional evaluation method for deep learning models used in SAR image object detection. This method evaluates models in a multi-dimensional manner, covering the training, testing, and application stages of the model, and constructs a multi-dimensional evaluation index system. The training stage includes assessing training efficiency and the impact of training samples; the testing stage includes model performance evaluation, application-based evaluation, and task-based evaluation; and the application stage includes model operation evaluation and model deployment evaluation. The evaluations of these three stages constitute the key links in the performance evaluation of deep learning models. Furthermore, this paper proposes a multi-indicator comprehensive evaluation method based on entropy weight correlation scaling, which calculates the weights of each evaluation indicator through test data, thereby providing a balanced and comprehensive evaluation mechanism for model performance. In the experiments, we designed specific interferences for SAR images in the testing stage and tested three models from the YOLO series. Finally, we constructed a multi-dimensional performance profile diagram for deep learning object detection models, providing a new visualization method to comprehensively characterize model performance in complex application scenarios. This can provide more accurate and comprehensive model performance evaluation for remote sensing data processing, thereby guiding model selection and optimization. The evaluation method proposed in this study adopts a multi-dimensional perspective, comprehensively assessing the three core stages of a model’s lifecycle: training, testing, and application. This framework demonstrates significant versatility and adaptability, enabling it to tra