Binary search of the optimal cut-point value in ROC analysis using the F 1 score

• Published in: Journal of physics. Conference series 2023-10, Vol.2609 (1), p.12002
• Main Authors: Tan, Swee Chuan, Zhu, Siying
• Format: Article
• Language: English
• ISSN: 1742-6588; 1742-6596
• DOI: 10.1088/1742-6596/2609/1/012002

This paper introduces a binary search algorithm for determining the optimal probability cut-point value ( C ) of binary classifiers. Cut-points are operating points on the receiver operating characteristic curve that divide positive and negative predictions. Compared to the traditional exhaustive search for optimal C value, the proposed method offers execution time efficiency ( O (log 2 ( k ))) and a small cut-point error of 1/2 n after k steps of binary search. Traditionally, the optimal C value is determined by stepping through all possible C values. This search is uninformed because there is no indication of the search direction. To address this issue, we derive the expectation of the F-Measure (aka F 1 score); and use it to guide the search process. Specifically, by comparing the F-Measure at the current cut-point with the F-Measure at expected cut-point, we can use the information to adjust C dynamically towards the optimal cut-point, resulting in optimal model performance. Our results on two classifiers trained from disease classification datasets suggest that the algorithm is robust and efficient, as compared to the traditional methods.