Automated segmentation of free-lying cell nuclei in Pap smears for malignancy-associated change analysis

• Main Authors: Moshavegh, R., Bejnordi, Babak Ehteshami, Mehnert, A., Sujathan, K., Malm, P., Bengtsson, E.
• Format: Conference Proceeding
• Language: English
• Subjects: Automated algorithms; Automated screening; Automated segmentation; Brightfield; Cell nucleus; Cervical cancers; Change analysis; Dice coefficient; Epithelial cells; Gray scale; Image segmentation; Manual segmentation; Microscope images; Nuclear measurements; Pap smear; Robustness; Segmentation algorithms; Sensitivity; Sensitivity and specificity; Shape; Transforms; Watershed segmentation
• ISSN: 1094-687X; 1557-170X; 1558-4615; 2694-0604
• DOI: 10.1109/EMBC.2012.6347208

This paper presents an automated algorithm for robustly detecting and segmenting free-lying cell nuclei in bright-field microscope images of Pap smears. This is an essential initial step in the development of an automated screening system for cervical cancer based on malignancy associated change (MAC) analysis. The proposed segmentation algorithm makes use of gray-scale annular closings to identify free-lying nuclei-like objects together with marker-based watershed segmentation to accurately delineate the nuclear boundaries. The algorithm also employs artifact rejection based on size, shape, and granularity to ensure only the nuclei of intermediate squamous epithelial cells are retained. An evaluation of the performance of the algorithm relative to expert manual segmentation of 33 fields-of-view from 11 Pap smear slides is also presented. The results show that the sensitivity and specificity of nucleus detection is 94.71% and 85.30% respectively, and that the accuracy of segmentation, measured using the Dice coefficient, of the detected nuclei is 97.30±1.3%.