Iterative feedback-based models for image and video polyp segmentation

• Published in: Computers in biology and medicine 2024-07, Vol.177, p.108569, Article 108569
• Main Authors: Wan, Liang, Chen, Zhihao, Xiao, Yefan, Zhao, Junting, Feng, Wei, Fu, Huazhu
• Format: Article
• Language: English
• Subjects: Algorithms; Colonic Polyps - diagnostic imaging; Colonoscopy - methods; Colorectal carcinoma; Colorectal Neoplasms - diagnostic imaging; Deep Learning; Defocusing; Feedback; Humans; Image Interpretation, Computer-Assisted - methods; Image polyp segmentation; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Iterative feedback; Medical imaging; Modules; Polyps; Segmentation; Video polyp segmentation
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108569

Accurate segmentation of polyps in colonoscopy images has gained significant attention in recent years, given its crucial role in automated colorectal cancer diagnosis. Many existing deep learning-based methods follow a one-stage processing pipeline, often involving feature fusion across different levels or utilizing boundary-related attention mechanisms. Drawing on the success of applying Iterative Feedback Units (IFU) in image polyp segmentation, this paper proposes FlowICBNet by extending the IFU to the domain of video polyp segmentation. By harnessing the unique capabilities of IFU to propagate and refine past segmentation results, our method proves effective in mitigating challenges linked to the inherent limitations of endoscopic imaging, notably the presence of frequent camera shake and frame defocusing. Furthermore, in FlowICBNet, we introduce two pivotal modules: Reference Frame Selection (RFS) and Flow Guided Warping (FGW). These modules play a crucial role in filtering and selecting the most suitable historical reference frames for the task at hand. The experimental results on a large video polyp segmentation dataset demonstrate that our method can significantly outperform state-of-the-art methods by notable margins achieving an average metrics improvement of 7.5% on SUN-SEG-Easy and 7.4% on SUN-SEG-Hard. Our code is available at https://github.com/eraserNut/ICBNet. •Video polyp segmentation is challenging due to camera shake and defocused frames.•The iterative feedback model is effectively applied to video polyp segmentation.•A novel frame selection and mask warp module are proposed in our method.•Our method achieved an improvement of approximately 7% on the public dataset.