Laryngeal surface reconstructions from monocular endoscopic videos: a structure from motion pipeline for periodic deformations

• Published in: International journal for computer assisted radiology and surgery 2024-09, Vol.19 (9), p.1895-1907
• Main Authors: Regef, Justin, Talasila, Likhit, Wiercigroch, Julia, Lin, R. Jun, Kahrs, Lueder A.
• Format: Article
• Language: English
• Subjects: Computer Imaging; Computer Science; Datasets; Health Informatics; Humans; Illumination; Image enhancement; Image filters; Image Processing, Computer-Assisted - methods; Image reconstruction; Imaging; Laryngoscopy - methods; Larynx - diagnostic imaging; Matching; Medical imaging; Medical instruments; Medicine; Medicine & Public Health; Motion perception; Original Article; Pattern Recognition and Graphics; Radiology; Surgery; Three dimensional models; Video; Video Recording - methods; Vision
• ISSN: 1861-6429; 1861-6410; 1861-6429
• DOI: 10.1007/s11548-024-03118-x

Purpose Surface reconstructions from laryngoscopic videos have the potential to assist clinicians in diagnosing, quantifying, and monitoring airway diseases using minimally invasive techniques. However, tissue movements and deformations make these reconstructions challenging using conventional pipelines. Methods To facilitate such reconstructions, we developed video frame pre-filtering and featureless dense matching steps to enhance the Alicevision Meshroom SfM pipeline. Time and the anterior glottic angle were used to approximate the rigid state of the airway and to collect frames with different camera poses. Featureless dense matches were tracked with a correspondence transformer across subsets of images to extract matched points that could be used to estimate the point cloud and reconstructed surface. The proposed pipeline was tested on a simulated dataset under various conditions like illumination and resolution as well as real laryngoscopic videos. Results Our pipeline was able to reconstruct the laryngeal region based on 4, 8, and 16 images obtained from simulated and real patient exams. The pipeline was robust to sparse inputs, blur, and extreme lighting conditions, unlike the Meshroom pipeline which failed to produce a point cloud for 6 of 15 simulated datasets. Conclusion The pre-filtering and featureless dense matching modules specialize the conventional SfM pipeline to handle the challenging laryngoscopic examinations, directly from patient videos. These 3D visualizations have the potential to improve spatial understanding of airway conditions.