Realistic endoscopic image generation method using virtual-to-real image-domain translation

• Published in: Healthcare technology letters 2019-12, Vol.6 (6), p.214-219
• Main Authors: Oda, Masahiro, Tanaka, Kiyohito, Takabatake, Hirotsugu, Mori, Masaki, Natori, Hiroshi, Mori, Kensaku
• Format: Article
• Language: English
• Subjects: biological organs; Blood vessels; computerised tomography; data visualisation; deep U‐Net; endoscope insertions; endoscopes; endoscopic diagnosis; endoscopic simulation systems; endoscopic treatment; Endoscopy; high-quality image-domain translation results; image cleansing; image segmentation; image-domain translator; Light; Medical diagnosis; medical image processing; nonrealistic virtual endoscopic images; Patients; Physicians; real endoscopic images; realistic endoscopic image generation method; realistic image generation method; realistic images; rendering (computer graphics); shallow U‐Net; Simulation; Special Issue: Papers from the 13th Workshop on Augmented Environments for Computer Assisted Interventions; Surgeons; Surgical apparatus & instruments; unpaired virtual images; virtual reality
• ISSN: 2053-3713; 2053-3713
• DOI: 10.1049/htl.2019.0071

A realistic image generation method for visualisation in endoscopic simulation systems is proposed in this study. Endoscopic diagnosis and treatment are performed in many hospitals. To reduce complications related to endoscope insertions, endoscopic simulation systems are used for training or rehearsal of endoscope insertions. However, current simulation systems generate non-realistic virtual endoscopic images. To improve the value of the simulation systems, improvement of the reality of their generated images is necessary. The authors propose a realistic image generation method for endoscopic simulation systems. Virtual endoscopic images are generated by using a volume rendering method from a CT volume of a patient. They improve the reality of the virtual endoscopic images using a virtual-to-real image-domain translation technique. The image-domain translator is implemented as a fully convolutional network (FCN). They train the FCN by minimising a cycle consistency loss function. The FCN is trained using unpaired virtual and real endoscopic images. To obtain high-quality image-domain translation results, they perform an image cleansing to the real endoscopic image set. They tested to use the shallow U-Net, U-Net, deep U-Net, and U-Net having residual units as the image-domain translator. The deep U-Net and U-Net having residual units generated quite realistic images.