A novel solution of using mixed reality in bowel and oral and maxillofacial surgical telepresence: 3D mean value cloning algorithm

• Published in: The international journal of medical robotics + computer assisted surgery 2021-08, Vol.17 (4), p.e2224-n/a
• Main Authors: Maharjan, Arjina, Alsadoon, Abeer, Prasad, P.W.C., AlSallami, Nada, Rashid, Tarik A., Alrubaie, Ahmad, Haddad, Sami
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; augmented reality; Cloning; Consistency; Errors; Luminous intensity; Mixed reality; Occlusion; Target recognition; telepresence surgery; Telepresence systems; virtual reality; Visualization
• ISSN: 1478-5951; 1478-596X
• DOI: 10.1002/rcs.2224

Background and Aim Most of the mixed reality models used in the surgical telepresence are suffering from the discrepancies in the boundary area and spatial–temporal inconsistency due to the illumination variation in the video frames. The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and virtual hand of the remote expertise surgeon. The purpose of the proposed solution is to decrease the processing time and enhance the accuracy of merged video by decreasing the overlay and visualization error and removing occlusion and artefacts. Methodology The proposed system enhanced mean‐value cloning algorithm that helps to maintain the spatial–temporal consistency of the final composite video. The enhanced algorithm includes the three‐dimensional mean‐value coordinates and improvised mean‐value interpolant in the image cloning process, which helps to reduce the sawtooth, smudging and discolouration artefacts around the blending region. Results The accuracy in terms of overlay error of the proposed solution is improved from 1.01 to 0.80 mm, whereas the accuracy in terms of visualization error is improved from 98.8% to 99.4%. The processing time is reduced to 0.173 s from 0.211 s. The processing time and the accuracy of the proposed solution are enhanced as compared to the state‐of‐art solution. Conclusion Our solution helps make the object of interest consistent with the light intensity of the target image by adding the space distance that helps maintain the spatial consistency in the final merged video.