A method for user-customized compensation of metamorphopsia through video see-through enabled head mounted display

• Published in: Pattern recognition letters 2021-11, Vol.151, p.252-258
• Main Authors: Cimmino, Lucia, Pero, Chiara, Ricciardi, Stefano, Wan, Shaohua
• Format: Article
• Language: English
• Subjects: Apexes; Augmented Reality; Blindness; Cameras; Compensation; Distortion; Finite element method; Helmet mounted displays; Image restoration; Information processing; Real time; Real-time image warping; User-adaptive video stream processing; Video data; Vision; Visual field; Visual fields; Visual impairment
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2021.09.010

•We propose an approach to compensate the visual defects caused by metamorphopsia•Our approach enables interactive measurement of distortion in user’s visual field•We compensate the warped visual field through a real-time processing of video streams•We conducted an experiment on 17 patients affected by metamorphopsia•The results show the proposed system is able to reduce visual field distortion Advances in Augmented Reality technologies and, particularly, the availability of video see-through enabled head mounted displays (HMD), are allowing to devise new strategies to help individuals with visual impairments in daily life. In this work, an approach is proposed to compensate a serious visual impairment, known as metamorphopsia, a vision disorder characterized by deformed images. The goal is to provide patients with a digitally restored visual field, through real-time processing of video see-through streams captured from the HMD. To this regard, we present two contributions, respectively, an interactive discrete modeling of patient’s eye-specific vision distortion and a compensation of the latter by means of corresponding real-time counter-distortion of incoming frames. Our approach, indeed, maps each of the video streams acquired by the stereoscopic video see-through cameras aboard the headset on a 2D polygonal mesh which is then counter-warped by moving its vertices based on the previously built distortion model and then displayed, restored, on the HMD’s screen. First user evaluations report promising results along with usability issues related to HMD technology.