Time-Distributed Vision Transformer Stacked With Transformer for Heart Failure Detection Based on Echocardiography Video

• Published in: IEEE access 2024, Vol.12, p.182438-182454
• Main Authors: Ramadhan, Mgs M. Luthfi, Yudha, Adyatma W. A. Nugraha, Rachmadi, Muhammad Febrian, Tandayu, Kevin Moses Hanky Jr, Liastuti, Lies Dina, Jatmiko, Wisnu
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial intelligence; Artificial neural networks; Cardiovascular diseases; Computer vision; Context; Convolutional neural networks; Data analysis; Deep learning; Echocardiography; Failure detection; Feature extraction; Frames (data processing); Hafnium; Heart; Heart failure; Machine learning; Medical imaging; pattern recognition; Sampling; Transformers; Wrapping
• ISSN: 2169-3536
• DOI: 10.1109/ACCESS.2024.3510774

Heart failure is a disease many consider to be the number one global cause of death. Despite its mortality, heart failure is still underdiagnosed clinically, especially in a remote area that experiences cardiologists shortage. Existing studies have employed artificial intelligence to help with heart failure screening and diagnosis processes based on echocardiography videos. Specifically, most existing studies use a convolutional neural network that only captures the local context of an image hindering it from learning the global context of an image. Moreover, the frame sampling algorithms only sample certain consecutive frames which makes it questionable whether the dynamic of the left ventricle during a cardiac cycle is included. This study proposed a novel deep learning model consisting of a time-distributed vision transformer stacked with a transformer. The time-distributed vision transformer learns the spatial feature and then feeds the result to the transformer to learn the temporal feature and make the final prediction afterward. We also proposed a frame sampling algorithm by squeezing the video and sampling the frame after a certain interval. Consequently, the video still contains the sequential information up until the end of the video with some in-between frames removed by a certain interval. Thus, the dynamic of the left ventricle is preserved. Our proposed method achieved an F1 score of 95.81%, 96.19%, and 93.43% for the apical four chamber view, apical two chamber view, and parasternal long axis view respectively. The overall trustworthiness of our model is quantified using the NetTrustScore and achieved a score of 0.9712, 0.9767, and 0.9527 for the apical four chamber view, apical two chamber view, and parasternal long axis view respectively.