U-LanD: Uncertainty-Driven Video Landmark Detection

• Published in: IEEE transactions on medical imaging 2022-04, Vol.41 (4), p.793-804
• Main Authors: Jafari, Mohammad H., Luong, Christina, Tsang, Michael, Gu, Ang Nan, Van Woudenberg, Nathan, Rohling, Robert, Tsang, Teresa, Abolmaesumi, Purang
• Format: Article
• Language: English
• Subjects: Bayes methods; Bayes Theorem; Bayesian analysis; Deep learning; echocardiography; Frames (data processing); Heating systems; Humans; Labels; landmark detection; Measurement uncertainty; Predictive models; sparse training labels; Task analysis; Training; Ultrasonography; Uncertainty; uncertainty estimation; Video; video analysis; Video Recording - methods
• ISSN: 0278-0062; 1558-254X
• DOI: 10.1109/TMI.2021.3123547

This paper presents U-LanD, a framework for automatic detection of landmarks on key frames of the video by leveraging the uncertainty of landmark prediction. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in {R}^{{2}} score, with almost no overhead imposed on the model size.