A Neuromorphic Vision-Based Measurement for Robust Relative Localization in Future Space Exploration Missions

• Published in: IEEE transactions on instrumentation and measurement 2024-01, Vol.73, p.1-1
• Main Authors: Salah, Mohammed, Chehadah, Mohammed, Humais, Muhammad, Wahbah, Mohammed, Ayyad, Abdulla, Azzam, Rana, Seneviratne, Lakmal, Zweiri, Yahya
• Format: Article
• Language: English
• Subjects: Algorithms; Computer vision; flickering landmarks; Gaussian mixture models (GMM); Helicopters; Image sensors; Kalman filters; landmark tracking Kalman filter (LTKF); Localization; Location awareness; Luminous intensity; Mars; Mars surface; Neuromorphic vision-based measurements (NVBM); Neuromorphics; Probabilistic models; Radiofrequency identification; Robustness; Sensors; Space exploration; space robotics; State estimation; Tracking; translation decoupled Kalman filter (TDKF)
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2022.3217513

Space exploration has witnessed revolutionary changes upon landing of the Perseverance Rover on the Martian surface and demonstrating the first flight beyond Earth by the Mars helicopter, Ingenuity. During their mission on Mars, Perseverance Rover and Ingenuity collaboratively explore the Martian surface, where Ingenuity scouts terrain information for rover's safe traversability. Hence, determining the relative poses between both the platforms is of paramount importance for the success of this mission. Driven by this necessity, this work proposes a robust relative localization system based on a fusion of neuromorphic vision-based measurements (NVBMs) and inertial measurements. The emergence of neuromorphic vision triggered a paradigm shift in the computer vision community, due to its unique working principle delineated with asynchronous events triggered by variations of light intensities occurring in the scene. This implies that observations cannot be acquired in static scenes due to illumination invariance. To circumvent this limitation, high frequency active landmarks are inserted in the scene to guarantee consistent event firing. These landmarks are adopted as salient features to facilitate relative localization. A novel event-based landmark identification algorithm using Gaussian Mixture Models (GMM) is developed for matching the landmarks correspondences formulating our NVBMs. The NVBMs are fused with inertial measurements in proposed state estimators, landmark tracking Kalman filter (LTKF) and translation decoupled Kalman filter (TDKF) for landmark tracking and relative localization, respectively. The proposed system was tested in a variety of experiments and has outperformed state-of-the-art approaches in accuracy and range.