Real-time motion tracking on a cellphone using inertial sensing and a rolling-shutter camera

• Main Authors: Mingyang Li, Byung Hyung Kim, Mourikis, Anastasios I.
• Format: Conference Proceeding
• Language: English
• Subjects: Cameras; Covariance matrices; Gyroscopes; Sensors; Time measurement; Vectors; Visualization
• ISSN: 1050-4729; 2577-087X
• DOI: 10.1109/ICRA.2013.6631248

All existing methods for vision-aided inertial navigation assume a camera with a global shutter, in which all the pixels in an image are captured simultaneously. However, the vast majority of consumer-grade cameras use rolling-shutter sensors, which capture each row of pixels at a slightly different time instant. The effects of the rolling shutter distortion when a camera is in motion can be very significant, and are not modelled by existing visual-inertial motion-tracking methods. In this paper we describe the first, to the best of our knowledge, method for vision-aided inertial navigation using rolling-shutter cameras. Specifically, we present an extended Kalman filter (EKF)-based method for visual-inertial odometry, which fuses the IMU measurements with observations of visual feature tracks provided by the camera. The key contribution of this work is a computationally tractable approach for taking into account the rolling-shutter effect, incurring only minimal approximations. The experimental results from the application of the method show that it is able to track, in real time, the position of a mobile phone moving in an unknown environment with an error accumulation of approximately 0.8% of the distance travelled, over hundreds of meters.