EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors

Human and environment sensing are two important topics in Computer Vision and Graphics. Human motion is often captured by inertial sensors, while the environment is mostly reconstructed using cameras. We integrate the two techniques together in EgoLocate, a system that simultaneously performs human...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on graphics 2023-08, Vol.42 (4), p.1-17, Article 76
Main Authors: Yi, Xinyu, Zhou, Yuxiao, Habermann, Marc, Golyanik, Vladislav, Pan, Shaohua, Theobalt, Christian, Xu, Feng
Format: Article
Language:English
Subjects:
Animation
Computer graphics
Computing methodologies
Motion capture
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Human and environment sensing are two important topics in Computer Vision and Graphics. Human motion is often captured by inertial sensors, while the environment is mostly reconstructed using cameras. We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera. On one hand, inertial mocap suffers from large translation drift due to the lack of the global positioning signal. EgoLo-cate leverages image-based simultaneous localization and mapping (SLAM) techniquesto locate the human in the reconstructed scene. Onthe other hand, SLAM often fails when the visual feature is poor. EgoLocate involves inertial mocap to provide a strong prior for the camera motion. Experiments show that localization, a key challenge for both two fields, is largely improved by our technique, compared with the state of the art of the two fields. Our codes are available for research at https://xinyu-yi.github.io/EgoLocate/.
ISSN:0730-0301
1557-7368
DOI:10.1145/3592099