Off-Line Evaluation of Indoor Positioning Systems in Different Scenarios: The Experiences From IPIN 2020 Competition

• Published in: IEEE sensors journal 2022-03, Vol.22 (6), p.5011-5054
• Main Authors: Potorti, Francesco, Torres-Sospedra, Joaquin, Quezada-Gaibor, Darwin, Jimenez, Antonio Ramon, Seco, Fernando, Perez-Navarro, Antoni, Ortiz, Miguel, Zhu, Ni, Renaudin, Valerie, Ichikari, Ryosuke, Shimomura, Ryo, Ohta, Nozomu, Kurata, Takeshi, Wei, Dongyan, Ji, Xinchun, Zhang, Wenchao, Kram, Sebastian, Stahlke, Maximilian, Mutschler, Christopher, Crivello, Antonino, Barsocchi, Paolo, Girolami, Michele, Palumbo, Filippo, Chen, Ruizhi, Wu, Yuan, Li, Wei, Yu, Yue, Xu, Shihao, Liu, Tao, Kuang, Jian, Niu, Xiaoji, Yoshida, Takuto, Nagata, Yoshiteru, Fukushima, Yuto, Fukatani, Nobuya, Hayashida, Nozomi, Asai, Yusuke, Urano, Kenta, Ge, Wenfei, Lee, Nien-Ting, Fang, Shih-Hau, Jie, You-Cheng, Young, Shawn-Rong, Chien, Ying-Ren, Yu, Chih-Chieh, Ma, Chengqi, Wu, Bang, Zhang, Wei, Wang, Yankun, Fan, Yonglei, Poslad, Stefan, Selviah, David R., Wang, Weixi, Yuan, Hong, Yonamoto, Yoshitomo, Yamaguchi, Masahiro, Kaichi, Tomoya, Zhou, Baoding, Liu, Xu, Gu, Zhining, Yang, Chengjing, Wu, Zhiqian, Xie, Doudou, Huang, Can, Zheng, Lingxiang, Peng, Ao, Jin, Ge, Wang, Qu, Luo, Haiyong, Xiong, Hao, Bao, Linfeng, Zhang, Pushuo, Zhao, Fang, Yu, Chia-An, Hung, Chun-Hao, Antsfeld, Leonid, Jiang, Haitao, Xia, Ming, Yan, Dayu, Li, Yuhang, Dong, Yitong, Silva, Ivo, Pendao, Cristiano, Meneses, Filipe, Nicolau, Maria Joao, Costa, Antonio, Moreira, Adriano, De Cock, Cedric, Plets, David, Opiela, Miroslav, Dzama, Jakub, Zhang, Liqiang, Li, Hu, Chen, Boxuan, Liu, Yu, Yean, Seanglidet, Lim, Bo Zhi, Teo, Wei Jie, Lee, Bu Sung, Oh, Hong Lye
• Format: Article
• Language: English
• Subjects: Algorithms; Competition; Engineering Sciences; evaluation; foot-mounted IMU; Indoor environments; Indoor navigation; Indoor positioning and navigation; positioning in industrial scenarios and factories; Received signal strength indicator; Recurrent neural networks; Satellite broadcasting; Sensor phenomena and characterization; smartphone-based positioning; State-of-the-art reviews; Testing; vehicle-positioning
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2021.3083149

Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1 m for the Smartphone Track and 0.5 m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements.