A Survey of Deep Learning on Mobile Devices: Applications, Optimizations, Challenges, and Research Opportunities

• Published in: Proceedings of the IEEE 2022-03, Vol.110 (3), p.334-354
• Main Authors: Zhao, Tianming, Xie, Yucheng, Wang, Yan, Cheng, Jerry, Guo, Xiaonan, Hu, Bin, Chen, Yingying
• Format: Article
• Language: English
• Subjects: Accelerators; Cloud computing; Communications systems; Deep learning; Deep learning (DL); Design methodology; Electronic devices; Hardware; hardware and software accelerator design; Human activity recognition; Machine learning; Mobile handsets; mobile security; mobile sensing; Moving object recognition; Optimization; Remote sensing; Security; Smartphones; Software; Speech recognition; Transportation
• ISSN: 0018-9219; 1558-2256
• DOI: 10.1109/JPROC.2022.3153408

Deep learning (DL) has demonstrated great performance in various applications on powerful computers and servers. Recently, with the advancement of more powerful mobile devices (e.g., smartphones and touch pads), researchers are seeking DL solutions that could be deployed on mobile devices. Compared to traditional DL solutions using cloud servers, deploying DL on mobile devices have unique advantages in data privacy, communication overhead, and system cost. This article provides a comprehensive survey for the current studies of adopting and deploying DL on mobile devices. Specifically, we summarize and compare the state-of-the-art DL techniques on mobile devices in various application domains involving vision, speech/speaker recognition, human activity recognition, transportation mode detection, and security. We generalize an optimization pipeline for bringing DL to mobile devices, including model-oriented optimization mechanisms (e.g., pruning and quantization) and nonmodel-oriented optimization mechanisms (e.g., software accelerator and hardware design). Moreover, we summarize popular DL libraries regarding their support to state-of-the-art models (software) and processors (hardware). Based on our summarization, we further provide insights into potential research opportunities for developing DL for mobile devices.