VCMaker: Content-aware configuration adaptation for video streaming and analysis in live augmented reality

The emergence of edge computing has enabled mobile Augmented Reality (AR) on edge servers. We notice that the video configurations, i.e., frames per second (fps) and resolution, significantly affect the key metrics such as detection accuracy, data transmission latency and energy consumption in real...

Full description

Saved in:
Bibliographic Details
Published in:Computer networks (Amsterdam, Netherlands : 1999) Netherlands : 1999), 2021-12, Vol.200, p.108513, Article 108513
Main Authors: Chen, Ning, Zhang, Sheng, Quan, Siyi, Ma, Zhi, Qian, Zhuzhong, Lu, Sanglu
Format: Article
Language:English
Subjects:
Accuracy
Augmented reality
Configurations
Data transmission
Decisions
Deep reinforcement learning
Edge computing
Empirical analysis
Energy consumption
Frames per second
Mobile augmented reality
Mobile computing
Motion perception
Network latency
Neural networks
Object recognition
Pharmacists
Video configuration adaptation
Video transmission
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The emergence of edge computing has enabled mobile Augmented Reality (AR) on edge servers. We notice that the video configurations, i.e., frames per second (fps) and resolution, significantly affect the key metrics such as detection accuracy, data transmission latency and energy consumption in real AR application. Besides the time-varying bandwidth, we observe that the video contents, such as moving velocities of target objects, have remarkable impacts on the configuration selection. In addition, we take the energy consumption on data transmission into consideration. In this paper, we propose VCMaker, a system that generates video configuration decisions using reinforcement learning (RL). VCMaker trains a neural network model that selects configuration for future video chunks based on the collected observations. Rather than rely on any pre-programmed models, VCMaker learns to make configuration decisions solely through empirical observations of the resulting performances of historical decisions. In addition, we leverage the dynamic Region of Interest (RoI) encoding and motion vector-based object detection mechanisms to advance VCMaker. We implemented VCMaker and conducted extensive evaluations. The results show that VCMaker achieves a 20.5%–32.8% higher detection accuracy, and 25.2%–45.7% lower energy consumption than several state-of-the-art schemes.
ISSN:1389-1286
1872-7069
DOI:10.1016/j.comnet.2021.108513