Urban Traffic Density Estimation Based on Ultrahigh-Resolution UAV Video and Deep Neural Network

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing 2018-12, Vol.11 (12), p.4968-4981
• Main Authors: Zhu, Jiasong, Sun, Ke, Jia, Sen, Li, Qingquan, Hou, Xianxu, Lin, Weidong, Liu, Bozhi, Qiu, Guoping
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Automobiles; Computer vision; Deep neural networks (DNNs); Density; Detection; Flight; Localization; Machine learning; Megacities; Neural networks; Recognition; Resolution; Road traffic; road traffic monitoring; Tracking; Traffic control; traffic density estimation; Traffic intersections; Traffic volume; Trucks; unmanned aerial vehicle (UAV); Unmanned aerial vehicles; Urban areas; vehicle counting; Vehicle detection; vehicle tracking
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2018.2879368

This paper presents an advanced urban traffic density estimation solution using the latest deep learning techniques to intelligently process ultrahigh-resolution traffic videos taken from an unmanned aerial vehicle (UAV). We first capture nearly an hour-long ultrahigh-resolution traffic video at five busy road intersections of a modern megacity by flying a UAV during the rush hours. We then randomly sampled over 17 K 512×512 pixel image patches from the video frames and manually annotated over 64 K vehicles to form a dataset for this paper, which will also be made available to the research community for research purposes. Our innovative urban traffics analysis solution consists of an advanced deep neural network (DNN) based vehicle detection and localization, type (car, bus, and truck) recognition, tracking, and vehicle counting over time. We will present extensive experimental results to demonstrate the effectiveness of our solution. We will show that our enhanced single shot multibox detector (Enhanced-SSD) outperforms other DNN-based techniques and that deep learning techniques are more effective than traditional computer vision techniques in traffic video analysis. We will also show that ultrahigh-resolution video provides more information that enables more accurate vehicle detection and recognition than lower resolution contents. This paper not only demonstrates the advantages of using the latest technological advancements (ultrahigh-resolution video and UAV), but also provides an advanced DNN-based solution for exploiting these technological advancements for urban traffic density estimation.