On Learning From Inaccurate and Incomplete Traffic Flow Data

• Published in: IEEE transactions on intelligent transportation systems 2018-11, Vol.19 (11), p.3698-3708
• Main Authors: Alesiani, Francesco, Moreira-Matias, Luis, Faizrahnemoon, Mahsa
• Format: Article
• Language: English
• Subjects: Automatic incident detection; Automation; clustering; ensemble learning; Filtration; fundamental diagram; Fuzzy logic; Gaussian process; local principal curves; Measurement; Multiple criterion; Noise level; Noise reduction; Outliers (statistics); Roads; Robustness; Traffic congestion; Traffic flow
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2018.2857622

Today, we live in an era where pervasive sensor networks both collect and broadcast rich digital footprints about the human mobility. However, most of this data often comes in an incomplete and/or inaccurate fashion. In this paper, we propose a knowledge discovery framework to handle such issues in the context of automatic incident detection systems fed with traffic flow data. This framework operates in three steps: 1) it clusters sensors with a novel multi-criteria distance metric tailored for this purpose, followed by a heuristic rule that labels the abnormal groups; 2) then, a spatial cross-correlation framework identifies seasonal and individual abnormal readings to perform a more fine-grained filtering; and 3) finally, we propose a novel fundamental diagram that discovers the critical density of a given road section/spot on a data-driven fashion that is resistant to both outliers and noise within the input data. Large-scale experiments were conducted over traffic flow data provided by a major Asian highway operator. The obtained results illustrate well the contributions of this framework: it drastically reduces the noise within the raw data, and it also allows determining reliable definitions of traffic states (congestion/no congestion) on a completely automated way.