Reinforcement Learning for True Adaptive Traffic Signal Control

• Published in: Journal of transportation engineering 2003-05, Vol.129 (3), p.278-285
• Main Authors: Abdulhai, Baher, Pringle, Rob, Karakoulas, Grigoris J
• Format: Article
• Language: English
• Subjects: Applied sciences; Buildings. Public works; Design; Engineering; Exact sciences and technology; Ground, air and sea transportation, marine construction; Highway transportation; Road operations (signalization, lighting, safety and accessories, snow clearance, acoustical panel, etc.); TECHNICAL PAPERS; Transportation infrastructure; Transportation planning, management and economics
• ISSN: 0733-947X; 1943-5436
• DOI: 10.1061/(ASCE)0733-947X(2003)129:3(278)

The ability to exert real-time, adaptive control of transportation processes is the core of many intelligent transportation systems decision support tools. Reinforcement learning, an artificial intelligence approach undergoing development in the machine-learning community, offers key advantages in this regard. The ability of a control agent to learn relationships between control actions and their effect on the environment while pursuing a goal is a distinct improvement over prespecified models of the environment. Prespecified models are a prerequisite of conventional control methods and their accuracy limits the performance of control agents. This paper contains an introduction to Q-learning, a simple yet powerful reinforcement learning algorithm, and presents a case study involving application to traffic signal control. Encouraging results of the application to an isolated traffic signal, particularly under variable traffic conditions, are presented. A broader research effort is outlined, including extension to linear and networked signal systems and integration with dynamic route guidance. The research objective involves optimal control of heavily congested traffic across a two-dimensional road network-a challenging task for conventional traffic signal control methodologies.