CoSLight: Co-optimizing Collaborator Selection and Decision-making to Enhance Traffic Signal Control
Abstract
Effective multi-intersection collaboration is pivotal for reinforcement-learning-based traffic signal control to alleviate congestion. Existing work mainly chooses neighboring intersections as collaborators. However, quite an amount of congestion, even some wide-range congestion, is caused by non-neighbors failing to collaborate. To address these issues, we propose to separate the collaborator selection as a second policy to be learned, concurrently being updated with the original signal-controlling policy. Specifically, the selection policy in real-time adaptively selects the best teammates according to phase- and intersection-level features. Empirical results on both synthetic and real-world datasets provide robust validation for the superiority of our approach, offering significant improvements over existing state-of-the-art methods.
What life mottos have we learned from traffic signal control? Wait! What? Yes, we learned that “you are your biggest collaborator” and “Collaboration should be mutually reciprocal” from the traffic light!
Every driver would have realized that the traffic patterns in different regions (residential v.s. downtown) at different hours (midday v.s. evening peak) are quite different. The same should have been true of the traffic signal coordination! In our CoSLight, we co-optimize the collaborating policy ρ (choosing which traffic light as a collaborator) and the decision-making policy π (delivering which traffic control phase for the next timeslot). It turns out to be working very, Very, VERY well: our CoSLight can choose different sets of traffic lights for collaboration at different times!
And we also learned life philosophies from the traffic light agents, very (ironically) deep: (1) When we drop the traffic light itself from being its own collaborator, the performance drops 2%: “You are your biggest collaborator”. (2) When you collaborate with me, but I don’t collaborate with you, a.k.a, not being mutually reciprocal (mathematically not being symmetric), the performance drops 300%!!!: “Collaboration should be mutually reciprocal”.
All right, “collaboration should be mutually reciprocal”: we learned the important lesson the hard way…