Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

要約

タイトル：不完全データ下での交通シグナル制御のための強化学習手法
要約：
-強化学習（RL）手法は、従来のルールベースの手法よりも交通シグナル制御タスクで優れたパフォーマンスを達成している。
-しかし、現実世界の都市シナリオでは、センサーの不足により交通状態の観測欠落が頻繁に発生するため、既存のRL手法は観測不足を伴う道路ネットワークで適用できない。
-本研究は、一部の交差点にセンサーが設置されていない実世界の環境下で交通信号制御を行うことを目的としており、交通状態を補完する2つのソリューションを提案している。
-提案手法により、従来のアプローチと比較して、様々な欠落率で一貫して優れた性能を発揮し、モデル性能への欠落データの影響についても詳しく調査を行った。

要約(オリジナル)

The emergence of reinforcement learning (RL) methods in traffic signal control tasks has achieved better performance than conventional rule-based approaches. Most RL approaches require the observation of the environment for the agent to decide which action is optimal for a long-term reward. However, in real-world urban scenarios, missing observation of traffic states may frequently occur due to the lack of sensors, which makes existing RL methods inapplicable on road networks with missing observation. In this work, we aim to control the traffic signals in a real-world setting, where some of the intersections in the road network are not installed with sensors and thus with no direct observations around them. To the best of our knowledge, we are the first to use RL methods to tackle the traffic signal control problem in this real-world setting. Specifically, we propose two solutions: the first one imputes the traffic states to enable adaptive control, and the second one imputes both states and rewards to enable adaptive control and the training of RL agents. Through extensive experiments on both synthetic and real-world road network traffic, we reveal that our method outperforms conventional approaches and performs consistently with different missing rates. We also provide further investigations on how missing data influences the performance of our model.

arxiv情報

提供元, 利用サービス

arxiv.jp, OpenAI