Rapid urbanization in cities like Bangalore has led to severe traffic congestion, making efficient Traffic Signal Control (TSC) essential. Multi-Agent Reinforcement Learning (MARL), often modeling each traffic signal as an independent agent using Q-learning, has emerged as a promising strategy to reduce average commuter delays. While prior work Prashant L A et. al has empirically demonstrated the effectiveness of this approach, a rigorous theoretical analysis of its stability and convergence properties in the context of traffic control has not been explored. This paper bridges that gap by focusing squarely on the theoretical basis of this multi-agent algorithm. We investigate the convergence problem inherent in using independent learners for the cooperative TSC task. Utilizing stochastic approximation methods, we formally analyze the learning dynamics. The primary contribution of this work is the proof that the specific multi-agent reinforcement learning algorithm for traffic control is proven to converge under the given conditions extending it from single agent convergence proofs for asynchronous value iteration.

翻译：班加罗尔等城市的快速城市化已导致严重的交通拥堵，使得高效的交通信号控制变得至关重要。多智能体强化学习通常将每个交通信号建模为使用Q学习的独立智能体，已成为减少平均通勤延迟的有前景策略。尽管先前Prashant L A等人的工作已通过实验证明了该方法的有效性，但其在交通控制背景下的稳定性与收敛性质的严格理论分析尚未得到探索。本文通过聚焦于该多智能体算法的理论基础来填补这一空白。我们研究了在协作交通信号控制任务中使用独立学习器所固有的收敛问题。利用随机逼近方法，我们对学习动态进行了形式化分析。本工作的主要贡献在于证明了该特定用于交通控制的多智能体强化学习算法在给定条件下能够收敛，这一结论从单智能体异步值迭代的收敛证明拓展而来。