Multi-Agent Reinforcement Learning for Distributed Optimization of Large-Scale Systems

Abstract

Large-scale computational science is a pressing issue in distributed optimization involving large dimensionality, decentralized decision making and dynamical system behavior. Traditional centralized optimization techniques have a scalability problem, communication overhead, and are not as robust in uncertain environments. To overcome these shortcomings, the current paper suggests a multi-agent reinforcement learning (MARL)-based model of distributed optimization to allow multiple agents to jointly learn about optimal policies using localized interactions and decentralized control. The proposed solution incorporates cooperative learning, reward shaping strategy, and sharing of parameters to improve the efficiency of convergence and overall system operation. Individual agents update policies on their own, and they work to a common global goal, achieving local and global optimization objectives. Large-scale dynamic environments are simulated massively to check the viability of the suggested approach. The findings publicly show that convergence speed, scalability and robustness are improved significantly over the conventional distributed optimization methods and reinforcement learning models on a single agent. In addition, the framework is efficient to deal with non-stationary and high-dimensional optimization problems at lower communication costs. These results point to the opportunities of MARL as a flexible and scalable protocol to solve complex distributed optimization tasks in real-world large-scale systems.