Adaptive Load Balancing for Industrial Edge Computing Systems: An AxTD3-Deep Reinforcement Learning Approach

Introducing edge computing into smart manufacturing can enhance factory efficiency and productivity. By leveraging a central scheduler to connect Edge Servers (ESs) in these factories, resource sharing can be achieved. However, the unpredictable nature of task offloading from factory IoT devices results in varying task loads at each ES, expanding the action space and complicating task scheduling coordination, thus impeding effective load balancing. To address this challenge, we propose an AxTD3-Deep Reinforcement Learning (DRL) method to balance the system while reducing system latency. Firstly, we consider that each ES has multiple virtual machines and propose a workload balancing algorithm to ensure more balanced computation among the virtual machines of each ES. Next, we construct this system as a reinforcement learning model and analyze its state and action spaces. Based on this analysis, we modify the system's states and actions to reduce its complexity without compromising utility. We then design the AxTD3-DRL to balance the system, i.e., A2TD3 and A3TD3, dividing a neural network into several parallel sub-networks to further reduce the action space and state space, thereby accelerating convergence. Finally, we compare the designed method with classic DRL algorithms (e.g., SAC, TD3) and heuristic approaches (e.g., PSO). The results show that our proposed AxTD3 algorithm not only balances the load across ESs but also reduces the average system latency.