DRL Connects Lyapunov in Delay and Stability Optimization for Offloading Proactive Sensing Tasks of RSUs

The integration of Roadside Units (RSUs) is vital for the development of autonomous driving technologies. Challenges arise from sinking computing capabilities into RSUs and vehicles in the paradigm of Vehicle Edge Computing (VEC), particularly due to heterogeneous computation and communication capacities of network nodes and multiple sources of computing tasks (node-mounted and offloading tasks). These challenges complicate network stability from the perspective of a long-term optimization evolving over time, considering unpredictable task distribution and environmental states. To tackle these challenges, we approach the problem of partial task offloading to minimize task delay while meeting the demand of system stability over time as a dynamic long-term optimization. Utilizing Lyapunov stochastic optimization tools, we successfully decouple the long-term delay minimization and stability constraint, transforming it into a per-slot scheduling problem. Since the per-slot scheduling problem with complicated Lyapunov drift functions can not be solved by numerical optimization at each time step, our solution leverages a proposed deep reinforcement learning algorithm, leading to extensive simulations that demonstrate the superior effectiveness and efficiency of our proposal compared to existing schemes.