HD3: Distributed Dueling DQN with Discrete-Continuous Hybrid Action Spaces for Live Video Streaming

Live streaming applications are becoming increasingly popular recently, and it exposes new technical challenges compared to regular video streaming. High video quality and low latency are two main requirements in live streaming scenarios. A live streaming application needs to make bitrate and target buffer level decisions as well as sets a continuous latency limit value to skip video frames. We formulate the live streaming task as a reinforcement learning problem with discrete-continuous hybrid action spaces, then propose a novel deep reinforcement learning (DRL) algorithm HD3 which can take hybrid actions to solve it. We compare HD3 with several state-of-the-art DRL algorithms on various network environments, and the simulation results show that HD3 can outperform all the other comparison schemes. We emphasize that HD3 generates a single agent which can perform well on different network conditions and video scenes.