DDPG-Based Throughput Optimization with AoI Constraint in Ambient Backscatter-Assisted Overlay CRN

The combination of ambient backscatter (AB) communications (ABCs) and RF-powered cognitive radio networks (CRNs) deals with challenges of both energy supply and spectrum shortage, and improves the network performances. With the expansion of wireless networks, many applications raise requirements for both high-throughput and timely data. Driven by these facts, we study the long-term throughput optimization of the secondary network in the AB-assisted overlay CRN (ABO-CRN), ABCs, and CRNs with the age of information (AoI) constraint, which is a novel metric for measuring the freshness of data received by receivers. Due to the dynamic environment, complete knowledge of the environment could not be obtained. Then, the deep deterministic policy gradient (DDPG), a deep reinforcement learning (DRL) method that addresses decision issues in both continuous and discrete spaces, is deployed to address the throughput optimization. We consider the impacts of time and energy allocation on the reward when the AoI constraint can not be satisfied, and develop the corresponding reward functions. Furthermore, we analyze the impacts of the minimum throughput requirement and maximum allowable AoI on the throughput and AoI of the secondary networks in the ABO-CRN, ABCs, and CRNs. We compare the throughput optimization scheme under the AoI constraint with two baseline schemes (i.e., throughput-optimal (T-O) and AoI-optimal (A-O) baseline schemes), and the simulation results show that the throughput of the ABO-CRN is close to the optimal throughput of the T-O baseline scheme, and the AoI of the ABO-CRN is close to the optimal AoI of the A-O baseline scheme.