Adaptive Radio Access Technology Selection on ESP32-based IoT Devices Using Reinforcement Learning

Abstract The rapid proliferation of Internet of Things (IoT) applications has led to the coexistence of multiple wireless connectivity technologies within the same environment. In this context, the ability to integrate various Radio Access Technologies (RATs) simultaneously is essential for enhancing system adaptability. In this paper, we implement a multi-RAT system on a resourceconstrained ESP32 module, combining Wi-Fi, Bluetooth Low Energy (BLE) and cellular networks to ensure flexible and resilient IoT connectivity. However, with multiple RATs available, the challenge of selecting the most appropriate RAT remains critical. To overcome this issue, we propose a multi-criteria adaptive decision algorithm based on Naive Reinforcement Learning (NRL). This solution dynamically selects the optimal RAT based on current network conditions and application requirements, while respecting the limited memory and processing capacities of ESP32-based devices. The practicality and effectiveness of our NRL-based selection algorithm are demonstrated in a real-world healthcare scenario that involves e-health. This demonstrates a significant increase in the performance of key metrics. The method increases Packet Delivery Ratio (PDR) as compared to using a single RAT and randomizing the selection of the RAT. It also reduces the average end-to-end time and energy consumption compared to the single RAT method and selecting RAT at random.