Self-aware Collaborative Edge Inference with Embedded Devices for Task-oriented IIoT

The computing and communication resources of embedded devices are constrained and heterogeneous, resulting in a low quality-of-experience for compute-intensive applications in task-oriented industrial Internet of Things (IIoT), such as edge inference. To address these challenges, we first propose a model partitioning-based self-aware collaborative edge inference framework. Furthermore, the throughput-aware collaborative inference algorithm is designed for typical IIoT scenario, stacking tasks. Via jointly optimizing the partition layer and collaborative device selection, the optimal inference efficiency, maximum inference throughput, can be obtained. Finally, the performance of our proposal is demonstrated by extensive simulations and tests based on 10 Raspberry Pi 4Bs and popular models. Specifically, with the proposed algorithm, our platform reaches up to 14.77× throughput speed up for stacking tasks, which indicates that the the proposed design can improve the inference efficiency.