A General Neural Network Deployment Framework for Edge Devices

As the expansion of edge devices, such as local edge servers or even IoT (Internet of Things) devices, edge computing reforms data processing flow, engendering improvement in faster insights, improved response times and better bandwidth availability. However, industry encounters computational resource constraints of machine learning on edge devices, e.g., limitation of memory and processing power. To tackle this situation, we attempt to enhance flexibility of neural network models for variable hardware constraints. In this paper, we propose a paradigm of neural network combination implementation as a primitive solution; further more, we manage to build up a general neural network deployment framework for edge devices.