A More Scalable Deep-learning Processing Unit For Depthwise Separable Convolution

Due to the excellent energy efficiency and real-time performance, FPGA has gradually become an important computing platform for CNN inference. However, most FPGA based Deep-learning Processing Units (DPU) are not scalable enough to cope with the rapid changes in both operator type and network structure of convolutional neural networks (CNNs). To solve this problem, we proposed the Dataflow Driven Multi-core Architecture to improve the scalability of DPU. It implements different computational functions in various modules, which are connected by the streaming structures. Firstly, we designed the Basic-DPU based on the architecture, which is very efficient for standard convolution such as VGG, SSD, etc. To verify the architecture’s scalability, we then added a function module into Basic-DPU to obtain the Extended-DPU, which can accelerate both standard convolution and depthwise separable convolution in high computational efficiency. Finally, the Basic-DPU and Extended-DPU are implemented and evaluated on Xilinx xczu9eg. The experimental results show that their FPGA resource consumptions are almost the same. For the standard convolution, the actual performance reaches 470.3GOPS and 471.5GOPS. With the same test algorithm, the computational efficiency is over 90% for both of them, which is almost 1.69 times higher than the equivalent FPGA implementation. For depthwise separable convolution, their actual performance reaches 183.3GOPS and 245.2GOPS. The computational efficiency of Extended-DPU is 1.3 times that of Basic-DPU and 2.1 times that of the peer FPGA implementation.