Hardware for Quantized Mixed-Precision Deep Neural Networks

Due to network and privacy issues, there is a push to perform Deep Learning (DL) computations on the edge rather than on the cloud. However, state-of-the-art Deep Neural Networks (DNNs) require vast resources that are limited on edge devices. This limitation has brought the need to design custom hardware to implement DL-specific optimizations. Traditionally DNNs have run on high-precision number formats; however, DNNs are surprisingly robust and do not require this much precision. Instead, using quantization, networks can run on low-bit widths while achieving fair accuracy. This finding suggests that edge devices can run resource-efficient DNNs at the cost of accuracy. In addition, layers within a network do not require the same precision. Therefore, a further optimization suggests using mixed-precision over uniform quantization. A comparison is made between an architecture—designed to support mixed-precision DNNs in hardware—and its single-precision counterpart.