Squeezing Deep Learning into Mobile and Embedded Devices

This department provides an overview the progress the authors have made to the emerging area of embedded and mobile forms of on-device deep learning. Their work addresses two core technical questions. First, how should deep learning principles and algorithms be applied to sensor inference problems that are central to this class of computing? Second, what is required for current and future deep learning innovations to be efficiently integrated into a variety of mobile resource-constrained systems? Toward answering such questions, the authors describe phone, watch, and embedded prototypes that can locally run large-scale deep networks processing audio, images, and inertial sensor data. These prototypes are enabled with a variety of algorithmic and system-level innovations that vastly reduce conventional inference-time overhead of deep models.