Knowledge Distillation Based Recommendation Systems: A Comprehensive Survey

Deep learning-driven deep recommendation systems have achieved remarkable success in recent years. However, the deployment of deep recommendation models on resource-constrained equipment and systems (e.g., mobile devices and embedded systems) is a significant challenge. To overcome this challenge, knowledge distillation has been introduced into the design of deep recommendation algorithms as a typical model compression and acceleration technique, and has gradually attracted the attention of both academia and industry. In this survey, the main efforts are: (1) to discuss the basic concepts, necessity and operation mechanism issues related to knowledge distillation recommendation systems; (2) to summarize and categorize existing representative knowledge distillation recommendation methods according to the viewpoint of knowledge distillation, and then analyze and elaborate their representative knowledge distillation recommendation methods in depth; (3) to introduce the analysis and discussion done by industry on how knowledge distillation can be applied to industrial recommendation systems, and summarize the ideas of applying knowledge distillation in various links of industrial recommendation systems; and (4) to present several possible future research directions for knowledge distillation recommendation systems, and summarize the results of the survey. This survey can guide researchers and practitioners to prepare and encourage further efforts to advance the field.