Automatic Food Recognition Using Deep Convolutional Neural Networks with Self-attention Mechanism

Abstract The significance of food in human health and well-being cannot be overemphasized. Nowadays, in our dynamic life, people are increasingly concerned about their health due to increased nutritional ailments. For this reason, mobile food-tracking applications that require a reliable and robust food classification system are gaining popularity. To address this, we propose a robust food recognition model using deep convolutional neural networks with a self-attention mechanism (FRCNNSAM). By training multiple FRCNNSAM structures with varying parameters, we combine their predictions through averaging. To prevent over-fitting and under-fitting data augmentation to generate extra training data, regularization to avoid excessive model complexity was used. The FRCNNSAM model is tested on two novel datasets: Food-101 and MA Food-121. The model achieved an impressive accuracy of 96.40% on the Food-101 dataset and 95.11% on MA Food-121. Compared to baseline transfer learning models, the FRCNNSAM model surpasses performance by 8.12%. Furthermore, the evaluation on random internet images demonstrates the model's strong generalization ability, rendering it suitable for food image recognition and classification tasks.