An Image-based AI Nutrition Analysis Platform for Food in Compartment Trays

This paper presents an image-based AI nutrition analysis platform that comprises a food image collection system and a food nutrition analysis system. The platform enables nutrition analysis of food in compartment trays. The food image collection system combines a Raspberry Pi microcontroller and a depth camera to capture images and depth information. The collected data is then uploaded to a cloud database for analysis. The food nutrition analysis system incorporates a nutrition estimation algorithm and provides users with visualized nutritional information. The proposed algorithm consists of three main processes: dish recognition, portion-size estimation, and nutrition analysis. In the dish recognition process, the input image of food in a compartment tray is used for region detection and dish recognition using instance segmentation models. We evaluated the performance of three segmentation models and found that CenterMask with VoVNetV2-99 as the backbone network outperformed others, achieving high accuracy metrics (90.14% in AP, 98.82% in AP50, 98.47% in AP75, 81.61% in APM, and 92.11% in APL). The portion-size estimation process involves preprocessing depth information to remove noise and estimate the volume of the dish. Ablation experiments demonstrated that the best performance was achieved by removing outliers and repairing zero values using the TELEA algorithm, resulting in an average coefficient of variation of 6.82%. In the dish nutrition analysis process, the information from the previous processes is used to identify the dish and determine its food nutrition using the proposed algorithms. To validate the effectiveness of our algorithms, we estimated the nutrition of dishes with varying portion sizes, totaling 14 dishes. The mean absolute error of our method was 15.68 kcal, and the mean relative error was 22.29%, demonstrating comparable accuracy to dietitians’ estimations. Additionally, our method significantly outperforms dietitians in terms of estimation speed.