Studying Children Food Exposure and Food Consumption using Deep Learning

Abstract

Children's eating behavior is one of the main pillars of a healthy life. Recent studies show that eating unhealthy food is highly associated with many chronic diseases including diabetes, obesity, and cancer. Such dietary habits are often shaped by complex factors influenced by the children's home, school, and neighborhood environments. However, studying the eating behaviors of children and analyzing the factors affecting them is currently done using traditional questionnaire-based methods, which often suffer from recall and bias issues. In this thesis, we developed a comprehensive approach to study children's food exposure and food consumption using deep learning. Our approach takes as input a set of images captured automatically using wearable cameras and that contain any exposure to food, including actual food items, food outlets, and food advertisements. Our approach then relies on a series of deep learning models to 1) classify food exposure images into one or more of the above-mentioned classes, and 2) to predict the healthiness of any food items consumed in all the images, using the NOVA classification system as a measure of healthiness. To be able to train all of these models, we relied on crowdsourcing to generate the training data. First, we built the food exposure dataset that contains 3,560 images that belong to the different food exposure classes. Then, the NOVA dataset that was labeled by Tunisian expert dietitians contains 3,728 food items labeled by bounding boxes that belong to the different NOVA groups. After training our models, we evaluated them on the testing datasets. The food exposure models achieved an average f1-score of 0.96. The food item detection model achieved a mAP@0.5 of 0.90. Finally, the average f1-score of the NOVA classification model was 0.86. After validating our models, we deployed them in a real-world case study in Greater Tunisia