Refined Image Segmentation for Calorie Estimation of Multiple-dish food items
Parth Poply, J. Angel Arul Jothi
Abstract
This paper proposes a Computer Vision and Deep Learning-based system for predicting the calorie contents of multiple-dish food items by taking their top view images. The system leverages Convolutional Neural Networks (CNNs) through current research in Object Detection and Semantic Segmentation to carry out a refined Image Segmentation procedure to loosely simulate Instance Segmentation, in which each pixel is identified from the instance of objects for each object detected in an image. The identified pixels are in the form of masks/segmentations associated with distinct food items. Then, these output masks are used for carrying out volume and mass estimation for the detected food items using a reference object. Using the estimated volume, mass, and other prior known information, the calories for food items are estimated through a calorie table lookup. The system developed in this research accomplishes transfer learning by also evaluating the results on previously unseen data. Upon evaluation, the system achieves a mean average precision (mAP) of 89.30% for object detection and a percentage accuracy of 93.06% for calorie prediction.