Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Food Portion Estimation via 3D Object Scaling (2404.12257v2)

Published 18 Apr 2024 in cs.CV, cs.AI, cs.LG, cs.MM, and eess.IV

Abstract: Image-based methods to analyze food images have alleviated the user burden and biases associated with traditional methods. However, accurate portion estimation remains a major challenge due to the loss of 3D information in the 2D representation of foods captured by smartphone cameras or wearable devices. In this paper, we propose a new framework to estimate both food volume and energy from 2D images by leveraging the power of 3D food models and physical reference in the eating scene. Our method estimates the pose of the camera and the food object in the input image and recreates the eating occasion by rendering an image of a 3D model of the food with the estimated poses. We also introduce a new dataset, SimpleFood45, which contains 2D images of 45 food items and associated annotations including food volume, weight, and energy. Our method achieves an average error of 31.10 kCal (17.67%) on this dataset, outperforming existing portion estimation methods. The dataset can be accessed at: https://lorenz.ecn.purdue.edu/~gvinod/simplefood45/ and the code can be accessed at: https://gitlab.com/viper-purdue/monocular-food-volume-3d

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023.
  2. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.
  3. Two-view 3d reconstruction for food volume estimation. IEEE Transactions on Multimedia, 19(5):1090–1099, 2017.
  4. Image-based estimation of real food size for accurate food calorie estimation. Proceedings of the 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pages 274–279, 2019.
  5. Food volume estimation for quantifying dietary intake with a wearable camera. Proceedings of the 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks, pages 110–113, 2018a.
  6. Food volume estimation for quantifying dietary intake with a wearable camera. Proceedings of the 15th International Conference on Wearable and Implantable Body Sensor Networks, pages 110–113, 2018b.
  7. Dpf-nutrition: Food nutrition estimation via depth prediction and fusion. Foods, 12(23), 2023.
  8. Multiple view geometry in computer vision. Cambridge University Press, 2003.
  9. Multi-task image-based dietary assessment for food recognition and portion size estimation. 2020 IEEE Conference on Multimedia Information Processing and Retrieval, pages 49–54, 2020.
  10. An end-to-end food image analysis system. Electronic Imaging, 2021(8):285–1, 2021.
  11. Long-tailed food classification. Nutrients, 15(12):2751, 2023.
  12. 3d localization of circular feature in 2d image and application to food volume estimation. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 4545–4548, 2012.
  13. Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera. Public health nutrition, 17(8):1671–1681, 2014.
  14. YOLO by Ultralytics, 2023.
  15. Segment anything. arXiv:2304.02643, 2023.
  16. 3d reconstruction and volume estimation of food using stereo vision techniques. Proceedings of the 2021 IEEE 21st International Conference on Bioinformatics and Bioengineering, pages 1–4, 2021.
  17. Ep n p: An accurate o (n) solution to the p n p problem. International journal of computer vision, 81:155–166, 2009.
  18. The dietary patterns methods project: synthesis of findings across cohorts and relevance to dietary guidance. The Journal of nutrition, 145(3):393–402, 2015.
  19. Food volume estimation based on deep learning view synthesis from a single depth map. Nutrients, 10(12):2005, 2018.
  20. Depth estimation based on a single close-up image with volumetric annotations in the wild: A pilot study. Proceedings of the 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pages 513–518, 2019.
  21. Image-Based Food Classification and Volume Estimation for Dietary Assessment: A Review. IEEE Journal of Biomedical and Health Informatics, 24(7):1926–1939, 2020.
  22. An improved encoder-decoder framework for food energy estimation. Proceedings of the 8th International Workshop on Multimedia Assisted Dietary Management, pages 53–59, 2023.
  23. Improving dietary assessment via integrated hierarchy food classification. 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), pages 1–6, 2021a.
  24. Visual aware hierarchy based food recognition. Proceedings of the International conference on pattern recognition, pages 571–598, 2021b.
  25. Usda food and nutrient database for dietary studies (fndds), 5.0. Procedia Food Science, 2:99–112, 2013. 36th National Nutrient Databank Conference.
  26. Karl Pearson. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559–572, 1901.
  27. Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice. British Journal of Nutrition, 101(S2):S73–S85, 2009.
  28. Recognition and volume estimation of food intake using a mobile device. Proceedings of the 2009 Workshop on Applications of Computer Vision, pages 1–8, 2009.
  29. Food volume estimation in a mobile phone based dietary assessment system. Proceedings of the 2012 8th International Conference on Signal Image Technology and Internet Based Systems, pages 988–995, 2012.
  30. Revopoint. Pop 2 3d scanner (infrared light — precision 0.05mm).
  31. Towards learning food portion from monocular images with cross-domain feature adaptation. Proceedings of 2021 IEEE 23rd International Workshop on Multimedia Signal Processing, pages 1–6, 2021a.
  32. An integrated system for mobile image-based dietary assessment. Proceedings of the 3rd Workshop on AIxFood, page 19–23, 2021b.
  33. An end-to-end food portion estimation framework based on shape reconstruction from monocular image. Proceedings of 2023 IEEE International Conference on Multimedia and Expo, pages 942–947, 2023.
  34. Foodverse: A dataset of 3d food models for nutritional intake estimation. Journal of Computational Vision and Imaging Systems, 8(1):23–26, 2023.
  35. Nutrition5k: Towards automatic nutritional understanding of generic food. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8903–8911, 2021.
  36. Image based food energy estimation with depth domain adaptation. Proceedings of 2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval, pages 262–267, 2022.
  37. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 803–814, 2023.
  38. Model-based food volume estimation using 3d pose. Proceedings of the 2013 IEEE International Conference on Image Processing, pages 2534–2538, 2013a.
  39. Image-based food volume estimation. Proceedings of the 5th international workshop on Multimedia for cooking & eating activities, pages 75–80, 2013b.
  40. Image-based food portion size estimation using a smartphone without a fiducial marker. Public health nutrition, 22(7):1180–1192, 2019.
  41. Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now