Recognizing Food Places in Egocentric Photo-Streams Using Multi-Scale Atrous Convolutional Networks and Self-Attention Mechanism

Wearable sensors (e.g., lifelogging cameras) represent very useful tools to monitor people's daily habits and lifestyle. Wearable cameras are able to continuously capture different moments of the day of their wearers, their environment, and interactions with objects, people, and places reflecti...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2019, Vol.7, p.39069-39082
Main Authors: Sarker, Md. Mostafa Kamal, Rashwan, Hatem A., Akram, Farhan, Talavera, Estefania, Banu, Syeda Furruka, Radeva, Petia, Puig, Domenec
Format: Article
Language:English
Subjects:
atrous convolutional networks
Biomedical monitoring
Cameras
Datasets
egocentric photo-streams
Feature extraction
Food
Food intake
Food places recognition
Habits
Image classification
Image recognition
Model accuracy
Monitoring
Obesity
Restaurants
scene classification
self-attention model
Streams
visual lifelogging
Visualization
Wearable technology
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Wearable sensors (e.g., lifelogging cameras) represent very useful tools to monitor people's daily habits and lifestyle. Wearable cameras are able to continuously capture different moments of the day of their wearers, their environment, and interactions with objects, people, and places reflecting their personal lifestyle. The food places where people eat, drink, and buy food, such as restaurants, bars, and supermarkets, can directly affect their daily dietary intake and behavior. Consequently, developing an automated monitoring system based on analyzing a person's food habits from daily recorded egocentric photo-streams of the food places can provide valuable means for people to improve their eating habits. This can be done by generating a detailed report of the time spent in specific food places by classifying the captured food place images to different groups. In this paper, we propose a self-attention mechanism with multi-scale atrous convolutional networks to generate discriminative features from image streams to recognize a predetermined set of food place categories. We apply our model on an egocentric food place dataset called "EgoFoodPlaces" that comprises of 43 392 images captured by 16 individuals using a lifelogging camera. The proposed model achieved an overall classification accuracy of 80% on the "EgoFoodPlaces" dataset, respectively, outperforming the baseline methods, such as VGG16, ResNet50, and InceptionV3.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2019.2902225