Incorporating Semantic Knowledge for Visual Lifelog Activity Recognition

Abstract

The advance in wearable technology has made lifelogging more feasible and more popular. Visual lifelogs collected by wearable cameras capture every single detail of individual`s life experience, offering a promising data source for deeper lifestyle analysis and better memory recall assistance. However, building a system for organizing and accessing visual lifelogs is a challenging task due to the semantic gap between visual data and semantic descriptions of life events. In this paper, we introduce semantic knowledge to reduce such a semantic gap for daily activity recognition and lifestyle understanding. We incorporate the semantic knowledge derived from external resources to enrich the training data for the proposed supervised learning model. Experimental results show that incorporating external semantic knowledge is beneficial for improving the performance of recognizing life events.