A mask matching approach for video segmentation on compressed data

Abstract

Video segmentation provides an easy and efficient way for video retrieval and browsing. A frame is detected as a shot change frame if its content is very different from its previous frames. The process of segmenting videos into shots is usually time consuming due to the large number of frames in the videos. In this paper, we propose a new approach for segmenting videos into shots on MPEG coded video data. This approach detects shot changes by computing the shot change probability for each frame. The MPEG coded video data are only partially decoded such that the time for decoding and processing video data frame by frame and pixel by pixel can be avoided. A set of masks for different types of MPEG coded frames (I, P, and B frames) is defined for the computation of the shot change probability. Experiments based on various parameters are performed to show a 95% of detection rate in average. With further consideration on detecting the dissolve effect, the result is improved to reach an average 98% recall and 96% precision of the detection. A video indexing tool based on this approach was implemented. The results of detected shot changes are kept such that video retrieval and browsing can be provided.