Publications-Theses

Title基於電影拍攝手法之電影場景情緒探勘
Emotion Discovery of Movie Content Based on Film Grammar
Creator廖家慧
Liao, Chia Hui
Contributor沈錳坤
Shan, Man Kwan
廖家慧
Liao, Chia Hui
Key Words內涵式分析
拍攝手法
電影場景
視聽覺特徵
情緒
content-based analysis
film grammar
movie scene
audiovisual features
emotion
affective classification
Date2007
Date Issued9-Apr-2010 13:17:50 (UTC+8)
Summary數位化的今天,電影逐漸成為人們日常生活的一部份,電影資料的內涵式分析也成為目前重要的研究主題。透過電影拍攝手法,我們知道電影視聽覺特徵與情緒之間有密不可分的關係。因此,在本研究中,我們希望利用探勘電影視聽覺特徵與情緒的關聯來達到自動判斷電影場景的情緒。

首先,先由人工標記訓練場景的情緒,之後,我們對所有的場景擷取定義的六類特徵值。特徵值包括電影場景的顏色、燈光、影片速度、特寫鏡頭、聲音和字幕六類。最後,我們利用Mixed Media Graph演算法來探勘場景情緒與特徵值之間的關聯,達到自動判斷電影場景情緒的功能。實驗結果顯示,準確率最高可達到70%。
Movies play an important role in our life nowadays. How to analyze the emotional content of movies becomes one of the major issues. Based on film grammar, there are many audiovisual cues in movies helpful for detecting the emotions of scenes. In this research, we investigate the discovery of the relationship between audiovisual cues and emotions of scenes and the automatic emotion annotation of scenes is achieved.

First, the training scenes are labeled with the emotions manually. Second, six classes of audiovisual features are extracted from all scenes. These classes of features consist of color, light, tempo, close-up, audio, and textual. Finally, the graph-based approach, Mixed Media Graph is modified to mine the association between audiovisual features and emotions of the scenes. The experiments show that the accuracy achieves 70%.
參考文獻 [1] B. Adams, C. Dorai, and S.Venkatesh, “Toward Automatic Extraction of Expressive Elements from Motion Pictures: Tempo,” IEEE Transactions on Multimedia, Vol. 4, No. 4, pp. 472-481, December 2002.
[2] D. Arijon, Grammar of the Film Language. CA: Silman-James Press, 1976.
[3] Christopher J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Journal of Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121-167, 1998.
[4] A. R. Damasio, The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace, 1999.
[5] R. Dietz and A. Lang, “Affective Agents: Effects of Agent Affect on Arousal, Attention, Liking and Learning,” Proceedings of Cognitive Technology Conference, San Francisco, CA, 1999.
[6] N. Dimitrova, J. Martino, H. Elenbaas, and L. Agnihotri, “Color SuperHistograms for Video Representation,” IEEE International Conference on Image Processing (ICIP ‘99), Kobe, Japan, Vol. 3, pp. 314-318, October 1999.
[7] P. Ekman, “Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion,” Journal of Personality and Social Psychology, Vol. 54, No. 4, pp. 712-717, October 1987.
[8] L. Giannetti, Understanding Movies, 10th ed. Englewood Cliffs, New Jersey: Prentice Hall, 2005.
[9] A. Hanjalic and L. Q. Xu, “Extracting Moods from Pictures and Sounds: Towards truly personalized TV,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 90-100, March 2006.
[10] A. Hanjalic and L. Q. Xu, “Affective Video Content Representation and Modeling,” IEEE Transaction on Multimedia, Vol. 7, No. 1, pp. 143-154, February 2005.
[11] A. Hanjalic and L. Q. Xu, “User-oriented Affective Video Content Analysis,” Proceedings of IEEE CBAIBL, Kauai, Hawaii, pp. 50-57, December 2001.
[12] H. B. Kang, “Affective Content Retrieval from Video with Relevance Feedback,” International Conference on Asian Digital Libraries, Kuala Lumpur, Malaysia, pp. 243-252, December 2003.
[13] H. B. Kang, “Affective Content Detection using HMMs,” Proceedings of ACM International Conference on Multimedia, Berkeley, California, U.S.A, pp. 259-262, November 2003.
[14] G. Kirouac, Les émotions: Monographies de psychologie. Sillery: Presses de l’Université du Québec, 1992.
[15] F. F. Kuo, M. F. Chiang, M. K. Shan, and S. Y. Lee, “Emotion-based Music Recommendation by Association Discovery from Film Music,” Proceedings of ACM International Conference on Multimedia, Singapore, pp. 507-510, November 2005.
[16] Y. Li, S. H. Lee, C. H. Yeh, and C. C. J. Kuo, ”Techniques for Movie Content Analysis and Skimming,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 79-89, March 2006.
[17] L. Lu, H. Jiang, and H. J. Zhang, "A Robust Audio Classification and Segmentation Method," Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 203-211, September 2001.
[18] L. Lu, H. J. Zhang, and H. Jiang, "Content Analysis for Audio Classification and Segmentation," IEEE Transaction on Speech and Audio Processing, Vol. 10, No. 7, pp. 504-516, October 2002.
[19] F. H. Mahnke, Color, Environmental and Human Response. New York: Van Nostrand Reinhold, 1996.
[20] S. Moncrieff, C. Dorai, and S. Venkatesh, “Affect Computing in Film through Sound Energy Dynamics,” Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 525-527, September 2001.
[21] A. Ortony, G. Clore, and A. Collins, The Cognitive Structure of Emotions. New York: Oxford University Press, 1988.
[22] C. E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning. Urbana, IL: University of Illinois Press, 1957.
[23] J. Y. Pan, H. J. Yang, P. Duygulu, and C. Faloutsos, “Automatic Image Captioning,” Proceedings of IEEE International Conference on Multimedia and Expo (ICME ’04), Taipei, Taiwan, pp. 1987-1990, June 2004.
[24] J. Y. Pan, H. J. Yang, C. Faloutsos, and P. Duygulu, “Automatic Multimedia Cross-modal Correlation Discovery,” Proceedings of ACM International Conference on Knowledge Discovery on Database (SIGKDD ‘04), Seattle, Washington, pp. 653-658, August 2004.
[25] D. S. Park, J. S. Park, and J. H. Han, “Image Indexing Using Color Histogram in the CIELUV Color Space,” Proceedings of 5th Japan-Korea Workshop on Computer Vision, Korea, pp. 126-132, 1999.
[26] G. Peeters, “A Large Set of Audio Features for Sound Description (Similarity and Classification),” in the CUIDADO project. Technical report, Ircam, Paris, France, April 2004.
[27] Z. Rasheed, Y. Sheikh, and M. Shah, “On the Use of Computable Features for Film Classification,” IEEE Transaction on Circuits and Systems for Video Technology (CSVT), Vol. 15, No. 1, pp. 52-64, January 2005.
[28] J. A. Russell and A. Mehrabian, “Evidence for a Three-Factor Theory of Emotions,” Journal of Research in Personality, Vol. 11, pp. 273-294, 1977.
[29] J. Saunders, “Real-Time Discrimination of Broadcast Speech/Music,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’96), Atlanta, Ga, Vol. 2, pp. 993-996, May 1996.
[30] E. Scheirer and M. Slaney, “Construction and Evaluation of a Robust Multifeature Speech/Music Discriminator”, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’97), Munich, Germany, Vol. 2, pp. 1331-1334, April 1997.
[31] R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford University Press, 1989.
[32] H. L. Wang and L. F. Cheong, “Affective Understanding in Film,” IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol. 16, No. 6, pp. 689-704, June 2006.
[33] C. Y. Wei, N. Dimitrova, and S.F. Chang, “Color-Mood Analysis of Films on Syntactic and Psychological Models,” Proceedings of IEEE International Conference on Multimedia and Expo. (ICME ’04), Taipei, Taiwan, pp. 831-834, June 2004.
[34] H. Zettl, Sight Sound Motion: Applied Media Aesthetics, 3rd ed. Belmont, CA: Wadsworth Publishing Company, 1998.
[35] http://www.intel.com/technology/computing/opencv/index.htm
[36] http://eqi.org/fw.htm
Description碩士
國立政治大學
資訊科學學系
94753027
96
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0094753027
Typethesis
dc.contributor.advisor 沈錳坤zh_TW
dc.contributor.advisor Shan, Man Kwanen_US
dc.contributor.author (Authors) 廖家慧zh_TW
dc.contributor.author (Authors) Liao, Chia Huien_US
dc.creator (作者) 廖家慧zh_TW
dc.creator (作者) Liao, Chia Huien_US
dc.date (日期) 2007en_US
dc.date.accessioned 9-Apr-2010 13:17:50 (UTC+8)-
dc.date.available 9-Apr-2010 13:17:50 (UTC+8)-
dc.date.issued (上傳時間) 9-Apr-2010 13:17:50 (UTC+8)-
dc.identifier (Other Identifiers) G0094753027en_US
dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/38537-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學學系zh_TW
dc.description (描述) 94753027zh_TW
dc.description (描述) 96zh_TW
dc.description.abstract (摘要) 數位化的今天,電影逐漸成為人們日常生活的一部份,電影資料的內涵式分析也成為目前重要的研究主題。透過電影拍攝手法,我們知道電影視聽覺特徵與情緒之間有密不可分的關係。因此,在本研究中,我們希望利用探勘電影視聽覺特徵與情緒的關聯來達到自動判斷電影場景的情緒。

首先,先由人工標記訓練場景的情緒,之後,我們對所有的場景擷取定義的六類特徵值。特徵值包括電影場景的顏色、燈光、影片速度、特寫鏡頭、聲音和字幕六類。最後,我們利用Mixed Media Graph演算法來探勘場景情緒與特徵值之間的關聯,達到自動判斷電影場景情緒的功能。實驗結果顯示,準確率最高可達到70%。
zh_TW
dc.description.abstract (摘要) Movies play an important role in our life nowadays. How to analyze the emotional content of movies becomes one of the major issues. Based on film grammar, there are many audiovisual cues in movies helpful for detecting the emotions of scenes. In this research, we investigate the discovery of the relationship between audiovisual cues and emotions of scenes and the automatic emotion annotation of scenes is achieved.

First, the training scenes are labeled with the emotions manually. Second, six classes of audiovisual features are extracted from all scenes. These classes of features consist of color, light, tempo, close-up, audio, and textual. Finally, the graph-based approach, Mixed Media Graph is modified to mine the association between audiovisual features and emotions of the scenes. The experiments show that the accuracy achieves 70%.
en_US
dc.description.tableofcontents ABSTRACT IN CHINESE........................................i
ABSTRACT..................................................ii
ACKNOLEGEMENTS............................................iv
TABLE OF CONTENTS.........................................vi
LIST OF TABLES..........................................viii
LIST OF FIGURES...........................................ix

CHAPTER 1 Introduction.....................................1

CHAPTER 2 Related Works....................................4
2.1 Affective Classification in Film Domain.............4

CHAPTER 3 Feature Extraction...............................7
3.1 Introduction........................................7
3.2 Emotion Discovery from Scenes......................10
3.3 Virtual Features...................................11
3.4 Audio Features.....................................18
3.5 Textual Feature....................................26

CHAPTER 4 Emotion Discovery...............................28
4.1 Emotion Taxonomy...................................28
4.2 Mixed Media Graph (MMG)............................32
4.3 Scene Affinity Graph (SAG).........................34
4.3.1 SAG with Separate Visual Representation.......35
4.3.2 SAG with Integrated Visual Representation.....40

CHAPTER 5 Experiments and Results.........................45
5.1 Implementation.....................................45
5.1.1 Preprocessing.................................47
5.1.2 Visual Feature Extraction.....................47
5.1.3 Audio Feature Extraction......................49
5.1.4 Textual Feature Extraction....................49
5.1.5 Emotion Discovery.............................52
5.2 Experiments on SAG with SVR........................52
5.3 Experiments on SAG with IVR........................57
5.4 Discussion.........................................62

CHAPTER 6 Conclusions.....................................64
6.1 Summary............................................64
6.2 Future Work........................................65

REFERENCES................................................66
zh_TW
dc.format.extent 44318 bytes-
dc.format.extent 60276 bytes-
dc.format.extent 101651 bytes-
dc.format.extent 19029 bytes-
dc.format.extent 17069 bytes-
dc.format.extent 18883 bytes-
dc.format.extent 254481 bytes-
dc.format.extent 226909 bytes-
dc.format.extent 72810 bytes-
dc.format.extent 15175 bytes-
dc.format.extent 22694 bytes-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.language.iso en_US-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0094753027en_US
dc.subject (關鍵詞) 內涵式分析zh_TW
dc.subject (關鍵詞) 拍攝手法zh_TW
dc.subject (關鍵詞) 電影場景zh_TW
dc.subject (關鍵詞) 視聽覺特徵zh_TW
dc.subject (關鍵詞) 情緒zh_TW
dc.subject (關鍵詞) content-based analysisen_US
dc.subject (關鍵詞) film grammaren_US
dc.subject (關鍵詞) movie sceneen_US
dc.subject (關鍵詞) audiovisual featuresen_US
dc.subject (關鍵詞) emotionen_US
dc.subject (關鍵詞) affective classificationen_US
dc.title (題名) 基於電影拍攝手法之電影場景情緒探勘zh_TW
dc.title (題名) Emotion Discovery of Movie Content Based on Film Grammaren_US
dc.type (資料類型) thesisen
dc.relation.reference (參考文獻) [1] B. Adams, C. Dorai, and S.Venkatesh, “Toward Automatic Extraction of Expressive Elements from Motion Pictures: Tempo,” IEEE Transactions on Multimedia, Vol. 4, No. 4, pp. 472-481, December 2002.zh_TW
dc.relation.reference (參考文獻) [2] D. Arijon, Grammar of the Film Language. CA: Silman-James Press, 1976.zh_TW
dc.relation.reference (參考文獻) [3] Christopher J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Journal of Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121-167, 1998.zh_TW
dc.relation.reference (參考文獻) [4] A. R. Damasio, The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace, 1999.zh_TW
dc.relation.reference (參考文獻) [5] R. Dietz and A. Lang, “Affective Agents: Effects of Agent Affect on Arousal, Attention, Liking and Learning,” Proceedings of Cognitive Technology Conference, San Francisco, CA, 1999.zh_TW
dc.relation.reference (參考文獻) [6] N. Dimitrova, J. Martino, H. Elenbaas, and L. Agnihotri, “Color SuperHistograms for Video Representation,” IEEE International Conference on Image Processing (ICIP ‘99), Kobe, Japan, Vol. 3, pp. 314-318, October 1999.zh_TW
dc.relation.reference (參考文獻) [7] P. Ekman, “Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion,” Journal of Personality and Social Psychology, Vol. 54, No. 4, pp. 712-717, October 1987.zh_TW
dc.relation.reference (參考文獻) [8] L. Giannetti, Understanding Movies, 10th ed. Englewood Cliffs, New Jersey: Prentice Hall, 2005.zh_TW
dc.relation.reference (參考文獻) [9] A. Hanjalic and L. Q. Xu, “Extracting Moods from Pictures and Sounds: Towards truly personalized TV,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 90-100, March 2006.zh_TW
dc.relation.reference (參考文獻) [10] A. Hanjalic and L. Q. Xu, “Affective Video Content Representation and Modeling,” IEEE Transaction on Multimedia, Vol. 7, No. 1, pp. 143-154, February 2005.zh_TW
dc.relation.reference (參考文獻) [11] A. Hanjalic and L. Q. Xu, “User-oriented Affective Video Content Analysis,” Proceedings of IEEE CBAIBL, Kauai, Hawaii, pp. 50-57, December 2001.zh_TW
dc.relation.reference (參考文獻) [12] H. B. Kang, “Affective Content Retrieval from Video with Relevance Feedback,” International Conference on Asian Digital Libraries, Kuala Lumpur, Malaysia, pp. 243-252, December 2003.zh_TW
dc.relation.reference (參考文獻) [13] H. B. Kang, “Affective Content Detection using HMMs,” Proceedings of ACM International Conference on Multimedia, Berkeley, California, U.S.A, pp. 259-262, November 2003.zh_TW
dc.relation.reference (參考文獻) [14] G. Kirouac, Les émotions: Monographies de psychologie. Sillery: Presses de l’Université du Québec, 1992.zh_TW
dc.relation.reference (參考文獻) [15] F. F. Kuo, M. F. Chiang, M. K. Shan, and S. Y. Lee, “Emotion-based Music Recommendation by Association Discovery from Film Music,” Proceedings of ACM International Conference on Multimedia, Singapore, pp. 507-510, November 2005.zh_TW
dc.relation.reference (參考文獻) [16] Y. Li, S. H. Lee, C. H. Yeh, and C. C. J. Kuo, ”Techniques for Movie Content Analysis and Skimming,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 79-89, March 2006.zh_TW
dc.relation.reference (參考文獻) [17] L. Lu, H. Jiang, and H. J. Zhang, "A Robust Audio Classification and Segmentation Method," Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 203-211, September 2001.zh_TW
dc.relation.reference (參考文獻) [18] L. Lu, H. J. Zhang, and H. Jiang, "Content Analysis for Audio Classification and Segmentation," IEEE Transaction on Speech and Audio Processing, Vol. 10, No. 7, pp. 504-516, October 2002.zh_TW
dc.relation.reference (參考文獻) [19] F. H. Mahnke, Color, Environmental and Human Response. New York: Van Nostrand Reinhold, 1996.zh_TW
dc.relation.reference (參考文獻) [20] S. Moncrieff, C. Dorai, and S. Venkatesh, “Affect Computing in Film through Sound Energy Dynamics,” Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 525-527, September 2001.zh_TW
dc.relation.reference (參考文獻) [21] A. Ortony, G. Clore, and A. Collins, The Cognitive Structure of Emotions. New York: Oxford University Press, 1988.zh_TW
dc.relation.reference (參考文獻) [22] C. E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning. Urbana, IL: University of Illinois Press, 1957.zh_TW
dc.relation.reference (參考文獻) [23] J. Y. Pan, H. J. Yang, P. Duygulu, and C. Faloutsos, “Automatic Image Captioning,” Proceedings of IEEE International Conference on Multimedia and Expo (ICME ’04), Taipei, Taiwan, pp. 1987-1990, June 2004.zh_TW
dc.relation.reference (參考文獻) [24] J. Y. Pan, H. J. Yang, C. Faloutsos, and P. Duygulu, “Automatic Multimedia Cross-modal Correlation Discovery,” Proceedings of ACM International Conference on Knowledge Discovery on Database (SIGKDD ‘04), Seattle, Washington, pp. 653-658, August 2004.zh_TW
dc.relation.reference (參考文獻) [25] D. S. Park, J. S. Park, and J. H. Han, “Image Indexing Using Color Histogram in the CIELUV Color Space,” Proceedings of 5th Japan-Korea Workshop on Computer Vision, Korea, pp. 126-132, 1999.zh_TW
dc.relation.reference (參考文獻) [26] G. Peeters, “A Large Set of Audio Features for Sound Description (Similarity and Classification),” in the CUIDADO project. Technical report, Ircam, Paris, France, April 2004.zh_TW
dc.relation.reference (參考文獻) [27] Z. Rasheed, Y. Sheikh, and M. Shah, “On the Use of Computable Features for Film Classification,” IEEE Transaction on Circuits and Systems for Video Technology (CSVT), Vol. 15, No. 1, pp. 52-64, January 2005.zh_TW
dc.relation.reference (參考文獻) [28] J. A. Russell and A. Mehrabian, “Evidence for a Three-Factor Theory of Emotions,” Journal of Research in Personality, Vol. 11, pp. 273-294, 1977.zh_TW
dc.relation.reference (參考文獻) [29] J. Saunders, “Real-Time Discrimination of Broadcast Speech/Music,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’96), Atlanta, Ga, Vol. 2, pp. 993-996, May 1996.zh_TW
dc.relation.reference (參考文獻) [30] E. Scheirer and M. Slaney, “Construction and Evaluation of a Robust Multifeature Speech/Music Discriminator”, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’97), Munich, Germany, Vol. 2, pp. 1331-1334, April 1997.zh_TW
dc.relation.reference (參考文獻) [31] R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford University Press, 1989.zh_TW
dc.relation.reference (參考文獻) [32] H. L. Wang and L. F. Cheong, “Affective Understanding in Film,” IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol. 16, No. 6, pp. 689-704, June 2006.zh_TW
dc.relation.reference (參考文獻) [33] C. Y. Wei, N. Dimitrova, and S.F. Chang, “Color-Mood Analysis of Films on Syntactic and Psychological Models,” Proceedings of IEEE International Conference on Multimedia and Expo. (ICME ’04), Taipei, Taiwan, pp. 831-834, June 2004.zh_TW
dc.relation.reference (參考文獻) [34] H. Zettl, Sight Sound Motion: Applied Media Aesthetics, 3rd ed. Belmont, CA: Wadsworth Publishing Company, 1998.zh_TW
dc.relation.reference (參考文獻) [35] http://www.intel.com/technology/computing/opencv/index.htmzh_TW
dc.relation.reference (參考文獻) [36] http://eqi.org/fw.htmzh_TW