Publications-Theses
Article View/Open
Publication Export
Google ScholarTM
NCCU Library
Citation Infomation
Related Publications in TAIR
Title | 基於電影拍攝手法之電影場景情緒探勘 Emotion Discovery of Movie Content Based on Film Grammar |
Creator | 廖家慧 Liao, Chia Hui |
Contributor | 沈錳坤 Shan, Man Kwan 廖家慧 Liao, Chia Hui |
Key Words | 內涵式分析 拍攝手法 電影場景 視聽覺特徵 情緒 content-based analysis film grammar movie scene audiovisual features emotion affective classification |
Date | 2007 |
Date Issued | 9-Apr-2010 13:17:50 (UTC+8) |
Summary | 數位化的今天,電影逐漸成為人們日常生活的一部份,電影資料的內涵式分析也成為目前重要的研究主題。透過電影拍攝手法,我們知道電影視聽覺特徵與情緒之間有密不可分的關係。因此,在本研究中,我們希望利用探勘電影視聽覺特徵與情緒的關聯來達到自動判斷電影場景的情緒。 首先,先由人工標記訓練場景的情緒,之後,我們對所有的場景擷取定義的六類特徵值。特徵值包括電影場景的顏色、燈光、影片速度、特寫鏡頭、聲音和字幕六類。最後,我們利用Mixed Media Graph演算法來探勘場景情緒與特徵值之間的關聯,達到自動判斷電影場景情緒的功能。實驗結果顯示,準確率最高可達到70%。 Movies play an important role in our life nowadays. How to analyze the emotional content of movies becomes one of the major issues. Based on film grammar, there are many audiovisual cues in movies helpful for detecting the emotions of scenes. In this research, we investigate the discovery of the relationship between audiovisual cues and emotions of scenes and the automatic emotion annotation of scenes is achieved. First, the training scenes are labeled with the emotions manually. Second, six classes of audiovisual features are extracted from all scenes. These classes of features consist of color, light, tempo, close-up, audio, and textual. Finally, the graph-based approach, Mixed Media Graph is modified to mine the association between audiovisual features and emotions of the scenes. The experiments show that the accuracy achieves 70%. |
參考文獻 | [1] B. Adams, C. Dorai, and S.Venkatesh, “Toward Automatic Extraction of Expressive Elements from Motion Pictures: Tempo,” IEEE Transactions on Multimedia, Vol. 4, No. 4, pp. 472-481, December 2002. [2] D. Arijon, Grammar of the Film Language. CA: Silman-James Press, 1976. [3] Christopher J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Journal of Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121-167, 1998. [4] A. R. Damasio, The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace, 1999. [5] R. Dietz and A. Lang, “Affective Agents: Effects of Agent Affect on Arousal, Attention, Liking and Learning,” Proceedings of Cognitive Technology Conference, San Francisco, CA, 1999. [6] N. Dimitrova, J. Martino, H. Elenbaas, and L. Agnihotri, “Color SuperHistograms for Video Representation,” IEEE International Conference on Image Processing (ICIP ‘99), Kobe, Japan, Vol. 3, pp. 314-318, October 1999. [7] P. Ekman, “Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion,” Journal of Personality and Social Psychology, Vol. 54, No. 4, pp. 712-717, October 1987. [8] L. Giannetti, Understanding Movies, 10th ed. Englewood Cliffs, New Jersey: Prentice Hall, 2005. [9] A. Hanjalic and L. Q. Xu, “Extracting Moods from Pictures and Sounds: Towards truly personalized TV,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 90-100, March 2006. [10] A. Hanjalic and L. Q. Xu, “Affective Video Content Representation and Modeling,” IEEE Transaction on Multimedia, Vol. 7, No. 1, pp. 143-154, February 2005. [11] A. Hanjalic and L. Q. Xu, “User-oriented Affective Video Content Analysis,” Proceedings of IEEE CBAIBL, Kauai, Hawaii, pp. 50-57, December 2001. [12] H. B. Kang, “Affective Content Retrieval from Video with Relevance Feedback,” International Conference on Asian Digital Libraries, Kuala Lumpur, Malaysia, pp. 243-252, December 2003. [13] H. B. Kang, “Affective Content Detection using HMMs,” Proceedings of ACM International Conference on Multimedia, Berkeley, California, U.S.A, pp. 259-262, November 2003. [14] G. Kirouac, Les émotions: Monographies de psychologie. Sillery: Presses de l’Université du Québec, 1992. [15] F. F. Kuo, M. F. Chiang, M. K. Shan, and S. Y. Lee, “Emotion-based Music Recommendation by Association Discovery from Film Music,” Proceedings of ACM International Conference on Multimedia, Singapore, pp. 507-510, November 2005. [16] Y. Li, S. H. Lee, C. H. Yeh, and C. C. J. Kuo, ”Techniques for Movie Content Analysis and Skimming,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 79-89, March 2006. [17] L. Lu, H. Jiang, and H. J. Zhang, "A Robust Audio Classification and Segmentation Method," Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 203-211, September 2001. [18] L. Lu, H. J. Zhang, and H. Jiang, "Content Analysis for Audio Classification and Segmentation," IEEE Transaction on Speech and Audio Processing, Vol. 10, No. 7, pp. 504-516, October 2002. [19] F. H. Mahnke, Color, Environmental and Human Response. New York: Van Nostrand Reinhold, 1996. [20] S. Moncrieff, C. Dorai, and S. Venkatesh, “Affect Computing in Film through Sound Energy Dynamics,” Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 525-527, September 2001. [21] A. Ortony, G. Clore, and A. Collins, The Cognitive Structure of Emotions. New York: Oxford University Press, 1988. [22] C. E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning. Urbana, IL: University of Illinois Press, 1957. [23] J. Y. Pan, H. J. Yang, P. Duygulu, and C. Faloutsos, “Automatic Image Captioning,” Proceedings of IEEE International Conference on Multimedia and Expo (ICME ’04), Taipei, Taiwan, pp. 1987-1990, June 2004. [24] J. Y. Pan, H. J. Yang, C. Faloutsos, and P. Duygulu, “Automatic Multimedia Cross-modal Correlation Discovery,” Proceedings of ACM International Conference on Knowledge Discovery on Database (SIGKDD ‘04), Seattle, Washington, pp. 653-658, August 2004. [25] D. S. Park, J. S. Park, and J. H. Han, “Image Indexing Using Color Histogram in the CIELUV Color Space,” Proceedings of 5th Japan-Korea Workshop on Computer Vision, Korea, pp. 126-132, 1999. [26] G. Peeters, “A Large Set of Audio Features for Sound Description (Similarity and Classification),” in the CUIDADO project. Technical report, Ircam, Paris, France, April 2004. [27] Z. Rasheed, Y. Sheikh, and M. Shah, “On the Use of Computable Features for Film Classification,” IEEE Transaction on Circuits and Systems for Video Technology (CSVT), Vol. 15, No. 1, pp. 52-64, January 2005. [28] J. A. Russell and A. Mehrabian, “Evidence for a Three-Factor Theory of Emotions,” Journal of Research in Personality, Vol. 11, pp. 273-294, 1977. [29] J. Saunders, “Real-Time Discrimination of Broadcast Speech/Music,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’96), Atlanta, Ga, Vol. 2, pp. 993-996, May 1996. [30] E. Scheirer and M. Slaney, “Construction and Evaluation of a Robust Multifeature Speech/Music Discriminator”, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’97), Munich, Germany, Vol. 2, pp. 1331-1334, April 1997. [31] R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford University Press, 1989. [32] H. L. Wang and L. F. Cheong, “Affective Understanding in Film,” IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol. 16, No. 6, pp. 689-704, June 2006. [33] C. Y. Wei, N. Dimitrova, and S.F. Chang, “Color-Mood Analysis of Films on Syntactic and Psychological Models,” Proceedings of IEEE International Conference on Multimedia and Expo. (ICME ’04), Taipei, Taiwan, pp. 831-834, June 2004. [34] H. Zettl, Sight Sound Motion: Applied Media Aesthetics, 3rd ed. Belmont, CA: Wadsworth Publishing Company, 1998. [35] http://www.intel.com/technology/computing/opencv/index.htm [36] http://eqi.org/fw.htm |
Description | 碩士 國立政治大學 資訊科學學系 94753027 96 |
資料來源 | http://thesis.lib.nccu.edu.tw/record/#G0094753027 |
Type | thesis |
dc.contributor.advisor | 沈錳坤 | zh_TW |
dc.contributor.advisor | Shan, Man Kwan | en_US |
dc.contributor.author (Authors) | 廖家慧 | zh_TW |
dc.contributor.author (Authors) | Liao, Chia Hui | en_US |
dc.creator (作者) | 廖家慧 | zh_TW |
dc.creator (作者) | Liao, Chia Hui | en_US |
dc.date (日期) | 2007 | en_US |
dc.date.accessioned | 9-Apr-2010 13:17:50 (UTC+8) | - |
dc.date.available | 9-Apr-2010 13:17:50 (UTC+8) | - |
dc.date.issued (上傳時間) | 9-Apr-2010 13:17:50 (UTC+8) | - |
dc.identifier (Other Identifiers) | G0094753027 | en_US |
dc.identifier.uri (URI) | http://nccur.lib.nccu.edu.tw/handle/140.119/38537 | - |
dc.description (描述) | 碩士 | zh_TW |
dc.description (描述) | 國立政治大學 | zh_TW |
dc.description (描述) | 資訊科學學系 | zh_TW |
dc.description (描述) | 94753027 | zh_TW |
dc.description (描述) | 96 | zh_TW |
dc.description.abstract (摘要) | 數位化的今天,電影逐漸成為人們日常生活的一部份,電影資料的內涵式分析也成為目前重要的研究主題。透過電影拍攝手法,我們知道電影視聽覺特徵與情緒之間有密不可分的關係。因此,在本研究中,我們希望利用探勘電影視聽覺特徵與情緒的關聯來達到自動判斷電影場景的情緒。 首先,先由人工標記訓練場景的情緒,之後,我們對所有的場景擷取定義的六類特徵值。特徵值包括電影場景的顏色、燈光、影片速度、特寫鏡頭、聲音和字幕六類。最後,我們利用Mixed Media Graph演算法來探勘場景情緒與特徵值之間的關聯,達到自動判斷電影場景情緒的功能。實驗結果顯示,準確率最高可達到70%。 | zh_TW |
dc.description.abstract (摘要) | Movies play an important role in our life nowadays. How to analyze the emotional content of movies becomes one of the major issues. Based on film grammar, there are many audiovisual cues in movies helpful for detecting the emotions of scenes. In this research, we investigate the discovery of the relationship between audiovisual cues and emotions of scenes and the automatic emotion annotation of scenes is achieved. First, the training scenes are labeled with the emotions manually. Second, six classes of audiovisual features are extracted from all scenes. These classes of features consist of color, light, tempo, close-up, audio, and textual. Finally, the graph-based approach, Mixed Media Graph is modified to mine the association between audiovisual features and emotions of the scenes. The experiments show that the accuracy achieves 70%. | en_US |
dc.description.tableofcontents | ABSTRACT IN CHINESE........................................i ABSTRACT..................................................ii ACKNOLEGEMENTS............................................iv TABLE OF CONTENTS.........................................vi LIST OF TABLES..........................................viii LIST OF FIGURES...........................................ix CHAPTER 1 Introduction.....................................1 CHAPTER 2 Related Works....................................4 2.1 Affective Classification in Film Domain.............4 CHAPTER 3 Feature Extraction...............................7 3.1 Introduction........................................7 3.2 Emotion Discovery from Scenes......................10 3.3 Virtual Features...................................11 3.4 Audio Features.....................................18 3.5 Textual Feature....................................26 CHAPTER 4 Emotion Discovery...............................28 4.1 Emotion Taxonomy...................................28 4.2 Mixed Media Graph (MMG)............................32 4.3 Scene Affinity Graph (SAG).........................34 4.3.1 SAG with Separate Visual Representation.......35 4.3.2 SAG with Integrated Visual Representation.....40 CHAPTER 5 Experiments and Results.........................45 5.1 Implementation.....................................45 5.1.1 Preprocessing.................................47 5.1.2 Visual Feature Extraction.....................47 5.1.3 Audio Feature Extraction......................49 5.1.4 Textual Feature Extraction....................49 5.1.5 Emotion Discovery.............................52 5.2 Experiments on SAG with SVR........................52 5.3 Experiments on SAG with IVR........................57 5.4 Discussion.........................................62 CHAPTER 6 Conclusions.....................................64 6.1 Summary............................................64 6.2 Future Work........................................65 REFERENCES................................................66 | zh_TW |
dc.format.extent | 44318 bytes | - |
dc.format.extent | 60276 bytes | - |
dc.format.extent | 101651 bytes | - |
dc.format.extent | 19029 bytes | - |
dc.format.extent | 17069 bytes | - |
dc.format.extent | 18883 bytes | - |
dc.format.extent | 254481 bytes | - |
dc.format.extent | 226909 bytes | - |
dc.format.extent | 72810 bytes | - |
dc.format.extent | 15175 bytes | - |
dc.format.extent | 22694 bytes | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.language.iso | en_US | - |
dc.source.uri (資料來源) | http://thesis.lib.nccu.edu.tw/record/#G0094753027 | en_US |
dc.subject (關鍵詞) | 內涵式分析 | zh_TW |
dc.subject (關鍵詞) | 拍攝手法 | zh_TW |
dc.subject (關鍵詞) | 電影場景 | zh_TW |
dc.subject (關鍵詞) | 視聽覺特徵 | zh_TW |
dc.subject (關鍵詞) | 情緒 | zh_TW |
dc.subject (關鍵詞) | content-based analysis | en_US |
dc.subject (關鍵詞) | film grammar | en_US |
dc.subject (關鍵詞) | movie scene | en_US |
dc.subject (關鍵詞) | audiovisual features | en_US |
dc.subject (關鍵詞) | emotion | en_US |
dc.subject (關鍵詞) | affective classification | en_US |
dc.title (題名) | 基於電影拍攝手法之電影場景情緒探勘 | zh_TW |
dc.title (題名) | Emotion Discovery of Movie Content Based on Film Grammar | en_US |
dc.type (資料類型) | thesis | en |
dc.relation.reference (參考文獻) | [1] B. Adams, C. Dorai, and S.Venkatesh, “Toward Automatic Extraction of Expressive Elements from Motion Pictures: Tempo,” IEEE Transactions on Multimedia, Vol. 4, No. 4, pp. 472-481, December 2002. | zh_TW |
dc.relation.reference (參考文獻) | [2] D. Arijon, Grammar of the Film Language. CA: Silman-James Press, 1976. | zh_TW |
dc.relation.reference (參考文獻) | [3] Christopher J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Journal of Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121-167, 1998. | zh_TW |
dc.relation.reference (參考文獻) | [4] A. R. Damasio, The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace, 1999. | zh_TW |
dc.relation.reference (參考文獻) | [5] R. Dietz and A. Lang, “Affective Agents: Effects of Agent Affect on Arousal, Attention, Liking and Learning,” Proceedings of Cognitive Technology Conference, San Francisco, CA, 1999. | zh_TW |
dc.relation.reference (參考文獻) | [6] N. Dimitrova, J. Martino, H. Elenbaas, and L. Agnihotri, “Color SuperHistograms for Video Representation,” IEEE International Conference on Image Processing (ICIP ‘99), Kobe, Japan, Vol. 3, pp. 314-318, October 1999. | zh_TW |
dc.relation.reference (參考文獻) | [7] P. Ekman, “Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion,” Journal of Personality and Social Psychology, Vol. 54, No. 4, pp. 712-717, October 1987. | zh_TW |
dc.relation.reference (參考文獻) | [8] L. Giannetti, Understanding Movies, 10th ed. Englewood Cliffs, New Jersey: Prentice Hall, 2005. | zh_TW |
dc.relation.reference (參考文獻) | [9] A. Hanjalic and L. Q. Xu, “Extracting Moods from Pictures and Sounds: Towards truly personalized TV,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 90-100, March 2006. | zh_TW |
dc.relation.reference (參考文獻) | [10] A. Hanjalic and L. Q. Xu, “Affective Video Content Representation and Modeling,” IEEE Transaction on Multimedia, Vol. 7, No. 1, pp. 143-154, February 2005. | zh_TW |
dc.relation.reference (參考文獻) | [11] A. Hanjalic and L. Q. Xu, “User-oriented Affective Video Content Analysis,” Proceedings of IEEE CBAIBL, Kauai, Hawaii, pp. 50-57, December 2001. | zh_TW |
dc.relation.reference (參考文獻) | [12] H. B. Kang, “Affective Content Retrieval from Video with Relevance Feedback,” International Conference on Asian Digital Libraries, Kuala Lumpur, Malaysia, pp. 243-252, December 2003. | zh_TW |
dc.relation.reference (參考文獻) | [13] H. B. Kang, “Affective Content Detection using HMMs,” Proceedings of ACM International Conference on Multimedia, Berkeley, California, U.S.A, pp. 259-262, November 2003. | zh_TW |
dc.relation.reference (參考文獻) | [14] G. Kirouac, Les émotions: Monographies de psychologie. Sillery: Presses de l’Université du Québec, 1992. | zh_TW |
dc.relation.reference (參考文獻) | [15] F. F. Kuo, M. F. Chiang, M. K. Shan, and S. Y. Lee, “Emotion-based Music Recommendation by Association Discovery from Film Music,” Proceedings of ACM International Conference on Multimedia, Singapore, pp. 507-510, November 2005. | zh_TW |
dc.relation.reference (參考文獻) | [16] Y. Li, S. H. Lee, C. H. Yeh, and C. C. J. Kuo, ”Techniques for Movie Content Analysis and Skimming,” IEEE Signal Processing Magazine, Vol. 23, No. 2, pp. 79-89, March 2006. | zh_TW |
dc.relation.reference (參考文獻) | [17] L. Lu, H. Jiang, and H. J. Zhang, "A Robust Audio Classification and Segmentation Method," Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 203-211, September 2001. | zh_TW |
dc.relation.reference (參考文獻) | [18] L. Lu, H. J. Zhang, and H. Jiang, "Content Analysis for Audio Classification and Segmentation," IEEE Transaction on Speech and Audio Processing, Vol. 10, No. 7, pp. 504-516, October 2002. | zh_TW |
dc.relation.reference (參考文獻) | [19] F. H. Mahnke, Color, Environmental and Human Response. New York: Van Nostrand Reinhold, 1996. | zh_TW |
dc.relation.reference (參考文獻) | [20] S. Moncrieff, C. Dorai, and S. Venkatesh, “Affect Computing in Film through Sound Energy Dynamics,” Proceedings of ACM International Conference on Multimedia, Ottawa, Ontario, Canada, pp. 525-527, September 2001. | zh_TW |
dc.relation.reference (參考文獻) | [21] A. Ortony, G. Clore, and A. Collins, The Cognitive Structure of Emotions. New York: Oxford University Press, 1988. | zh_TW |
dc.relation.reference (參考文獻) | [22] C. E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning. Urbana, IL: University of Illinois Press, 1957. | zh_TW |
dc.relation.reference (參考文獻) | [23] J. Y. Pan, H. J. Yang, P. Duygulu, and C. Faloutsos, “Automatic Image Captioning,” Proceedings of IEEE International Conference on Multimedia and Expo (ICME ’04), Taipei, Taiwan, pp. 1987-1990, June 2004. | zh_TW |
dc.relation.reference (參考文獻) | [24] J. Y. Pan, H. J. Yang, C. Faloutsos, and P. Duygulu, “Automatic Multimedia Cross-modal Correlation Discovery,” Proceedings of ACM International Conference on Knowledge Discovery on Database (SIGKDD ‘04), Seattle, Washington, pp. 653-658, August 2004. | zh_TW |
dc.relation.reference (參考文獻) | [25] D. S. Park, J. S. Park, and J. H. Han, “Image Indexing Using Color Histogram in the CIELUV Color Space,” Proceedings of 5th Japan-Korea Workshop on Computer Vision, Korea, pp. 126-132, 1999. | zh_TW |
dc.relation.reference (參考文獻) | [26] G. Peeters, “A Large Set of Audio Features for Sound Description (Similarity and Classification),” in the CUIDADO project. Technical report, Ircam, Paris, France, April 2004. | zh_TW |
dc.relation.reference (參考文獻) | [27] Z. Rasheed, Y. Sheikh, and M. Shah, “On the Use of Computable Features for Film Classification,” IEEE Transaction on Circuits and Systems for Video Technology (CSVT), Vol. 15, No. 1, pp. 52-64, January 2005. | zh_TW |
dc.relation.reference (參考文獻) | [28] J. A. Russell and A. Mehrabian, “Evidence for a Three-Factor Theory of Emotions,” Journal of Research in Personality, Vol. 11, pp. 273-294, 1977. | zh_TW |
dc.relation.reference (參考文獻) | [29] J. Saunders, “Real-Time Discrimination of Broadcast Speech/Music,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’96), Atlanta, Ga, Vol. 2, pp. 993-996, May 1996. | zh_TW |
dc.relation.reference (參考文獻) | [30] E. Scheirer and M. Slaney, “Construction and Evaluation of a Robust Multifeature Speech/Music Discriminator”, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’97), Munich, Germany, Vol. 2, pp. 1331-1334, April 1997. | zh_TW |
dc.relation.reference (參考文獻) | [31] R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford University Press, 1989. | zh_TW |
dc.relation.reference (參考文獻) | [32] H. L. Wang and L. F. Cheong, “Affective Understanding in Film,” IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol. 16, No. 6, pp. 689-704, June 2006. | zh_TW |
dc.relation.reference (參考文獻) | [33] C. Y. Wei, N. Dimitrova, and S.F. Chang, “Color-Mood Analysis of Films on Syntactic and Psychological Models,” Proceedings of IEEE International Conference on Multimedia and Expo. (ICME ’04), Taipei, Taiwan, pp. 831-834, June 2004. | zh_TW |
dc.relation.reference (參考文獻) | [34] H. Zettl, Sight Sound Motion: Applied Media Aesthetics, 3rd ed. Belmont, CA: Wadsworth Publishing Company, 1998. | zh_TW |
dc.relation.reference (參考文獻) | [35] http://www.intel.com/technology/computing/opencv/index.htm | zh_TW |
dc.relation.reference (參考文獻) | [36] http://eqi.org/fw.htm | zh_TW |