學術產出-Theses
Article View/Open
Publication Export
-
題名 動漫魔鏡：運用圖文關聯探勘的動漫網站搜索引擎
Comirror: A Search Engine for Comic Web Based on Textual and Visual Correlation Mining作者 孫世通
Sun, Shi Tong貢獻者 沈錳坤
Shan, Man Kwan
孫世通
Sun, Shi Tong關鍵詞 動漫網站
搜尋引擎
圖文關連
動漫風格
ACG Websites
Search Engines
Correlation Mining
Comic Styles日期 2013 上傳時間 1-Nov-2013 11:44:27 (UTC+8) 摘要 近年來，動漫(動畫、漫畫與線上遊戲)越來越受歡迎。全球資訊網也陸續出現收集大量包括故事情節、動漫角色、作者等動漫相關資訊的動漫網站。多數動漫網站都提供使用者文字檢索的功能，以搜尋動漫網站文字內容。但是動站網站若能提供根據文字與圖形風格來搜尋圖文內容，對於動漫使用者而言，將更為方便。圖文風格可能是漫畫人物的繪畫風格、動畫故事的敘事風格等等。為了方便使用者以圖文風格進行搜尋相關資訊，本論文根據動漫關連探勘技術，研究並開發一個動漫網站的搜尋引擎：動漫魔鏡，以提供使用者根據圖文關連來搜尋動漫網站中風格相似的動漫資訊。本論文的搜索方法關聯了圖像特徵和文字特徵。首先，針對圖像特徵，由於動漫角色是動漫的靈魂，因此經過動漫臉部偵測後，我們以電腦視覺中的局部二值樣式(Local Binary Pattern, LBP)與灰階值分佈來抽取並表示動漫角色的臉部特徵。針對文字特徵，我們利用一般全文檢索技術來擷取文字特徵。接著，運用階層式分群技術將文字特徵與圖像特徵值轉換為文字與圖像關鍵詞。最後，以語意主題模型中的隱含狄利克雷分布(Latent Dirichlet Allocation, LDA)分析圖文關鍵詞的潛在語意，並據此計算動漫網頁之間的相似性。實驗結果顯示本論文所研發的風格搜尋，其效果優於其他三種基本作法。
Animations, comics, and games (ACG) have become more and more popular in recent years. There exist many ACG web sites which contain lots of textual and visual information on stories, characters and authors of animations, comics and games. Most ACG web sites provide users text retrieval capability to search for textual contents. However, there is a need for users to search for textual and visual contents by styles. Examples of styles are drawing styles of comic characters, narrative styles of animation stories and so on. In order to help users to search for textual and visual contents by similar styles, this thesis investigates and develops a search engine, Comirror, for ACG web sites based on latent correlation between textual and visual contents. First, while facial styles of characters play important roles in ACG, after comic face detection, Local Binary Pattern (LBP) along with gray-value histogram is utilized to extract and represent the visual features. For the textual contents, traditional full-text indexing technique is employed to extract textual features. Then, hierarchical clustering is performed to quantize and transform the textual and visual features into textual and visual words. Finally, Latent Dirichlet Allocation (LDA) is utilized to discover the latent semantic correlation between visual and textual words. Experiments show that the developed approach performs better than the other baseline approaches.參考文獻 [1] H. Bay, T. Tuytelaars and L. Van Gool, "SURF: Speeded up Robust Features," European Conference on Computer Vision, 2006.[2] D. M. Blei, Andrew Ng and M. Jordan. "Latent Dirichlet Allocation," The Journal of Machine Learning Research, Vol.3, pp. 993-1022, 2003.[3] M. Brown and D. Lowe, "Recognizing Panoramas," The 9th International Conference on Computer Vision, pp. 1218-1227, 2003.[4] J. Canny. "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.8, No.6, pp. 679-698, 1986.[5] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 88–893, 2005.[6] S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman, "Indexing by latent semantic analysis," Journal of the American Society for Information Science and Technology , Vol.41, pp. 391-407, 1990.[7] T. Gritti , C. Shan , V. Jeanne and R. Braspenning, "Local features based facial expression recognition with face registration errors," Automatic Face & Gesture Recognition, FG `08. 8th IEEE International Conference, pp. 1-8, 2008.[8] T. Hofmann, "Probabilistic Latent Semantic Analysis," Uncertainty in Artificial Intelligence, UAI’99, pp. 289-296, 1999.[9] L. Juan and O. Gwun, "A Comparison of SIFT, PCA-SIFT and. SURF," International Journal of Image Processing, Vol. 65, pp. 143-152, 2009. [10] Y. Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," Computer Vision and Pattern Recognition, Vol.2, 2004.[11] M. La Cascia , S. Sethi , S. Sclaroff, "Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web," IEEE Workshop on Content - Based Access of Image and Video Libraries, pp.24, June. 1998[12] R. Lienhart and J. Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection," IEEE International Conference on Image Processing 2002, Vol. 1, pp. 900-903, Sep. 2002.[13] D. G. Lowe, "Object Recognition from Local Scale-invariant Features," the International Conference on Computer Vision, Vol.2, pp. 1150–1157, 1999.[14] D. G. Lowe, "Distinctive Image Features from Scale-invariant Keypoints," International Journal of Computer Vision, Vol.60, No.2, pp. 91-110, 2004.[15] K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp. 1615-1630, 2005.[16] T. Ojala, M. Pietikäinen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," IAPR International Conference on Pattern Recognition (ICPR), Vol. 1, pp.582–585, 1994.[17] T. Ojala, M. Pietikäinen, and D. Harwood, "A Comparative Study of Texture Measures with Classification Based on Feature Distributions," Pattern Recognition, Vol. 29, pp. 51-59, 1996.[18] T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 24, No. 7, pp.971−987, 2002.[19] T. Reenskaug, "Thing-model-view-editor - an Example from a Planning System, " Xerox PARC Technical Note, 1979.[20] G. Salton, A. Wong, and C. S. Yang, "A Vector Space Model for Automatic Indexing," Communications of the ACM, Vol. 18, No. 11, pp. 613–620, 1975.[21] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," IEEE Conference on Computer Vision and Pattern Recognition, 2000.[22] W. Sun and K. Kise, "Detecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds," International Conference on Document Analysis and Recognition, pp. 909–919, 2009.[23] W. Sun and K. Kise, "Similar Partial Copy Detection of Line Drawings Using a Cascade Classifier and Feature Matching," International Workshop on Computational Forensics, pp.126-137, 2010.[24] W. Sun and K. Kise, "Similar Manga Retrieval Using Visual Vocabulary Based on Regions of Interest," International Conference on Document Analysis and Recognition, pp. 1075-1079, 2011.[25] S. Tollari, H. Glotin, "Web Image Retrieval on ImagEVAL: Evidences on Visualness and Textualness Concept Dependency in Fusion Model," ACM International Conference on Image and Video Retrieval (CIVR), July. 2007.[26] P. Viola and M. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, No. 2, pp.137–154, 2004.[27] Q. Zhu, S. Avidan, M. Yeh and K. Cheng, "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients," IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp.1491–1498, 2006.[28] Naotoshi Seo`s note site, http://note.sonots.com/[29] 百度百科，http://baike.baidu.com/[30] 日本動漫，http://baike.baidu.com/view/495014.htm 描述 碩士
國立政治大學
資訊科學學系
100753041
102資料來源 http://thesis.lib.nccu.edu.tw/record/#G0100753041 資料類型 thesis dc.contributor.advisor 沈錳坤 zh_TW dc.contributor.advisor Shan, Man Kwan en_US dc.contributor.author (Authors) 孫世通 zh_TW dc.contributor.author (Authors) Sun, Shi Tong en_US dc.creator (作者) 孫世通 zh_TW dc.creator (作者) Sun, Shi Tong en_US dc.date (日期) 2013 en_US dc.date.accessioned 1-Nov-2013 11:44:27 (UTC+8) - dc.date.available 1-Nov-2013 11:44:27 (UTC+8) - dc.date.issued (上傳時間) 1-Nov-2013 11:44:27 (UTC+8) - dc.identifier (Other Identifiers) G0100753041 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/61493 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學學系 zh_TW dc.description (描述) 100753041 zh_TW dc.description (描述) 102 zh_TW dc.description.abstract (摘要) 近年來，動漫(動畫、漫畫與線上遊戲)越來越受歡迎。全球資訊網也陸續出現收集大量包括故事情節、動漫角色、作者等動漫相關資訊的動漫網站。多數動漫網站都提供使用者文字檢索的功能，以搜尋動漫網站文字內容。但是動站網站若能提供根據文字與圖形風格來搜尋圖文內容，對於動漫使用者而言，將更為方便。圖文風格可能是漫畫人物的繪畫風格、動畫故事的敘事風格等等。為了方便使用者以圖文風格進行搜尋相關資訊，本論文根據動漫關連探勘技術，研究並開發一個動漫網站的搜尋引擎：動漫魔鏡，以提供使用者根據圖文關連來搜尋動漫網站中風格相似的動漫資訊。本論文的搜索方法關聯了圖像特徵和文字特徵。首先，針對圖像特徵，由於動漫角色是動漫的靈魂，因此經過動漫臉部偵測後，我們以電腦視覺中的局部二值樣式(Local Binary Pattern, LBP)與灰階值分佈來抽取並表示動漫角色的臉部特徵。針對文字特徵，我們利用一般全文檢索技術來擷取文字特徵。接著，運用階層式分群技術將文字特徵與圖像特徵值轉換為文字與圖像關鍵詞。最後，以語意主題模型中的隱含狄利克雷分布(Latent Dirichlet Allocation, LDA)分析圖文關鍵詞的潛在語意，並據此計算動漫網頁之間的相似性。實驗結果顯示本論文所研發的風格搜尋，其效果優於其他三種基本作法。 zh_TW dc.description.abstract (摘要) Animations, comics, and games (ACG) have become more and more popular in recent years. There exist many ACG web sites which contain lots of textual and visual information on stories, characters and authors of animations, comics and games. Most ACG web sites provide users text retrieval capability to search for textual contents. However, there is a need for users to search for textual and visual contents by styles. Examples of styles are drawing styles of comic characters, narrative styles of animation stories and so on. In order to help users to search for textual and visual contents by similar styles, this thesis investigates and develops a search engine, Comirror, for ACG web sites based on latent correlation between textual and visual contents. First, while facial styles of characters play important roles in ACG, after comic face detection, Local Binary Pattern (LBP) along with gray-value histogram is utilized to extract and represent the visual features. For the textual contents, traditional full-text indexing technique is employed to extract textual features. Then, hierarchical clustering is performed to quantize and transform the textual and visual features into textual and visual words. Finally, Latent Dirichlet Allocation (LDA) is utilized to discover the latent semantic correlation between visual and textual words. Experiments show that the developed approach performs better than the other baseline approaches. en_US dc.description.tableofcontents 中文摘要 i英文摘要 ii致謝 iii目錄 v圖目錄 vii表目錄 ix第1章 前言 1第2章 相關研究 32.1. 動漫圖像的獨有特性及其特徵區域選取 32.2. 動漫圖像的特徵描述 52.2.1. LBP Descriptor 52.2.2. SIFT Descriptor 62.2.3. HOG Descriptor 72.3. Web Image Retrieval 8第3章 研究方法 93.1. 動漫圖像特徵 103.1.1. Detection of Comic Face 103.1.2. Representation of Comic Face 143.1.3. Facial Word Transformation 173.2. 動漫文字特徵 183.2.1. 中文環境下的Text Segmentation 及Bag-of-Words 183.3. 圖文整合分析 203.3.1. Feature Combination 203.3.2. Feature Correlation between Image and Text 21第4章 系統實作 254.1. 資料來源 254.2. 線下部份實作流程 264.2.1. Comic Face的訓練與測試 264.2.2. Facial Word Matrix的產生 294.2.3. 網頁文字處理 314.2.4. 圖文矩陣 & 網頁相似度 314.3. 網站應用工具及介面 33第5章 實驗評估 365.1. 實驗資料來源 365.2. 實驗目的 375.3. 實驗設計 375.4. 實驗結果 38第6章 結論與未來工作 43參考文獻 45 zh_TW dc.format.extent 2394504 bytes - dc.format.mimetype application/pdf - dc.language.iso en_US - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0100753041 en_US dc.subject (關鍵詞) 動漫網站 zh_TW dc.subject (關鍵詞) 搜尋引擎 zh_TW dc.subject (關鍵詞) 圖文關連 zh_TW dc.subject (關鍵詞) 動漫風格 zh_TW dc.subject (關鍵詞) ACG Websites en_US dc.subject (關鍵詞) Search Engines en_US dc.subject (關鍵詞) Correlation Mining en_US dc.subject (關鍵詞) Comic Styles en_US dc.title (題名) 動漫魔鏡：運用圖文關聯探勘的動漫網站搜索引擎 zh_TW dc.title (題名) Comirror: A Search Engine for Comic Web Based on Textual and Visual Correlation Mining en_US dc.type (資料類型) thesis en dc.relation.reference (參考文獻) [1] H. Bay, T. Tuytelaars and L. Van Gool, "SURF: Speeded up Robust Features," European Conference on Computer Vision, 2006.[2] D. M. Blei, Andrew Ng and M. Jordan. "Latent Dirichlet Allocation," The Journal of Machine Learning Research, Vol.3, pp. 993-1022, 2003.[3] M. Brown and D. Lowe, "Recognizing Panoramas," The 9th International Conference on Computer Vision, pp. 1218-1227, 2003.[4] J. Canny. "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.8, No.6, pp. 679-698, 1986.[5] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 88–893, 2005.[6] S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman, "Indexing by latent semantic analysis," Journal of the American Society for Information Science and Technology , Vol.41, pp. 391-407, 1990.[7] T. Gritti , C. Shan , V. Jeanne and R. Braspenning, "Local features based facial expression recognition with face registration errors," Automatic Face & Gesture Recognition, FG `08. 8th IEEE International Conference, pp. 1-8, 2008.[8] T. Hofmann, "Probabilistic Latent Semantic Analysis," Uncertainty in Artificial Intelligence, UAI’99, pp. 289-296, 1999.[9] L. Juan and O. Gwun, "A Comparison of SIFT, PCA-SIFT and. SURF," International Journal of Image Processing, Vol. 65, pp. 143-152, 2009. [10] Y. Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," Computer Vision and Pattern Recognition, Vol.2, 2004.[11] M. La Cascia , S. Sethi , S. Sclaroff, "Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web," IEEE Workshop on Content - Based Access of Image and Video Libraries, pp.24, June. 1998[12] R. Lienhart and J. Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection," IEEE International Conference on Image Processing 2002, Vol. 1, pp. 900-903, Sep. 2002.[13] D. G. Lowe, "Object Recognition from Local Scale-invariant Features," the International Conference on Computer Vision, Vol.2, pp. 1150–1157, 1999.[14] D. G. Lowe, "Distinctive Image Features from Scale-invariant Keypoints," International Journal of Computer Vision, Vol.60, No.2, pp. 91-110, 2004.[15] K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp. 1615-1630, 2005.[16] T. Ojala, M. Pietikäinen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," IAPR International Conference on Pattern Recognition (ICPR), Vol. 1, pp.582–585, 1994.[17] T. Ojala, M. Pietikäinen, and D. Harwood, "A Comparative Study of Texture Measures with Classification Based on Feature Distributions," Pattern Recognition, Vol. 29, pp. 51-59, 1996.[18] T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 24, No. 7, pp.971−987, 2002.[19] T. Reenskaug, "Thing-model-view-editor - an Example from a Planning System, " Xerox PARC Technical Note, 1979.[20] G. Salton, A. Wong, and C. S. Yang, "A Vector Space Model for Automatic Indexing," Communications of the ACM, Vol. 18, No. 11, pp. 613–620, 1975.[21] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," IEEE Conference on Computer Vision and Pattern Recognition, 2000.[22] W. Sun and K. Kise, "Detecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds," International Conference on Document Analysis and Recognition, pp. 909–919, 2009.[23] W. Sun and K. Kise, "Similar Partial Copy Detection of Line Drawings Using a Cascade Classifier and Feature Matching," International Workshop on Computational Forensics, pp.126-137, 2010.[24] W. Sun and K. Kise, "Similar Manga Retrieval Using Visual Vocabulary Based on Regions of Interest," International Conference on Document Analysis and Recognition, pp. 1075-1079, 2011.[25] S. Tollari, H. Glotin, "Web Image Retrieval on ImagEVAL: Evidences on Visualness and Textualness Concept Dependency in Fusion Model," ACM International Conference on Image and Video Retrieval (CIVR), July. 2007.[26] P. Viola and M. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, No. 2, pp.137–154, 2004.[27] Q. Zhu, S. Avidan, M. Yeh and K. Cheng, "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients," IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp.1491–1498, 2006.[28] Naotoshi Seo`s note site, http://note.sonots.com/[29] 百度百科，http://baike.baidu.com/[30] 日本動漫，http://baike.baidu.com/view/495014.htm zh_TW
