1. NCCU Academic Hub
  2. Publications
  3. College of Informatics
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 動漫魔鏡：運用圖文關聯探勘的動漫網站搜索引擎
Comirror: A Search Engine for Comic Web Based on Textual and Visual Correlation Mining
作者 孫世通
Sun, Shi Tong
貢獻者 沈錳坤
Shan, Man Kwan
孫世通
Sun, Shi Tong
關鍵詞 動漫網站
搜尋引擎
圖文關連
動漫風格
ACG Websites
Search Engines
Correlation Mining
Comic Styles
日期 2013
上傳時間 1-Nov-2013 11:44:27 (UTC+8)
摘要 近年來，動漫(動畫、漫畫與線上遊戲)越來越受歡迎。全球資訊網也陸續出現收集大量包括故事情節、動漫角色、作者等動漫相關資訊的動漫網站。多數動漫網站都提供使用者文字檢索的功能，以搜尋動漫網站文字內容。但是動站網站若能提供根據文字與圖形風格來搜尋圖文內容，對於動漫使用者而言，將更為方便。圖文風格可能是漫畫人物的繪畫風格、動畫故事的敘事風格等等。
為了方便使用者以圖文風格進行搜尋相關資訊，本論文根據動漫關連探勘技術，研究並開發一個動漫網站的搜尋引擎：動漫魔鏡，以提供使用者根據圖文關連來搜尋動漫網站中風格相似的動漫資訊。本論文的搜索方法關聯了圖像特徵和文字特徵。首先，針對圖像特徵，由於動漫角色是動漫的靈魂，因此經過動漫臉部偵測後，我們以電腦視覺中的局部二值樣式(Local Binary Pattern, LBP)與灰階值分佈來抽取並表示動漫角色的臉部特徵。針對文字特徵，我們利用一般全文檢索技術來擷取文字特徵。接著，運用階層式分群技術將文字特徵與圖像特徵值轉換為文字與圖像關鍵詞。最後，以語意主題模型中的隱含狄利克雷分布(Latent Dirichlet Allocation, LDA)分析圖文關鍵詞的潛在語意，並據此計算動漫網頁之間的相似性。實驗結果顯示本論文所研發的風格搜尋，其效果優於其他三種基本作法。
Animations, comics, and games (ACG) have become more and more popular in recent years. There exist many ACG web sites which contain lots of textual and visual information on stories, characters and authors of animations, comics and games. Most ACG web sites provide users text retrieval capability to search for textual contents. However, there is a need for users to search for textual and visual contents by styles. Examples of styles are drawing styles of comic characters, narrative styles of animation stories and so on.
In order to help users to search for textual and visual contents by similar styles, this thesis investigates and develops a search engine, Comirror, for ACG web sites based on latent correlation between textual and visual contents. First, while facial styles of characters play important roles in ACG, after comic face detection, Local Binary Pattern (LBP) along with gray-value histogram is utilized to extract and represent the visual features. For the textual contents, traditional full-text indexing technique is employed to extract textual features. Then, hierarchical clustering is performed to quantize and transform the textual and visual features into textual and visual words. Finally, Latent Dirichlet Allocation (LDA) is utilized to discover the latent semantic correlation between visual and textual words. Experiments show that the developed approach performs better than the other baseline approaches.
參考文獻 [1] H. Bay, T. Tuytelaars and L. Van Gool, "SURF: Speeded up Robust Features," European Conference on Computer Vision, 2006.
[2] D. M. Blei, Andrew Ng and M. Jordan. "Latent Dirichlet Allocation," The Journal of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
[3] M. Brown and D. Lowe, "Recognizing Panoramas," The 9th International Conference on Computer Vision, pp. 1218-1227, 2003.
[4] J. Canny. "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.8, No.6, pp. 679-698, 1986.
[5] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 88–893, 2005.
[6] S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman, "Indexing by latent semantic analysis," Journal of the American Society for Information Science and Technology , Vol.41, pp. 391-407, 1990.
[7] T. Gritti , C. Shan , V. Jeanne and R. Braspenning, "Local features based facial expression recognition with face registration errors," Automatic Face & Gesture Recognition, FG `08. 8th IEEE International Conference, pp. 1-8, 2008.
[8] T. Hofmann, "Probabilistic Latent Semantic Analysis," Uncertainty in Artificial Intelligence, UAI’99, pp. 289-296, 1999.
[9] L. Juan and O. Gwun, "A Comparison of SIFT, PCA-SIFT and. SURF," International Journal of Image Processing, Vol. 65, pp. 143-152, 2009.
[10] Y. Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," Computer Vision and Pattern Recognition, Vol.2, 2004.
[11] M. La Cascia , S. Sethi , S. Sclaroff, "Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web," IEEE Workshop on Content - Based Access of Image and Video Libraries, pp.24, June. 1998
[12] R. Lienhart and J. Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection," IEEE International Conference on Image Processing 2002, Vol. 1, pp. 900-903, Sep. 2002.
[13] D. G. Lowe, "Object Recognition from Local Scale-invariant Features," the International Conference on Computer Vision, Vol.2, pp. 1150–1157, 1999.
[14] D. G. Lowe, "Distinctive Image Features from Scale-invariant Keypoints," International Journal of Computer Vision, Vol.60, No.2, pp. 91-110, 2004.
[15] K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp. 1615-1630, 2005.
[16] T. Ojala, M. Pietikäinen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," IAPR International Conference on Pattern Recognition (ICPR), Vol. 1, pp.582–585, 1994.
[17] T. Ojala, M. Pietikäinen, and D. Harwood, "A Comparative Study of Texture Measures with Classification Based on Feature Distributions," Pattern Recognition, Vol. 29, pp. 51-59, 1996.
[18] T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 24, No. 7, pp.971−987, 2002.
[19] T. Reenskaug, "Thing-model-view-editor - an Example from a Planning System, " Xerox PARC Technical Note, 1979.
[20] G. Salton, A. Wong, and C. S. Yang, "A Vector Space Model for Automatic Indexing," Communications of the ACM, Vol. 18, No. 11, pp. 613–620, 1975.
[21] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," IEEE Conference on Computer Vision and Pattern Recognition, 2000.
[22] W. Sun and K. Kise, "Detecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds," International Conference on Document Analysis and Recognition, pp. 909–919, 2009.
[23] W. Sun and K. Kise, "Similar Partial Copy Detection of Line Drawings Using a Cascade Classifier and Feature Matching," International Workshop on Computational Forensics, pp.126-137, 2010.
[24] W. Sun and K. Kise, "Similar Manga Retrieval Using Visual Vocabulary Based on Regions of Interest," International Conference on Document Analysis and Recognition, pp. 1075-1079, 2011.
[25] S. Tollari, H. Glotin, "Web Image Retrieval on ImagEVAL: Evidences on Visualness and Textualness Concept Dependency in Fusion Model," ACM International Conference on Image and Video Retrieval (CIVR), July. 2007.
[26] P. Viola and M. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, No. 2, pp.137–154, 2004.
[27] Q. Zhu, S. Avidan, M. Yeh and K. Cheng, "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients," IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp.1491–1498, 2006.
[28] Naotoshi Seo`s note site, http://note.sonots.com/
[29] 百度百科，http://baike.baidu.com/
[30] 日本動漫，http://baike.baidu.com/view/495014.htm
描述 碩士
國立政治大學
資訊科學學系
100753041
102
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0100753041
資料類型 thesis
dc.contributor.advisor 沈錳坤zh_TW
dc.contributor.advisor Shan, Man Kwanen_US
dc.contributor.author (Authors) 孫世通zh_TW
dc.contributor.author (Authors) Sun, Shi Tongen_US
dc.creator (作者) 孫世通zh_TW
dc.creator (作者) Sun, Shi Tongen_US
dc.date (日期) 2013en_US
dc.date.accessioned 1-Nov-2013 11:44:27 (UTC+8)-
dc.date.available 1-Nov-2013 11:44:27 (UTC+8)-
dc.date.issued (上傳時間) 1-Nov-2013 11:44:27 (UTC+8)-
dc.identifier (Other Identifiers) G0100753041en_US
dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/61493-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學學系zh_TW
dc.description (描述) 100753041zh_TW
dc.description (描述) 102zh_TW
dc.description.abstract (摘要) 近年來，動漫(動畫、漫畫與線上遊戲)越來越受歡迎。全球資訊網也陸續出現收集大量包括故事情節、動漫角色、作者等動漫相關資訊的動漫網站。多數動漫網站都提供使用者文字檢索的功能，以搜尋動漫網站文字內容。但是動站網站若能提供根據文字與圖形風格來搜尋圖文內容，對於動漫使用者而言，將更為方便。圖文風格可能是漫畫人物的繪畫風格、動畫故事的敘事風格等等。
為了方便使用者以圖文風格進行搜尋相關資訊，本論文根據動漫關連探勘技術，研究並開發一個動漫網站的搜尋引擎：動漫魔鏡，以提供使用者根據圖文關連來搜尋動漫網站中風格相似的動漫資訊。本論文的搜索方法關聯了圖像特徵和文字特徵。首先，針對圖像特徵，由於動漫角色是動漫的靈魂，因此經過動漫臉部偵測後，我們以電腦視覺中的局部二值樣式(Local Binary Pattern, LBP)與灰階值分佈來抽取並表示動漫角色的臉部特徵。針對文字特徵，我們利用一般全文檢索技術來擷取文字特徵。接著，運用階層式分群技術將文字特徵與圖像特徵值轉換為文字與圖像關鍵詞。最後，以語意主題模型中的隱含狄利克雷分布(Latent Dirichlet Allocation, LDA)分析圖文關鍵詞的潛在語意，並據此計算動漫網頁之間的相似性。實驗結果顯示本論文所研發的風格搜尋，其效果優於其他三種基本作法。		zh_TW
dc.description.abstract (摘要) Animations, comics, and games (ACG) have become more and more popular in recent years. There exist many ACG web sites which contain lots of textual and visual information on stories, characters and authors of animations, comics and games. Most ACG web sites provide users text retrieval capability to search for textual contents. However, there is a need for users to search for textual and visual contents by styles. Examples of styles are drawing styles of comic characters, narrative styles of animation stories and so on.
In order to help users to search for textual and visual contents by similar styles, this thesis investigates and develops a search engine, Comirror, for ACG web sites based on latent correlation between textual and visual contents. First, while facial styles of characters play important roles in ACG, after comic face detection, Local Binary Pattern (LBP) along with gray-value histogram is utilized to extract and represent the visual features. For the textual contents, traditional full-text indexing technique is employed to extract textual features. Then, hierarchical clustering is performed to quantize and transform the textual and visual features into textual and visual words. Finally, Latent Dirichlet Allocation (LDA) is utilized to discover the latent semantic correlation between visual and textual words. Experiments show that the developed approach performs better than the other baseline approaches.		en_US
dc.description.tableofcontents 中文摘要 i
英文摘要 ii
致謝 iii
目錄 v
圖目錄 vii
表目錄 ix
第1章 前言 1
第2章 相關研究 3
2.1. 動漫圖像的獨有特性及其特徵區域選取 3
2.2. 動漫圖像的特徵描述 5
2.2.1. LBP Descriptor 5
2.2.2. SIFT Descriptor 6
2.2.3. HOG Descriptor 7
2.3. Web Image Retrieval 8
第3章 研究方法 9
3.1. 動漫圖像特徵 10
3.1.1. Detection of Comic Face 10
3.1.2. Representation of Comic Face 14
3.1.3. Facial Word Transformation 17
3.2. 動漫文字特徵 18
3.2.1. 中文環境下的Text Segmentation 及Bag-of-Words 18
3.3. 圖文整合分析 20
3.3.1. Feature Combination 20
3.3.2. Feature Correlation between Image and Text 21
第4章 系統實作 25
4.1. 資料來源 25
4.2. 線下部份實作流程 26
4.2.1. Comic Face的訓練與測試 26
4.2.2. Facial Word Matrix的產生 29
4.2.3. 網頁文字處理 31
4.2.4. 圖文矩陣 & 網頁相似度 31
4.3. 網站應用工具及介面 33
第5章 實驗評估 36
5.1. 實驗資料來源 36
5.2. 實驗目的 37
5.3. 實驗設計 37
5.4. 實驗結果 38
第6章 結論與未來工作 43
參考文獻 45		zh_TW
dc.format.extent 2394504 bytes-
dc.format.mimetype application/pdf-
dc.language.iso en_US-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0100753041en_US
dc.subject (關鍵詞) 動漫網站zh_TW
dc.subject (關鍵詞) 搜尋引擎zh_TW
dc.subject (關鍵詞) 圖文關連zh_TW
dc.subject (關鍵詞) 動漫風格zh_TW
dc.subject (關鍵詞) ACG Websitesen_US
dc.subject (關鍵詞) Search Enginesen_US
dc.subject (關鍵詞) Correlation Miningen_US
dc.subject (關鍵詞) Comic Stylesen_US
dc.title (題名) 動漫魔鏡：運用圖文關聯探勘的動漫網站搜索引擎zh_TW
dc.title (題名) Comirror: A Search Engine for Comic Web Based on Textual and Visual Correlation Miningen_US
dc.type (資料類型) thesisen
dc.relation.reference (參考文獻) [1] H. Bay, T. Tuytelaars and L. Van Gool, "SURF: Speeded up Robust Features," European Conference on Computer Vision, 2006.
[2] D. M. Blei, Andrew Ng and M. Jordan. "Latent Dirichlet Allocation," The Journal of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
[3] M. Brown and D. Lowe, "Recognizing Panoramas," The 9th International Conference on Computer Vision, pp. 1218-1227, 2003.
[4] J. Canny. "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.8, No.6, pp. 679-698, 1986.
[5] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE Conference on Computer Vision and Pattern Recognition, Vol.1, pp. 88–893, 2005.
[6] S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman, "Indexing by latent semantic analysis," Journal of the American Society for Information Science and Technology , Vol.41, pp. 391-407, 1990.
[7] T. Gritti , C. Shan , V. Jeanne and R. Braspenning, "Local features based facial expression recognition with face registration errors," Automatic Face & Gesture Recognition, FG `08. 8th IEEE International Conference, pp. 1-8, 2008.
[8] T. Hofmann, "Probabilistic Latent Semantic Analysis," Uncertainty in Artificial Intelligence, UAI’99, pp. 289-296, 1999.
[9] L. Juan and O. Gwun, "A Comparison of SIFT, PCA-SIFT and. SURF," International Journal of Image Processing, Vol. 65, pp. 143-152, 2009.
[10] Y. Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," Computer Vision and Pattern Recognition, Vol.2, 2004.
[11] M. La Cascia , S. Sethi , S. Sclaroff, "Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web," IEEE Workshop on Content - Based Access of Image and Video Libraries, pp.24, June. 1998
[12] R. Lienhart and J. Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection," IEEE International Conference on Image Processing 2002, Vol. 1, pp. 900-903, Sep. 2002.
[13] D. G. Lowe, "Object Recognition from Local Scale-invariant Features," the International Conference on Computer Vision, Vol.2, pp. 1150–1157, 1999.
[14] D. G. Lowe, "Distinctive Image Features from Scale-invariant Keypoints," International Journal of Computer Vision, Vol.60, No.2, pp. 91-110, 2004.
[15] K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp. 1615-1630, 2005.
[16] T. Ojala, M. Pietikäinen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," IAPR International Conference on Pattern Recognition (ICPR), Vol. 1, pp.582–585, 1994.
[17] T. Ojala, M. Pietikäinen, and D. Harwood, "A Comparative Study of Texture Measures with Classification Based on Feature Distributions," Pattern Recognition, Vol. 29, pp. 51-59, 1996.
[18] T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 24, No. 7, pp.971−987, 2002.
[19] T. Reenskaug, "Thing-model-view-editor - an Example from a Planning System, " Xerox PARC Technical Note, 1979.
[20] G. Salton, A. Wong, and C. S. Yang, "A Vector Space Model for Automatic Indexing," Communications of the ACM, Vol. 18, No. 11, pp. 613–620, 1975.
[21] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," IEEE Conference on Computer Vision and Pattern Recognition, 2000.
[22] W. Sun and K. Kise, "Detecting Printed and Handwritten Partial Copies of Line Drawings Embedded in Complex Backgrounds," International Conference on Document Analysis and Recognition, pp. 909–919, 2009.
[23] W. Sun and K. Kise, "Similar Partial Copy Detection of Line Drawings Using a Cascade Classifier and Feature Matching," International Workshop on Computational Forensics, pp.126-137, 2010.
[24] W. Sun and K. Kise, "Similar Manga Retrieval Using Visual Vocabulary Based on Regions of Interest," International Conference on Document Analysis and Recognition, pp. 1075-1079, 2011.
[25] S. Tollari, H. Glotin, "Web Image Retrieval on ImagEVAL: Evidences on Visualness and Textualness Concept Dependency in Fusion Model," ACM International Conference on Image and Video Retrieval (CIVR), July. 2007.
[26] P. Viola and M. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, No. 2, pp.137–154, 2004.
[27] Q. Zhu, S. Avidan, M. Yeh and K. Cheng, "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients," IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp.1491–1498, 2006.
[28] Naotoshi Seo`s note site, http://note.sonots.com/
[29] 百度百科，http://baike.baidu.com/
[30] 日本動漫，http://baike.baidu.com/view/495014.htm		zh_TW