Publications-Theses

Title以眼動資訊增進基於內容的圖像檢索效能
Improving the Performance of Content Based Image Retrieval by Eye Tracking
Creator張京文
Jhang ,Jing Wun
Contributor陳良弼<br>蔡介立
Chen, Arbee L.P.<br>Tsai, Jie Li
張京文
Jhang ,Jing Wun
Key Words圖像檢索
眼動軌跡
眼動資訊
image retrieval
eye tracking
eye movement
Date2008
Date Issued19-Sep-2009 12:10:39 (UTC+8)
Summary在現今的基於內容的圖像檢索的研究中,會將人的主觀認知考慮進去。因為傳統的圖像檢索中採取低階特徵來找出圖片上可能的重要區域的方法和人的感覺還是有著相當大的語意上的鴻溝。然而藉由考慮人對圖片的主觀認知,可以讓人找到對它而言圖片上重要的部分,再去做圖像檢索,找出使用者想要的圖片。這樣的作法是比較自然且直觀的。還能達到個人化的效果,因為每個人對同一張圖片上覺得重要的物體可能不盡相同。在本論文中的圖像檢索系統採用眼動軌跡當作人的主觀認知來輔助檢索。因為在心理學的研究中有提到,人在看圖片的時候會有較多的凝視點落在他覺得重要的區域上。所以藉由這個理論,本論文利用使用者看圖片的眼動軌跡即時的調整圖片上物體的重要性。最後將重要性高的數個物體去做圖像檢索,找出含有這些對這個使用者是重要的物體的圖片。經由實驗證實,眼動軌跡輔助圖像檢索的確可以減少不重要的物體對圖像檢索的干擾,繼而可以提升圖像檢索系統的效能。
Recently, researches in Content-Based Image Retrieval (CBIR) focuses on incorporation of knowledge about human perception in the systems’ design and implementation process. This enables the design of more natural and intuitive image retrieval techniques in order to overcome some of the challenges faced by modern CBIR system such as the difficulty to extract important regions of an image. By researches of psychology, user’s eye tracking reflects his interest. So, in my CBIR system, user’s eye movements were used online to adjust the importance for objects in query image. Thus in my system, only those images with important objects will be retrieved. One experiment was performed: record the eye movement of participants on query images. Then compare my approach with a classic CBIR system according to performance. The results reveal that higher retrieval performance of my image retrieval system because of decreasing the influence of not importance objects to image retrieval system.
參考文獻 [1] Nozha Boujemaa, Julien Fauqueur,and Valérie Gouet, “What`s beyond query by example?, “ International Conference on Image and Signal Processing, 2003.
[2]Songhe Feng, De Xu, Xu Yang, and Aimin Wu, ”A Novel Region-Based Image Retrieval Algorithm Using Selective Visual Attention Model,” International Conference on Advanced Concept for Intelligent Vision system, pp.235-242,2005.

[3] John M Henderson, and A Hollingworth, “High-Level Scene Perception, Annual Review of Psychology,” Annual Review of Psychology, vol. 50, pp.243-271, 1999.
[4] Essig Kai , and Ritter Helge, “Visual-Based Image Retrieval (VBIR) - A New Approach for Natural and Intuitive Image Retrieval,” Proceedings of the 13th European Conference on Eye Movements, Aug.2005.
[5] Scherffig Lasse, “It`s in Your Eyes - Gaze Based Image Retrieval in Context,” Edited by Hans H. Diebner, Institute for Basic Research, Karlsruhe, 2005.
[6]O. Marques, L. M. Mayron,G. B. Borba, and H. R. Gamba, ”An Attention-Driven Model for Grouping Similar Images with Image Retrieval Applications,” Journal on Advances in Signal Processing, 2007.
[7] O.K. Oyekoya, “Eye Tracking: A Perceptual Interface for Content Based Image Retrieval,” Ph.D. Thesis, Department of Electronic & Electrical Engineering Adastral Park Campus University College London, April 2007.
[8] W Osberger, and A.J.Maeder, ”Automatic Identification of Perceptually Important Regions in an Image,” IEEE Proceedings, Fourteenth International Conference on Pattern Recognition, vol.1,pp. 701-704, 1998.
[9]K Rayner, A. W. Inhoff, R.E. Morrison, M.L. Slowiaczek, ” Masking of foveal and parafoveal vision during eye fixations in reading.” Journal of Experiment Psychology:Human Perception and Performance, vol.7,pp. 167-179,1981.
[10] Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, and Andrew Zisserman, ” Using Multiple Segmentations to Discover Objects and their Extent in Image Collections,” IEEE Conference on Computer Vision and Pattern Recognition, 2006.
[11] Dirk Walther and Christof Koch,” Modeling attention to salient proto-objects,” Neural networks, pp.1395 -1407, 2006.
[12]Jing Zhang, Lansun Shen, and David Dagan Feng,”A Personalized Image Retrieval Based on Visual Perception,” Journal of Electronics (China), Jan. 2008.
Description碩士
國立政治大學
資訊科學學系
95753036
97
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0095753036
Typethesis
dc.contributor.advisor 陳良弼<br>蔡介立zh_TW
dc.contributor.advisor Chen, Arbee L.P.<br>Tsai, Jie Lien_US
dc.contributor.author (Authors) 張京文zh_TW
dc.contributor.author (Authors) Jhang ,Jing Wunen_US
dc.creator (作者) 張京文zh_TW
dc.creator (作者) Jhang ,Jing Wunen_US
dc.date (日期) 2008en_US
dc.date.accessioned 19-Sep-2009 12:10:39 (UTC+8)-
dc.date.available 19-Sep-2009 12:10:39 (UTC+8)-
dc.date.issued (上傳時間) 19-Sep-2009 12:10:39 (UTC+8)-
dc.identifier (Other Identifiers) G0095753036en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/37111-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學學系zh_TW
dc.description (描述) 95753036zh_TW
dc.description (描述) 97zh_TW
dc.description.abstract (摘要) 在現今的基於內容的圖像檢索的研究中,會將人的主觀認知考慮進去。因為傳統的圖像檢索中採取低階特徵來找出圖片上可能的重要區域的方法和人的感覺還是有著相當大的語意上的鴻溝。然而藉由考慮人對圖片的主觀認知,可以讓人找到對它而言圖片上重要的部分,再去做圖像檢索,找出使用者想要的圖片。這樣的作法是比較自然且直觀的。還能達到個人化的效果,因為每個人對同一張圖片上覺得重要的物體可能不盡相同。在本論文中的圖像檢索系統採用眼動軌跡當作人的主觀認知來輔助檢索。因為在心理學的研究中有提到,人在看圖片的時候會有較多的凝視點落在他覺得重要的區域上。所以藉由這個理論,本論文利用使用者看圖片的眼動軌跡即時的調整圖片上物體的重要性。最後將重要性高的數個物體去做圖像檢索,找出含有這些對這個使用者是重要的物體的圖片。經由實驗證實,眼動軌跡輔助圖像檢索的確可以減少不重要的物體對圖像檢索的干擾,繼而可以提升圖像檢索系統的效能。zh_TW
dc.description.abstract (摘要) Recently, researches in Content-Based Image Retrieval (CBIR) focuses on incorporation of knowledge about human perception in the systems’ design and implementation process. This enables the design of more natural and intuitive image retrieval techniques in order to overcome some of the challenges faced by modern CBIR system such as the difficulty to extract important regions of an image. By researches of psychology, user’s eye tracking reflects his interest. So, in my CBIR system, user’s eye movements were used online to adjust the importance for objects in query image. Thus in my system, only those images with important objects will be retrieved. One experiment was performed: record the eye movement of participants on query images. Then compare my approach with a classic CBIR system according to performance. The results reveal that higher retrieval performance of my image retrieval system because of decreasing the influence of not importance objects to image retrieval system.en_US
dc.description.tableofcontents 中文摘要.............................................. i
英文摘要..............................................ii
致謝................................................ iii
目錄................................................. iv
表目錄............................................... vi
圖目錄...............................................vii
第一章 導論及研究動機................................. 1
第二章 相關研究....................................... 3
2.1 眼動軌跡...........................................3
2.2 視覺注意力.........................................4
2.3 以眼動輔助圖像檢索的相關研究.......................6
第三章 眼動輔助圖片檢索系統架構.......................11
3.1 圖片的處理........................................12
3.1.1 圖片的物體擷取..................................12
3.1.2 圖片的相似度定義................................16
3.2 眼動軌跡的處理....................................17
3.2.1 眼動軌跡的前處理................................17
3.2.2 以凝視點計算圖片上物體的重要性..................18
3.2.3 找出重要的物體..................................21
3.3 與相似論文方法之比較..............................22
3.3.1 凝視點與在看的物體的關係........................23
3.3.2 距離區域中心較遠的凝視點較不重要................24
第四章 實驗方法與驗證.................................26
4.1 實驗方法..........................................26
4.1.1 眼動儀與實驗環境介紹............................26
4.1.2 圖片資料庫的選擇................................26
4.1.3 實驗流程........................................28
4.2 實驗結果初步分析..................................29
第五章 結論以及未來的展望.............................33
5.1 結論..............................................33
5.2 未來展望..........................................34
5.2.1 改善物體擷取....................................34
5.2.2 改善選擇重要物體的方法..........................35
5.2.3 系統開發與其他眼動行為..........................36

表目錄
表4.1:受測者對檢索系統滿意表。....................................30
表4.2:總凝視時間較高的物體與使用者認為圖片相似的物體的對應表。....................................31
表5.1 每位受測者對每張圖片的每一物體的總凝視時間與所有物體的總凝視時
間的百分比經小數點以下四捨五入的結果,單位為%,由左到右依照物體的順序,
單位為ms。..............................35

圖目錄
圖2.1:眼動軌跡示意圖...................................4
圖2.2:Saliency map 示意圖..............................6
圖2.3:Kai[4]論文中圖片計算圖片重要區域的示意圖.........9
圖3.1:本論文之眼動輔助圖像檢索系統示意圖..............12
圖3.2:圖片的分割和saliency map 範例...................14
圖3.3:利用Walther[11]這篇論文提出的工具輔助找出圖片上saliency map 顯著的區域.......................................15
圖3.4:利用註釋的工具記錄物體的外型和名稱..............16
圖3.5:原始眼動軌跡格式................................17
圖3.6:原始眼動資料中注視點的格式......................18
圖3.7:某一張圖片的凝視點..............................24
圖4.1:本論文中的實驗程式顯示檢索結果的介面............29
zh_TW
dc.format.extent 118958 bytes-
dc.format.extent 123760 bytes-
dc.format.extent 119567 bytes-
dc.format.extent 196218 bytes-
dc.format.extent 124770 bytes-
dc.format.extent 235512 bytes-
dc.format.extent 511453 bytes-
dc.format.extent 225985 bytes-
dc.format.extent 170092 bytes-
dc.format.extent 89031 bytes-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.language.iso en_US-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0095753036en_US
dc.subject (關鍵詞) 圖像檢索zh_TW
dc.subject (關鍵詞) 眼動軌跡zh_TW
dc.subject (關鍵詞) 眼動資訊zh_TW
dc.subject (關鍵詞) image retrievalen_US
dc.subject (關鍵詞) eye trackingen_US
dc.subject (關鍵詞) eye movementen_US
dc.title (題名) 以眼動資訊增進基於內容的圖像檢索效能zh_TW
dc.title (題名) Improving the Performance of Content Based Image Retrieval by Eye Trackingen_US
dc.type (資料類型) thesisen
dc.relation.reference (參考文獻) [1] Nozha Boujemaa, Julien Fauqueur,and Valérie Gouet, “What`s beyond query by example?, “ International Conference on Image and Signal Processing, 2003.zh_TW
dc.relation.reference (參考文獻) [2]Songhe Feng, De Xu, Xu Yang, and Aimin Wu, ”A Novel Region-Based Image Retrieval Algorithm Using Selective Visual Attention Model,” International Conference on Advanced Concept for Intelligent Vision system, pp.235-242,2005.zh_TW
dc.relation.reference (參考文獻) zh_TW
dc.relation.reference (參考文獻) [3] John M Henderson, and A Hollingworth, “High-Level Scene Perception, Annual Review of Psychology,” Annual Review of Psychology, vol. 50, pp.243-271, 1999.zh_TW
dc.relation.reference (參考文獻) [4] Essig Kai , and Ritter Helge, “Visual-Based Image Retrieval (VBIR) - A New Approach for Natural and Intuitive Image Retrieval,” Proceedings of the 13th European Conference on Eye Movements, Aug.2005.zh_TW
dc.relation.reference (參考文獻) [5] Scherffig Lasse, “It`s in Your Eyes - Gaze Based Image Retrieval in Context,” Edited by Hans H. Diebner, Institute for Basic Research, Karlsruhe, 2005.zh_TW
dc.relation.reference (參考文獻) [6]O. Marques, L. M. Mayron,G. B. Borba, and H. R. Gamba, ”An Attention-Driven Model for Grouping Similar Images with Image Retrieval Applications,” Journal on Advances in Signal Processing, 2007.zh_TW
dc.relation.reference (參考文獻) [7] O.K. Oyekoya, “Eye Tracking: A Perceptual Interface for Content Based Image Retrieval,” Ph.D. Thesis, Department of Electronic & Electrical Engineering Adastral Park Campus University College London, April 2007.zh_TW
dc.relation.reference (參考文獻) [8] W Osberger, and A.J.Maeder, ”Automatic Identification of Perceptually Important Regions in an Image,” IEEE Proceedings, Fourteenth International Conference on Pattern Recognition, vol.1,pp. 701-704, 1998.zh_TW
dc.relation.reference (參考文獻) [9]K Rayner, A. W. Inhoff, R.E. Morrison, M.L. Slowiaczek, ” Masking of foveal and parafoveal vision during eye fixations in reading.” Journal of Experiment Psychology:Human Perception and Performance, vol.7,pp. 167-179,1981.zh_TW
dc.relation.reference (參考文獻) [10] Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, and Andrew Zisserman, ” Using Multiple Segmentations to Discover Objects and their Extent in Image Collections,” IEEE Conference on Computer Vision and Pattern Recognition, 2006.zh_TW
dc.relation.reference (參考文獻) [11] Dirk Walther and Christof Koch,” Modeling attention to salient proto-objects,” Neural networks, pp.1395 -1407, 2006.zh_TW
dc.relation.reference (參考文獻) [12]Jing Zhang, Lansun Shen, and David Dagan Feng,”A Personalized Image Retrieval Based on Visual Perception,” Journal of Electronics (China), Jan. 2008.zh_TW