Publications-Theses
Article View/Open
Publication Export
Google ScholarTM
NCCU Library
Citation Infomation
Related Publications in TAIR
Title | 以眼動資訊增進基於內容的圖像檢索效能 Improving the Performance of Content Based Image Retrieval by Eye Tracking |
Creator | 張京文 Jhang ,Jing Wun |
Contributor | 陳良弼<br>蔡介立 Chen, Arbee L.P.<br>Tsai, Jie Li 張京文 Jhang ,Jing Wun |
Key Words | 圖像檢索 眼動軌跡 眼動資訊 image retrieval eye tracking eye movement |
Date | 2008 |
Date Issued | 19-Sep-2009 12:10:39 (UTC+8) |
Summary | 在現今的基於內容的圖像檢索的研究中,會將人的主觀認知考慮進去。因為傳統的圖像檢索中採取低階特徵來找出圖片上可能的重要區域的方法和人的感覺還是有著相當大的語意上的鴻溝。然而藉由考慮人對圖片的主觀認知,可以讓人找到對它而言圖片上重要的部分,再去做圖像檢索,找出使用者想要的圖片。這樣的作法是比較自然且直觀的。還能達到個人化的效果,因為每個人對同一張圖片上覺得重要的物體可能不盡相同。在本論文中的圖像檢索系統採用眼動軌跡當作人的主觀認知來輔助檢索。因為在心理學的研究中有提到,人在看圖片的時候會有較多的凝視點落在他覺得重要的區域上。所以藉由這個理論,本論文利用使用者看圖片的眼動軌跡即時的調整圖片上物體的重要性。最後將重要性高的數個物體去做圖像檢索,找出含有這些對這個使用者是重要的物體的圖片。經由實驗證實,眼動軌跡輔助圖像檢索的確可以減少不重要的物體對圖像檢索的干擾,繼而可以提升圖像檢索系統的效能。 Recently, researches in Content-Based Image Retrieval (CBIR) focuses on incorporation of knowledge about human perception in the systems’ design and implementation process. This enables the design of more natural and intuitive image retrieval techniques in order to overcome some of the challenges faced by modern CBIR system such as the difficulty to extract important regions of an image. By researches of psychology, user’s eye tracking reflects his interest. So, in my CBIR system, user’s eye movements were used online to adjust the importance for objects in query image. Thus in my system, only those images with important objects will be retrieved. One experiment was performed: record the eye movement of participants on query images. Then compare my approach with a classic CBIR system according to performance. The results reveal that higher retrieval performance of my image retrieval system because of decreasing the influence of not importance objects to image retrieval system. |
參考文獻 | [1] Nozha Boujemaa, Julien Fauqueur,and Valérie Gouet, “What`s beyond query by example?, “ International Conference on Image and Signal Processing, 2003. [2]Songhe Feng, De Xu, Xu Yang, and Aimin Wu, ”A Novel Region-Based Image Retrieval Algorithm Using Selective Visual Attention Model,” International Conference on Advanced Concept for Intelligent Vision system, pp.235-242,2005. [3] John M Henderson, and A Hollingworth, “High-Level Scene Perception, Annual Review of Psychology,” Annual Review of Psychology, vol. 50, pp.243-271, 1999. [4] Essig Kai , and Ritter Helge, “Visual-Based Image Retrieval (VBIR) - A New Approach for Natural and Intuitive Image Retrieval,” Proceedings of the 13th European Conference on Eye Movements, Aug.2005. [5] Scherffig Lasse, “It`s in Your Eyes - Gaze Based Image Retrieval in Context,” Edited by Hans H. Diebner, Institute for Basic Research, Karlsruhe, 2005. [6]O. Marques, L. M. Mayron,G. B. Borba, and H. R. Gamba, ”An Attention-Driven Model for Grouping Similar Images with Image Retrieval Applications,” Journal on Advances in Signal Processing, 2007. [7] O.K. Oyekoya, “Eye Tracking: A Perceptual Interface for Content Based Image Retrieval,” Ph.D. Thesis, Department of Electronic & Electrical Engineering Adastral Park Campus University College London, April 2007. [8] W Osberger, and A.J.Maeder, ”Automatic Identification of Perceptually Important Regions in an Image,” IEEE Proceedings, Fourteenth International Conference on Pattern Recognition, vol.1,pp. 701-704, 1998. [9]K Rayner, A. W. Inhoff, R.E. Morrison, M.L. Slowiaczek, ” Masking of foveal and parafoveal vision during eye fixations in reading.” Journal of Experiment Psychology:Human Perception and Performance, vol.7,pp. 167-179,1981. [10] Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, and Andrew Zisserman, ” Using Multiple Segmentations to Discover Objects and their Extent in Image Collections,” IEEE Conference on Computer Vision and Pattern Recognition, 2006. [11] Dirk Walther and Christof Koch,” Modeling attention to salient proto-objects,” Neural networks, pp.1395 -1407, 2006. [12]Jing Zhang, Lansun Shen, and David Dagan Feng,”A Personalized Image Retrieval Based on Visual Perception,” Journal of Electronics (China), Jan. 2008. |
Description | 碩士 國立政治大學 資訊科學學系 95753036 97 |
資料來源 | http://thesis.lib.nccu.edu.tw/record/#G0095753036 |
Type | thesis |
dc.contributor.advisor | 陳良弼<br>蔡介立 | zh_TW |
dc.contributor.advisor | Chen, Arbee L.P.<br>Tsai, Jie Li | en_US |
dc.contributor.author (Authors) | 張京文 | zh_TW |
dc.contributor.author (Authors) | Jhang ,Jing Wun | en_US |
dc.creator (作者) | 張京文 | zh_TW |
dc.creator (作者) | Jhang ,Jing Wun | en_US |
dc.date (日期) | 2008 | en_US |
dc.date.accessioned | 19-Sep-2009 12:10:39 (UTC+8) | - |
dc.date.available | 19-Sep-2009 12:10:39 (UTC+8) | - |
dc.date.issued (上傳時間) | 19-Sep-2009 12:10:39 (UTC+8) | - |
dc.identifier (Other Identifiers) | G0095753036 | en_US |
dc.identifier.uri (URI) | https://nccur.lib.nccu.edu.tw/handle/140.119/37111 | - |
dc.description (描述) | 碩士 | zh_TW |
dc.description (描述) | 國立政治大學 | zh_TW |
dc.description (描述) | 資訊科學學系 | zh_TW |
dc.description (描述) | 95753036 | zh_TW |
dc.description (描述) | 97 | zh_TW |
dc.description.abstract (摘要) | 在現今的基於內容的圖像檢索的研究中,會將人的主觀認知考慮進去。因為傳統的圖像檢索中採取低階特徵來找出圖片上可能的重要區域的方法和人的感覺還是有著相當大的語意上的鴻溝。然而藉由考慮人對圖片的主觀認知,可以讓人找到對它而言圖片上重要的部分,再去做圖像檢索,找出使用者想要的圖片。這樣的作法是比較自然且直觀的。還能達到個人化的效果,因為每個人對同一張圖片上覺得重要的物體可能不盡相同。在本論文中的圖像檢索系統採用眼動軌跡當作人的主觀認知來輔助檢索。因為在心理學的研究中有提到,人在看圖片的時候會有較多的凝視點落在他覺得重要的區域上。所以藉由這個理論,本論文利用使用者看圖片的眼動軌跡即時的調整圖片上物體的重要性。最後將重要性高的數個物體去做圖像檢索,找出含有這些對這個使用者是重要的物體的圖片。經由實驗證實,眼動軌跡輔助圖像檢索的確可以減少不重要的物體對圖像檢索的干擾,繼而可以提升圖像檢索系統的效能。 | zh_TW |
dc.description.abstract (摘要) | Recently, researches in Content-Based Image Retrieval (CBIR) focuses on incorporation of knowledge about human perception in the systems’ design and implementation process. This enables the design of more natural and intuitive image retrieval techniques in order to overcome some of the challenges faced by modern CBIR system such as the difficulty to extract important regions of an image. By researches of psychology, user’s eye tracking reflects his interest. So, in my CBIR system, user’s eye movements were used online to adjust the importance for objects in query image. Thus in my system, only those images with important objects will be retrieved. One experiment was performed: record the eye movement of participants on query images. Then compare my approach with a classic CBIR system according to performance. The results reveal that higher retrieval performance of my image retrieval system because of decreasing the influence of not importance objects to image retrieval system. | en_US |
dc.description.tableofcontents | 中文摘要.............................................. i 英文摘要..............................................ii 致謝................................................ iii 目錄................................................. iv 表目錄............................................... vi 圖目錄...............................................vii 第一章 導論及研究動機................................. 1 第二章 相關研究....................................... 3 2.1 眼動軌跡...........................................3 2.2 視覺注意力.........................................4 2.3 以眼動輔助圖像檢索的相關研究.......................6 第三章 眼動輔助圖片檢索系統架構.......................11 3.1 圖片的處理........................................12 3.1.1 圖片的物體擷取..................................12 3.1.2 圖片的相似度定義................................16 3.2 眼動軌跡的處理....................................17 3.2.1 眼動軌跡的前處理................................17 3.2.2 以凝視點計算圖片上物體的重要性..................18 3.2.3 找出重要的物體..................................21 3.3 與相似論文方法之比較..............................22 3.3.1 凝視點與在看的物體的關係........................23 3.3.2 距離區域中心較遠的凝視點較不重要................24 第四章 實驗方法與驗證.................................26 4.1 實驗方法..........................................26 4.1.1 眼動儀與實驗環境介紹............................26 4.1.2 圖片資料庫的選擇................................26 4.1.3 實驗流程........................................28 4.2 實驗結果初步分析..................................29 第五章 結論以及未來的展望.............................33 5.1 結論..............................................33 5.2 未來展望..........................................34 5.2.1 改善物體擷取....................................34 5.2.2 改善選擇重要物體的方法..........................35 5.2.3 系統開發與其他眼動行為..........................36 表目錄 表4.1:受測者對檢索系統滿意表。....................................30 表4.2:總凝視時間較高的物體與使用者認為圖片相似的物體的對應表。....................................31 表5.1 每位受測者對每張圖片的每一物體的總凝視時間與所有物體的總凝視時 間的百分比經小數點以下四捨五入的結果,單位為%,由左到右依照物體的順序, 單位為ms。..............................35 圖目錄 圖2.1:眼動軌跡示意圖...................................4 圖2.2:Saliency map 示意圖..............................6 圖2.3:Kai[4]論文中圖片計算圖片重要區域的示意圖.........9 圖3.1:本論文之眼動輔助圖像檢索系統示意圖..............12 圖3.2:圖片的分割和saliency map 範例...................14 圖3.3:利用Walther[11]這篇論文提出的工具輔助找出圖片上saliency map 顯著的區域.......................................15 圖3.4:利用註釋的工具記錄物體的外型和名稱..............16 圖3.5:原始眼動軌跡格式................................17 圖3.6:原始眼動資料中注視點的格式......................18 圖3.7:某一張圖片的凝視點..............................24 圖4.1:本論文中的實驗程式顯示檢索結果的介面............29 | zh_TW |
dc.format.extent | 118958 bytes | - |
dc.format.extent | 123760 bytes | - |
dc.format.extent | 119567 bytes | - |
dc.format.extent | 196218 bytes | - |
dc.format.extent | 124770 bytes | - |
dc.format.extent | 235512 bytes | - |
dc.format.extent | 511453 bytes | - |
dc.format.extent | 225985 bytes | - |
dc.format.extent | 170092 bytes | - |
dc.format.extent | 89031 bytes | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.format.mimetype | application/pdf | - |
dc.language.iso | en_US | - |
dc.source.uri (資料來源) | http://thesis.lib.nccu.edu.tw/record/#G0095753036 | en_US |
dc.subject (關鍵詞) | 圖像檢索 | zh_TW |
dc.subject (關鍵詞) | 眼動軌跡 | zh_TW |
dc.subject (關鍵詞) | 眼動資訊 | zh_TW |
dc.subject (關鍵詞) | image retrieval | en_US |
dc.subject (關鍵詞) | eye tracking | en_US |
dc.subject (關鍵詞) | eye movement | en_US |
dc.title (題名) | 以眼動資訊增進基於內容的圖像檢索效能 | zh_TW |
dc.title (題名) | Improving the Performance of Content Based Image Retrieval by Eye Tracking | en_US |
dc.type (資料類型) | thesis | en |
dc.relation.reference (參考文獻) | [1] Nozha Boujemaa, Julien Fauqueur,and Valérie Gouet, “What`s beyond query by example?, “ International Conference on Image and Signal Processing, 2003. | zh_TW |
dc.relation.reference (參考文獻) | [2]Songhe Feng, De Xu, Xu Yang, and Aimin Wu, ”A Novel Region-Based Image Retrieval Algorithm Using Selective Visual Attention Model,” International Conference on Advanced Concept for Intelligent Vision system, pp.235-242,2005. | zh_TW |
dc.relation.reference (參考文獻) | zh_TW | |
dc.relation.reference (參考文獻) | [3] John M Henderson, and A Hollingworth, “High-Level Scene Perception, Annual Review of Psychology,” Annual Review of Psychology, vol. 50, pp.243-271, 1999. | zh_TW |
dc.relation.reference (參考文獻) | [4] Essig Kai , and Ritter Helge, “Visual-Based Image Retrieval (VBIR) - A New Approach for Natural and Intuitive Image Retrieval,” Proceedings of the 13th European Conference on Eye Movements, Aug.2005. | zh_TW |
dc.relation.reference (參考文獻) | [5] Scherffig Lasse, “It`s in Your Eyes - Gaze Based Image Retrieval in Context,” Edited by Hans H. Diebner, Institute for Basic Research, Karlsruhe, 2005. | zh_TW |
dc.relation.reference (參考文獻) | [6]O. Marques, L. M. Mayron,G. B. Borba, and H. R. Gamba, ”An Attention-Driven Model for Grouping Similar Images with Image Retrieval Applications,” Journal on Advances in Signal Processing, 2007. | zh_TW |
dc.relation.reference (參考文獻) | [7] O.K. Oyekoya, “Eye Tracking: A Perceptual Interface for Content Based Image Retrieval,” Ph.D. Thesis, Department of Electronic & Electrical Engineering Adastral Park Campus University College London, April 2007. | zh_TW |
dc.relation.reference (參考文獻) | [8] W Osberger, and A.J.Maeder, ”Automatic Identification of Perceptually Important Regions in an Image,” IEEE Proceedings, Fourteenth International Conference on Pattern Recognition, vol.1,pp. 701-704, 1998. | zh_TW |
dc.relation.reference (參考文獻) | [9]K Rayner, A. W. Inhoff, R.E. Morrison, M.L. Slowiaczek, ” Masking of foveal and parafoveal vision during eye fixations in reading.” Journal of Experiment Psychology:Human Perception and Performance, vol.7,pp. 167-179,1981. | zh_TW |
dc.relation.reference (參考文獻) | [10] Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, and Andrew Zisserman, ” Using Multiple Segmentations to Discover Objects and their Extent in Image Collections,” IEEE Conference on Computer Vision and Pattern Recognition, 2006. | zh_TW |
dc.relation.reference (參考文獻) | [11] Dirk Walther and Christof Koch,” Modeling attention to salient proto-objects,” Neural networks, pp.1395 -1407, 2006. | zh_TW |
dc.relation.reference (參考文獻) | [12]Jing Zhang, Lansun Shen, and David Dagan Feng,”A Personalized Image Retrieval Based on Visual Perception,” Journal of Electronics (China), Jan. 2008. | zh_TW |