學術產出-Theses
Article View/Open
Publication Export
-
題名 基於資訊理論熵之特徵選取
Entropy based feature selection作者 許立農 貢獻者 周珮婷
許立農關鍵詞 機器學習
特徵選取
維度縮減
entropy
Machine learning
Feature selection
Dimension reduction
Entropy日期 2017 上傳時間 11-Jul-2017 11:25:43 (UTC+8) 摘要 特徵選取為機器學習常見的資料前處理的方法，現今已有許多不同的特徵選取演算法，然而並不存在一個在所有資料上都優於其他方法的演算法，且由於現今的資料種類繁多，所以研發新的方法能夠帶來更多有關資料的資訊並且根據資料的特性採用不同的變數選取演算法是較好的做法。 本研究使用資訊理論entropy的概念依照變數之間資料雲幾何樹的分群結果定義變數之間的相關性，且依此選取資料的特徵，並與同樣使用entropy概念的FCBF方法、Lasso、F-score、隨機森林、基因演算法互相比較，本研究使用階層式分群法與多數決投票法套用在真實的資料上判斷預測率。結果顯示，本研究使用的entropy方法在各個不同的資料集上有較穩定的預測率提升表現，同時資料縮減的維度也相對穩定。
Feature selection is a common preprocessing technique in machine learning. Although a large pool of feature selection techniques has existed, there is no such a dominant method in all datasets. Because of the complexity of various data formats, establishing a new method can bring more insight into data, and applying proper techniques to analyzing data would be the best choice. In this study, we used the concept of entropy from information theory to build a similarity matrix between features. Additionally, we constructed a DCG-tree to separate variables into clusters. Each core cluster consists of rather uniform variables, which share similar covariate information. With the core clusters, we reduced the dimension of a high-dimensional dataset. We assessed our method by comparing it with FCBF, Lasso, F-score, random forest and genetic algorithm. The performances of prediction were demonstrated through real-world datasets using hierarchical clustering with voting algorithm as the classifier. The results showed that our entropy method has more stable prediction performances and reduces sufficient dimensions of the datasets simultaneously.參考文獻 Akay, M. F. (2009). Support vector machines combined with feature selection for breast cancer diagnosis. Expert systems with applications, 36(2), 3240-3247. Chen, Y.-W., & Lin, C.-J. (2006). Combining SVMs with various feature selection strategies Feature extraction (pp. 315-324): Springer.Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7(1), 3. doi:10.1186/1471-2105-7-3Fushing, H., & McAssey, M. P. (2010). Time, temperature, and data cloud geometry. Physical Review E, 82(6), 061110. Fushing, H., Wang, H., VanderWaal, K., McCowan, B., & Koehl, P. (2013). Multi-scale clustering by building a robust and self correcting ultrametric topology on data points. PloS one, 8(2), e56259. Golub, M. S., Hogrefe, C. E., Widaman, K. F., & Capitanio, J. P. (2009). Iron deficiency anemia and affective response in rhesus monkey infants. Developmental psychobiology, 51(1), 47-59. Lee, O. (2017). Data-driven computation for pattern information. ProQuest, UMI Dissertations Publishing.Raymer, M. L., Punch, W. F., Goodman, E. D., Kuhn, L. A., & Jain, A. K. (2000). Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(2), 164-171. doi:10.1109/4235.850656Saeys, Y., Abeel, T., & Van de Peer, Y. (2008). Robust Feature Selection Using Ensemble Feature Selection Techniques. In W. Daelemans, B. Goethals, & K. Morik (Eds.), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II (pp. 313-325). Berlin, Heidelberg: Springer Berlin Heidelberg.Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. bioinformatics, 23(19), 2507-2517. Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004). Application of Breiman’s Random Forest to Modeling Structure-Activity Relationships of Pharmaceutical Molecules. In F. Roli, J. Kittler, & T. Windeatt (Eds.), Multiple Classifier Systems: 5th International Workshop, MCS 2004, Cagliari, Italy, June 9-11, 2004. Proceedings (pp. 334-343). Berlin, Heidelberg: Springer Berlin Heidelberg.Yu, L., & Liu, H. (2003). Feature selection for high-dimensional data: A fast correlation-based filter solution. Paper presented at the ICML. 描述 碩士
國立政治大學
統計學系
104354013資料來源 http://thesis.lib.nccu.edu.tw/record/#G0104354013 資料類型 thesis dc.contributor.advisor 周珮婷 zh_TW dc.contributor.author (Authors) 許立農 zh_TW dc.creator (作者) 許立農 zh_TW dc.date (日期) 2017 en_US dc.date.accessioned 11-Jul-2017 11:25:43 (UTC+8) - dc.date.available 11-Jul-2017 11:25:43 (UTC+8) - dc.date.issued (上傳時間) 11-Jul-2017 11:25:43 (UTC+8) - dc.identifier (Other Identifiers) G0104354013 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/110782 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 統計學系 zh_TW dc.description (描述) 104354013 zh_TW dc.description.abstract (摘要) 特徵選取為機器學習常見的資料前處理的方法，現今已有許多不同的特徵選取演算法，然而並不存在一個在所有資料上都優於其他方法的演算法，且由於現今的資料種類繁多，所以研發新的方法能夠帶來更多有關資料的資訊並且根據資料的特性採用不同的變數選取演算法是較好的做法。 本研究使用資訊理論entropy的概念依照變數之間資料雲幾何樹的分群結果定義變數之間的相關性，且依此選取資料的特徵，並與同樣使用entropy概念的FCBF方法、Lasso、F-score、隨機森林、基因演算法互相比較，本研究使用階層式分群法與多數決投票法套用在真實的資料上判斷預測率。結果顯示，本研究使用的entropy方法在各個不同的資料集上有較穩定的預測率提升表現，同時資料縮減的維度也相對穩定。 zh_TW dc.description.abstract (摘要) Feature selection is a common preprocessing technique in machine learning. Although a large pool of feature selection techniques has existed, there is no such a dominant method in all datasets. Because of the complexity of various data formats, establishing a new method can bring more insight into data, and applying proper techniques to analyzing data would be the best choice. In this study, we used the concept of entropy from information theory to build a similarity matrix between features. Additionally, we constructed a DCG-tree to separate variables into clusters. Each core cluster consists of rather uniform variables, which share similar covariate information. With the core clusters, we reduced the dimension of a high-dimensional dataset. We assessed our method by comparing it with FCBF, Lasso, F-score, random forest and genetic algorithm. The performances of prediction were demonstrated through real-world datasets using hierarchical clustering with voting algorithm as the classifier. The results showed that our entropy method has more stable prediction performances and reduces sufficient dimensions of the datasets simultaneously. en_US dc.description.tableofcontents 第一章 緒論 1第二章 文獻探討 3第三章 資料介紹與敘述 9 一、生物行為評估專案 9 二、美國威斯康辛州診斷乳癌 11 三、Connectionist Bench(Sonar, Mines vs. Rocks)13 四、SPECTF heart 15第四章 研究方法與過程 17 第一節 分類預測模型 17 ㄧ、階層式分群法(Hierarchical clustering) 17 二、多數決投票法(Voting) 17 第二節 變數選擇方法 19 ㄧ、不選擇：直接使用所有的變數 19 二、entropy 應用 19 三、FCBF(Fast Correlation-Based Filter) 25 四、Lasso 27 五、F-score 28 六、隨機森林（Random Forest） 29 七、遺傳演算法（Genetic Algorithm） 29 第三節 研究過程 30 ㄧ、entropy 應用 30 二、Lasso 31 三、F-score 32 四、隨機森林 33 五、遺傳演算法 34第五章 研究結果與結論 35 第一節 研究結果 35 ㄧ、生物行為評估專案資料集 36 二、美國威斯康辛州診斷乳癌資料集 38 三、聲納 40 四、SPECTF 42 第二節 結論 43參考文獻 45 zh_TW dc.format.extent 6507025 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0104354013 en_US dc.subject (關鍵詞) 機器學習 zh_TW dc.subject (關鍵詞) 特徵選取 zh_TW dc.subject (關鍵詞) 維度縮減 zh_TW dc.subject (關鍵詞) entropy zh_TW dc.subject (關鍵詞) Machine learning en_US dc.subject (關鍵詞) Feature selection en_US dc.subject (關鍵詞) Dimension reduction en_US dc.subject (關鍵詞) Entropy en_US dc.title (題名) 基於資訊理論熵之特徵選取 zh_TW dc.title (題名) Entropy based feature selection en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) Akay, M. F. (2009). Support vector machines combined with feature selection for breast cancer diagnosis. Expert systems with applications, 36(2), 3240-3247. Chen, Y.-W., & Lin, C.-J. (2006). Combining SVMs with various feature selection strategies Feature extraction (pp. 315-324): Springer.Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7(1), 3. doi:10.1186/1471-2105-7-3Fushing, H., & McAssey, M. P. (2010). Time, temperature, and data cloud geometry. Physical Review E, 82(6), 061110. Fushing, H., Wang, H., VanderWaal, K., McCowan, B., & Koehl, P. (2013). Multi-scale clustering by building a robust and self correcting ultrametric topology on data points. PloS one, 8(2), e56259. Golub, M. S., Hogrefe, C. E., Widaman, K. F., & Capitanio, J. P. (2009). Iron deficiency anemia and affective response in rhesus monkey infants. Developmental psychobiology, 51(1), 47-59. Lee, O. (2017). Data-driven computation for pattern information. ProQuest, UMI Dissertations Publishing.Raymer, M. L., Punch, W. F., Goodman, E. D., Kuhn, L. A., & Jain, A. K. (2000). Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(2), 164-171. doi:10.1109/4235.850656Saeys, Y., Abeel, T., & Van de Peer, Y. (2008). Robust Feature Selection Using Ensemble Feature Selection Techniques. In W. Daelemans, B. Goethals, & K. Morik (Eds.), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II (pp. 313-325). Berlin, Heidelberg: Springer Berlin Heidelberg.Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. bioinformatics, 23(19), 2507-2517. Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004). Application of Breiman’s Random Forest to Modeling Structure-Activity Relationships of Pharmaceutical Molecules. In F. Roli, J. Kittler, & T. Windeatt (Eds.), Multiple Classifier Systems: 5th International Workshop, MCS 2004, Cagliari, Italy, June 9-11, 2004. Proceedings (pp. 334-343). Berlin, Heidelberg: Springer Berlin Heidelberg.Yu, L., & Liu, H. (2003). Feature selection for high-dimensional data: A fast correlation-based filter solution. Paper presented at the ICML. zh_TW
