1. NCCU Academic Hub
  2. Publications
  3. College of Commerce
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 基於資訊理論熵之特徵選取
Entropy based feature selection
作者 許立農
貢獻者 周珮婷
許立農
關鍵詞 機器學習
特徵選取
維度縮減
entropy
Machine learning
Feature selection
Dimension reduction
Entropy
日期 2017
上傳時間 11-Jul-2017 11:25:43 (UTC+8)
摘要 特徵選取為機器學習常見的資料前處理的方法，現今已有許多不同的特徵選取演算法，然而並不存在一個在所有資料上都優於其他方法的演算法，且由於現今的資料種類繁多，所以研發新的方法能夠帶來更多有關資料的資訊並且根據資料的特性採用不同的變數選取演算法是較好的做法。
本研究使用資訊理論entropy的概念依照變數之間資料雲幾何樹的分群結果定義變數之間的相關性，且依此選取資料的特徵，並與同樣使用entropy概念的FCBF方法、Lasso、F-score、隨機森林、基因演算法互相比較，本研究使用階層式分群法與多數決投票法套用在真實的資料上判斷預測率。結果顯示，本研究使用的entropy方法在各個不同的資料集上有較穩定的預測率提升表現，同時資料縮減的維度也相對穩定。
Feature selection is a common preprocessing technique in machine learning. Although a large pool of feature selection techniques has existed, there is no such a dominant method in all datasets. Because of the complexity of various data formats, establishing a new method can bring more insight into data, and applying proper techniques to analyzing data would be the best choice.
In this study, we used the concept of entropy from information theory to build a similarity matrix between features. Additionally, we constructed a DCG-tree to separate variables into clusters. Each core cluster consists of rather uniform variables, which share similar covariate information. With the core clusters, we reduced the dimension of a high-dimensional dataset. We assessed our method by comparing it with FCBF, Lasso, F-score, random forest and genetic algorithm. The performances of prediction were demonstrated through real-world datasets using hierarchical clustering with voting algorithm as the classifier. The results showed that our entropy method has more stable prediction performances and reduces sufficient dimensions of the datasets simultaneously.
參考文獻 Akay, M. F. (2009). Support vector machines combined with feature selection for breast cancer diagnosis. Expert systems with applications, 36(2), 3240-3247.
Chen, Y.-W., & Lin, C.-J. (2006). Combining SVMs with various feature selection strategies Feature extraction (pp. 315-324): Springer.
Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7(1), 3. doi:10.1186/1471-2105-7-3
Fushing, H., & McAssey, M. P. (2010). Time, temperature, and data cloud geometry. Physical Review E, 82(6), 061110.
Fushing, H., Wang, H., VanderWaal, K., McCowan, B., & Koehl, P. (2013). Multi-scale clustering by building a robust and self correcting ultrametric topology on data points. PloS one, 8(2), e56259.
Golub, M. S., Hogrefe, C. E., Widaman, K. F., & Capitanio, J. P. (2009). Iron deficiency anemia and affective response in rhesus monkey infants. Developmental psychobiology, 51(1), 47-59.
Lee, O. (2017). Data-driven computation for pattern information. ProQuest, UMI
Dissertations Publishing.

Raymer, M. L., Punch, W. F., Goodman, E. D., Kuhn, L. A., & Jain, A. K. (2000). Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(2), 164-171. doi:10.1109/4235.850656
Saeys, Y., Abeel, T., & Van de Peer, Y. (2008). Robust Feature Selection Using Ensemble Feature Selection Techniques. In W. Daelemans, B. Goethals, & K. Morik (Eds.), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II (pp. 313-325). Berlin, Heidelberg: Springer Berlin Heidelberg.
Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. bioinformatics, 23(19), 2507-2517.
Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004). Application of Breiman’s Random Forest to Modeling Structure-Activity Relationships of Pharmaceutical Molecules. In F. Roli, J. Kittler, & T. Windeatt (Eds.), Multiple Classifier Systems: 5th International Workshop, MCS 2004, Cagliari, Italy, June 9-11, 2004. Proceedings (pp. 334-343). Berlin, Heidelberg: Springer Berlin Heidelberg.
Yu, L., & Liu, H. (2003). Feature selection for high-dimensional data: A fast correlation-based filter solution. Paper presented at the ICML.
描述 碩士
國立政治大學
統計學系
104354013
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0104354013
資料類型 thesis
dc.contributor.advisor 周珮婷zh_TW
dc.contributor.author (Authors) 許立農zh_TW
dc.creator (作者) 許立農zh_TW
dc.date (日期) 2017en_US
dc.date.accessioned 11-Jul-2017 11:25:43 (UTC+8)-
dc.date.available 11-Jul-2017 11:25:43 (UTC+8)-
dc.date.issued (上傳時間) 11-Jul-2017 11:25:43 (UTC+8)-
dc.identifier (Other Identifiers) G0104354013en_US
dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/110782-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 統計學系zh_TW
dc.description (描述) 104354013zh_TW
dc.description.abstract (摘要) 特徵選取為機器學習常見的資料前處理的方法，現今已有許多不同的特徵選取演算法，然而並不存在一個在所有資料上都優於其他方法的演算法，且由於現今的資料種類繁多，所以研發新的方法能夠帶來更多有關資料的資訊並且根據資料的特性採用不同的變數選取演算法是較好的做法。
本研究使用資訊理論entropy的概念依照變數之間資料雲幾何樹的分群結果定義變數之間的相關性，且依此選取資料的特徵，並與同樣使用entropy概念的FCBF方法、Lasso、F-score、隨機森林、基因演算法互相比較，本研究使用階層式分群法與多數決投票法套用在真實的資料上判斷預測率。結果顯示，本研究使用的entropy方法在各個不同的資料集上有較穩定的預測率提升表現，同時資料縮減的維度也相對穩定。		zh_TW
dc.description.abstract (摘要) Feature selection is a common preprocessing technique in machine learning. Although a large pool of feature selection techniques has existed, there is no such a dominant method in all datasets. Because of the complexity of various data formats, establishing a new method can bring more insight into data, and applying proper techniques to analyzing data would be the best choice.
In this study, we used the concept of entropy from information theory to build a similarity matrix between features. Additionally, we constructed a DCG-tree to separate variables into clusters. Each core cluster consists of rather uniform variables, which share similar covariate information. With the core clusters, we reduced the dimension of a high-dimensional dataset. We assessed our method by comparing it with FCBF, Lasso, F-score, random forest and genetic algorithm. The performances of prediction were demonstrated through real-world datasets using hierarchical clustering with voting algorithm as the classifier. The results showed that our entropy method has more stable prediction performances and reduces sufficient dimensions of the datasets simultaneously.		en_US
dc.description.tableofcontents 第一章 緒論 1
第二章 文獻探討 3
第三章 資料介紹與敘述 9
一、生物行為評估專案 9
二、美國威斯康辛州診斷乳癌 11
三、Connectionist Bench(Sonar, Mines vs. Rocks)13
四、SPECTF heart 15
第四章 研究方法與過程 17
第一節 分類預測模型 17
ㄧ、階層式分群法(Hierarchical clustering) 17
二、多數決投票法(Voting) 17
第二節 變數選擇方法 19
ㄧ、不選擇：直接使用所有的變數 19
二、entropy 應用 19
三、FCBF(Fast Correlation-Based Filter) 25
四、Lasso 27
五、F-score 28
六、隨機森林（Random Forest） 29
七、遺傳演算法（Genetic Algorithm） 29
第三節 研究過程 30
ㄧ、entropy 應用 30
二、Lasso 31
三、F-score 32
四、隨機森林 33
五、遺傳演算法 34
第五章 研究結果與結論 35
第一節 研究結果 35
ㄧ、生物行為評估專案資料集 36
二、美國威斯康辛州診斷乳癌資料集 38
三、聲納 40
四、SPECTF 42
第二節 結論 43
參考文獻 45		zh_TW
dc.format.extent 6507025 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0104354013en_US
dc.subject (關鍵詞) 機器學習zh_TW
dc.subject (關鍵詞) 特徵選取zh_TW
dc.subject (關鍵詞) 維度縮減zh_TW
dc.subject (關鍵詞) entropyzh_TW
dc.subject (關鍵詞) Machine learningen_US
dc.subject (關鍵詞) Feature selectionen_US
dc.subject (關鍵詞) Dimension reductionen_US
dc.subject (關鍵詞) Entropyen_US
dc.title (題名) 基於資訊理論熵之特徵選取zh_TW
dc.title (題名) Entropy based feature selectionen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) Akay, M. F. (2009). Support vector machines combined with feature selection for breast cancer diagnosis. Expert systems with applications, 36(2), 3240-3247.
Chen, Y.-W., & Lin, C.-J. (2006). Combining SVMs with various feature selection strategies Feature extraction (pp. 315-324): Springer.
Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7(1), 3. doi:10.1186/1471-2105-7-3
Fushing, H., & McAssey, M. P. (2010). Time, temperature, and data cloud geometry. Physical Review E, 82(6), 061110.
Fushing, H., Wang, H., VanderWaal, K., McCowan, B., & Koehl, P. (2013). Multi-scale clustering by building a robust and self correcting ultrametric topology on data points. PloS one, 8(2), e56259.
Golub, M. S., Hogrefe, C. E., Widaman, K. F., & Capitanio, J. P. (2009). Iron deficiency anemia and affective response in rhesus monkey infants. Developmental psychobiology, 51(1), 47-59.
Lee, O. (2017). Data-driven computation for pattern information. ProQuest, UMI
Dissertations Publishing.

Raymer, M. L., Punch, W. F., Goodman, E. D., Kuhn, L. A., & Jain, A. K. (2000). Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(2), 164-171. doi:10.1109/4235.850656
Saeys, Y., Abeel, T., & Van de Peer, Y. (2008). Robust Feature Selection Using Ensemble Feature Selection Techniques. In W. Daelemans, B. Goethals, & K. Morik (Eds.), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II (pp. 313-325). Berlin, Heidelberg: Springer Berlin Heidelberg.
Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. bioinformatics, 23(19), 2507-2517.
Svetnik, V., Liaw, A., Tong, C., & Wang, T. (2004). Application of Breiman’s Random Forest to Modeling Structure-Activity Relationships of Pharmaceutical Molecules. In F. Roli, J. Kittler, & T. Windeatt (Eds.), Multiple Classifier Systems: 5th International Workshop, MCS 2004, Cagliari, Italy, June 9-11, 2004. Proceedings (pp. 334-343). Berlin, Heidelberg: Springer Berlin Heidelberg.
Yu, L., & Liu, H. (2003). Feature selection for high-dimensional data: A fast correlation-based filter solution. Paper presented at the ICML.		zh_TW