基於支持向量機計算的相互熵之特徵選取

Abstract

特徵選取為機器學習領域中一重要部分，適當的選取特徵(變數)，除了減少機器運算時間、人力與金錢外，也可以避免模型過度配適或是欠擬和的情況發生。雖然經過多年發展已有很多特徵選取的方法，但同一種模型，不一定適用所有資料情況，因此提出新方法希望在特徵選取上會有更多選擇。\n 本文提出一新方法衡量變數之間的關係，使用資訊理論中熵的概念，結合分類器支持向量機，獲取變數間關係，並將變數分群，給予每群適當的相對應分數，以此篩選變數。本文採用半監督式學習，計算敏感度、特異度與準確度之平均及所使用變數數量，並以高斯混合模型搭配EM演算法利用KS檢定之檢定統計量定義資料重要變數，評估方法能否選取重要變數，本文一共使用2筆模擬資料與5筆真實資料，並將結果與各大方法比較，結果顯示在各資料集中皆有穩定表現，即使在變數少的情況下也能有不錯表現。

Feature selection technique plays a significant role in machine learning. Selecting features (variables) adequately can not only reduce the expenditure, operating time in machine and the cost of labor but also prevent under fitting or overfitting. Although lots of feature selection methods have been developed for decades, it is impossible to apply a unique method to all types of data sets. In this study, we propose a new method to calculate the correlation between variables based on the Shannon entropy from information theory and SVM classifier. Variables are grouped into several clusters and selected by the new correlation measurement. Besides, we define the importance of variable by the test statistic of KS test using Gaussian mixed model and E-M algorithm for the propose of result assessment. The performance of proposed method on two simulated data and five real data are demonstrated and compared with other feature selection methods. The predicted results are stable through the proposed method with a reduced dataset.