學術產出-Theses
Article View/Open
Publication Export
-
題名 以成對共識分數評估多個分類器中含雜亂標籤的分類結果
Evaluate the Classification Result with Cacophonous Labels of Multiple Classifiers by Pairwise Consensus Score作者 陳璿心
Chen, Hsuan-Hsin貢獻者 蕭舜文
Hsiao, Shun-Wen
陳璿心
Chen, Hsuan-Hsin關鍵詞 基因演算法
惡意程式標籤
惡意程式家族
Malware labeling
AV labels
Malware family日期 2021 上傳時間 2-Sep-2021 15:53:16 (UTC+8) 摘要 在資訊安全的領域中，有許多惡意軟體分類器，這些分類器的目的是給予不同惡意軟體其家族名稱。然而這些家族名稱不像是在做圖形辨識，例如判斷手寫數字的標籤是基於事實，而這些惡意程式家族是基於不同觀點給予不同的標籤。我們想知道哪一種觀點是被大眾所接受，所以發展一個不同於多數決的投票方法，而是採用一次比較一對分類器中的一對惡意軟體，並從每一對分類器中加總計算不同對惡意軟體之間的共識分數，最後這些分數就會成為我們判斷獲得最多大眾觀點的依據。此外建立在成對的共識分數機制上，我們另外採用了基因演算法，設法交換出具有最高分數的分類結果，成為在分類惡意軟體的結果可依循的答案。除了設計演算法來尋找受到較多支持的惡意軟體偵測廠商外，本研究也嘗試使用三種不同來源的惡意程式資料，並加入經基因演算法取得的最佳解來計算每個來源個別的共識分數，並證明取得的最佳解經過交換後分數都會比為交換前來的更高分。
In the field of cybersecurity, there are lots of classifiers (AV vendors) and each classifier will give malware samples classified results, namely naming labels to include malware families. Unfortunately, each label does not have a fixed answer based on fact like handwritten number recognition but based on each classifiers’ viewpoints, thus, we want to know which classifier receives the most support from others. Instead of using majority voting, we develop a scoring system Pairwise Consensus Score-PCS with the idea of pairwise comparison. In addition, based on the scoring system, we propose a heuristic genetic algorithm-HAGL to obtain a group of labels that unify all classifiers and get the optimized consensus score. In the research, we found that our method had a better performance than other traditional data mining methods and the score reach a higher level after value exchange.參考文獻 [1] SonicWall Inc., “2020 SonicWall Cyber Threat Report,”sonicwall.com,2020. [Online].Available:https://www.sonicwall.com/medialibrary/en/infographic/infographic-2020-sonicwall-cyber-threat-report.pdf. [Accessed May. 29, 2021].[2] VirusTotal. Accessed: May 29, 2021. [Online]. Available: https://www.virustotal.com/[3] CARO. Caro naming convention. [Online]. Available: http://www.caro.org/articles/naming.html[4] F. Maggi, A. Bellini,G. Salvaneschi, S. Zanero, “Finding Non-trivial Malware Naming Inconsistencies”, in International Conference on Information Systems Security(Lecture Notes in Computer Science), vol 7093, pp. 144-159, Springer, Berlin, Heidelberg, 2016,doi:10.1007/978-3-642-25560-110[5] M. Bailey, J. Oberheide, J. Andersen, Z. M. Mao, F. Jahanian, and J. Nazario. “Automated Classification and Analysis of Internet Malware,” in International Symposium on Recent Advances in Intrusion Detection(RAID), 2007,doi: 10.1007/978-3-540-74320-010.[6] M. Sebasti ́an, R. Rivera, P. Kotzias, and J. Caballero, “Avclass: A tool for massive malware labeling,” in International Symposium on Research in Attacks, Intrusions, and Defenses(Lecture Notes in Computer Science), vol 9854, pp 144-159, Cham, Switzerland: Springer,2016,doi:10.1007/978-3-642-25560-110[7] =M. Hurier et al., “Euphony: Harmonious unification of cacophonous anti-virus vendor la-bels for android malware,” in2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), May 2017, pp. 425-435, doi:10.1109/MSR.2017.57.[8] L. Alexandre, A. Campilho and M. Kamel, “On combining classifiers using sum and product rules,” in Pattern Recognition Letters,2001, vol. 22, no. 12, pp. 1283-1289, doi:10.1016/s0167-8655(01)00073-3[9] Y. Sun, M. S. Kamel, and A. K. Wong, “Empirical Study on Weighted Voting Multiple Classifiers,” in Pattern Recognition and Data Mining,2005, pp. 335–34[10] D. Hand and K.Yu, “Idiot’s Bayes: Not So Stupid after All?,” in International Statistical Review / Revue Internationale De Statistique,2001, vol. 69, no. 3, pp. 385-398,doi:10.2307/1403452[11] C.C. Chang and C.J. Lin, “LIBSVM: a library for support vector machines,” in ACM transactions on intelligent systems and technology (TIST),2011, vol. 2, no. 3, pp.1-27,doi:10.1145/1961189.1961199[12] Y. Freund, R. Schapire, and N. Abe, “A short introduction to boosting,” in Journal-Japanese Society For Artificial Intelligence, 1999, vol. 14, no. 771-780, pp.1612[13] Sung, A. H., Xu, J., Chavez, P., and Mukkamala, S., “Static analyzer of vicious executables(save),” in20th Annual Computer Security Applications Conference. IEEE, 2004, pp. 326-334[14] U. Bayer, A. Moser, C. Kruegel, and E. Kirda, “Dynamic analysis of malicious code,” Journal in Computer Virology, 2006, vol. 2, pp. 66-77[15] L. Alexandre, A. Campilho and M. Kamel, “On combining classifiers using sum and product rules,” Pattern Recognition Letters, 2001, vol. 22, no. 12, pp. 1283-1289, doi: 10.1016/s0167-8655(01)00073-3[16] S. S. Hansen, T. M. T. Larsen, M. Stevanovic and J. M. Pedersen, “An approach for detection and family classification of malware based on behavioral analysis,”2016 InternationalConference on Computing, Networking and Communications (ICNC), 2016, pp. 1-5, doi:10.1109/ICCNC.2016.7440587.[17] K.-F. Man, K.-S. Tang and S. Kwong, “Genetic algorithms: concepts and applications [inengineering design],” IEEE transactions on Industrial Electronics, 1996, vol.43, pp.519-534[18] J. H. Holland, “Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence,” MIT press, 1922[19] Genta Aoki and Yasubumi Sakakibara, “Convolutional neural networks for classification of alignments of non-coding RNA sequences,” Bioinformatics,2018, vol. 34,pp. 237-244[20] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,1997, vol.9, no. 8, pp.1735-178[21] R. Pascanu, J. W. Stokes, H. Sanossian, M. Marinescu and A. Thomas, “Malware classification with recurrent networks,”2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp.1916-1920[22] G. Sun and Q. Qian, “Deep Learning and Visualization for Identifying Malware Families,” in IEEE Transactions on Dependable and Secure Computing,2021, vol. 18, no. 1, pp. 283-295,doi: 10.1109/TDSC.2018.2884928.[23] Malware Knowledge Base. Accessed: July 7, 2021. [Online]. Available: https://owl.nchc.org.tw/[24] W. J. Chiu, “Automated Malware Family Signature Generation based on Runtime API Call Sequence,” Unpublished master’s thesis, 2018, National Taiwan University, Taipei, Taiwan, doi: 10.6342/NTU201802357 描述 碩士
國立政治大學
資訊管理學系
108356017資料來源 http://thesis.lib.nccu.edu.tw/record/#G0108356017 資料類型 thesis dc.contributor.advisor 蕭舜文 zh_TW dc.contributor.advisor Hsiao, Shun-Wen en_US dc.contributor.author (Authors) 陳璿心 zh_TW dc.contributor.author (Authors) Chen, Hsuan-Hsin en_US dc.creator (作者) 陳璿心 zh_TW dc.creator (作者) Chen, Hsuan-Hsin en_US dc.date (日期) 2021 en_US dc.date.accessioned 2-Sep-2021 15:53:16 (UTC+8) - dc.date.available 2-Sep-2021 15:53:16 (UTC+8) - dc.date.issued (上傳時間) 2-Sep-2021 15:53:16 (UTC+8) - dc.identifier (Other Identifiers) G0108356017 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/136843 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊管理學系 zh_TW dc.description (描述) 108356017 zh_TW dc.description.abstract (摘要) 在資訊安全的領域中，有許多惡意軟體分類器，這些分類器的目的是給予不同惡意軟體其家族名稱。然而這些家族名稱不像是在做圖形辨識，例如判斷手寫數字的標籤是基於事實，而這些惡意程式家族是基於不同觀點給予不同的標籤。我們想知道哪一種觀點是被大眾所接受，所以發展一個不同於多數決的投票方法，而是採用一次比較一對分類器中的一對惡意軟體，並從每一對分類器中加總計算不同對惡意軟體之間的共識分數，最後這些分數就會成為我們判斷獲得最多大眾觀點的依據。此外建立在成對的共識分數機制上，我們另外採用了基因演算法，設法交換出具有最高分數的分類結果，成為在分類惡意軟體的結果可依循的答案。除了設計演算法來尋找受到較多支持的惡意軟體偵測廠商外，本研究也嘗試使用三種不同來源的惡意程式資料，並加入經基因演算法取得的最佳解來計算每個來源個別的共識分數，並證明取得的最佳解經過交換後分數都會比為交換前來的更高分。 zh_TW dc.description.abstract (摘要) In the field of cybersecurity, there are lots of classifiers (AV vendors) and each classifier will give malware samples classified results, namely naming labels to include malware families. Unfortunately, each label does not have a fixed answer based on fact like handwritten number recognition but based on each classifiers’ viewpoints, thus, we want to know which classifier receives the most support from others. Instead of using majority voting, we develop a scoring system Pairwise Consensus Score-PCS with the idea of pairwise comparison. In addition, based on the scoring system, we propose a heuristic genetic algorithm-HAGL to obtain a group of labels that unify all classifiers and get the optimized consensus score. In the research, we found that our method had a better performance than other traditional data mining methods and the score reach a higher level after value exchange. en_US dc.description.tableofcontents 1 Introduction 12 Related Work 62.1 Existing efforts on malware labeling 62.2 Ensemble methods 62.3 Malware Analysis 72.4 Malware Family Classification and Machine Learning 82.5 NN-based Algorithms for Classifying malware 92.6 Genetic Algorithm for Optimization 103 Design 123.1 Pairwise Consensus Score System 123.2 Heuristic Assigned Genetic Labeling Algorithm 184 Evaluation 224.1 Results of PCS with 1000 malware samples 224.2 Different Types of Dataset for Experiment 234.3 Different Strategies of HAGL 254.4 Additional Data Sources 305 Discussion 335.1 Total PCS score 335.2 Meeting Design Requirement 335.3 Limitations 356 Conclusion 35References 36 zh_TW dc.format.extent 1981475 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0108356017 en_US dc.subject (關鍵詞) 基因演算法 zh_TW dc.subject (關鍵詞) 惡意程式標籤 zh_TW dc.subject (關鍵詞) 惡意程式家族 zh_TW dc.subject (關鍵詞) Malware labeling en_US dc.subject (關鍵詞) AV labels en_US dc.subject (關鍵詞) Malware family en_US dc.title (題名) 以成對共識分數評估多個分類器中含雜亂標籤的分類結果 zh_TW dc.title (題名) Evaluate the Classification Result with Cacophonous Labels of Multiple Classifiers by Pairwise Consensus Score en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] SonicWall Inc., “2020 SonicWall Cyber Threat Report,”sonicwall.com,2020. [Online].Available:https://www.sonicwall.com/medialibrary/en/infographic/infographic-2020-sonicwall-cyber-threat-report.pdf. [Accessed May. 29, 2021].[2] VirusTotal. Accessed: May 29, 2021. [Online]. Available: https://www.virustotal.com/[3] CARO. Caro naming convention. [Online]. Available: http://www.caro.org/articles/naming.html[4] F. Maggi, A. Bellini,G. Salvaneschi, S. Zanero, “Finding Non-trivial Malware Naming Inconsistencies”, in International Conference on Information Systems Security(Lecture Notes in Computer Science), vol 7093, pp. 144-159, Springer, Berlin, Heidelberg, 2016,doi:10.1007/978-3-642-25560-110[5] M. Bailey, J. Oberheide, J. Andersen, Z. M. Mao, F. Jahanian, and J. Nazario. “Automated Classification and Analysis of Internet Malware,” in International Symposium on Recent Advances in Intrusion Detection(RAID), 2007,doi: 10.1007/978-3-540-74320-010.[6] M. Sebasti ́an, R. Rivera, P. Kotzias, and J. Caballero, “Avclass: A tool for massive malware labeling,” in International Symposium on Research in Attacks, Intrusions, and Defenses(Lecture Notes in Computer Science), vol 9854, pp 144-159, Cham, Switzerland: Springer,2016,doi:10.1007/978-3-642-25560-110[7] =M. Hurier et al., “Euphony: Harmonious unification of cacophonous anti-virus vendor la-bels for android malware,” in2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), May 2017, pp. 425-435, doi:10.1109/MSR.2017.57.[8] L. Alexandre, A. Campilho and M. Kamel, “On combining classifiers using sum and product rules,” in Pattern Recognition Letters,2001, vol. 22, no. 12, pp. 1283-1289, doi:10.1016/s0167-8655(01)00073-3[9] Y. Sun, M. S. Kamel, and A. K. Wong, “Empirical Study on Weighted Voting Multiple Classifiers,” in Pattern Recognition and Data Mining,2005, pp. 335–34[10] D. Hand and K.Yu, “Idiot’s Bayes: Not So Stupid after All?,” in International Statistical Review / Revue Internationale De Statistique,2001, vol. 69, no. 3, pp. 385-398,doi:10.2307/1403452[11] C.C. Chang and C.J. Lin, “LIBSVM: a library for support vector machines,” in ACM transactions on intelligent systems and technology (TIST),2011, vol. 2, no. 3, pp.1-27,doi:10.1145/1961189.1961199[12] Y. Freund, R. Schapire, and N. Abe, “A short introduction to boosting,” in Journal-Japanese Society For Artificial Intelligence, 1999, vol. 14, no. 771-780, pp.1612[13] Sung, A. H., Xu, J., Chavez, P., and Mukkamala, S., “Static analyzer of vicious executables(save),” in20th Annual Computer Security Applications Conference. IEEE, 2004, pp. 326-334[14] U. Bayer, A. Moser, C. Kruegel, and E. Kirda, “Dynamic analysis of malicious code,” Journal in Computer Virology, 2006, vol. 2, pp. 66-77[15] L. Alexandre, A. Campilho and M. Kamel, “On combining classifiers using sum and product rules,” Pattern Recognition Letters, 2001, vol. 22, no. 12, pp. 1283-1289, doi: 10.1016/s0167-8655(01)00073-3[16] S. S. Hansen, T. M. T. Larsen, M. Stevanovic and J. M. Pedersen, “An approach for detection and family classification of malware based on behavioral analysis,”2016 InternationalConference on Computing, Networking and Communications (ICNC), 2016, pp. 1-5, doi:10.1109/ICCNC.2016.7440587.[17] K.-F. Man, K.-S. Tang and S. Kwong, “Genetic algorithms: concepts and applications [inengineering design],” IEEE transactions on Industrial Electronics, 1996, vol.43, pp.519-534[18] J. H. Holland, “Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence,” MIT press, 1922[19] Genta Aoki and Yasubumi Sakakibara, “Convolutional neural networks for classification of alignments of non-coding RNA sequences,” Bioinformatics,2018, vol. 34,pp. 237-244[20] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,1997, vol.9, no. 8, pp.1735-178[21] R. Pascanu, J. W. Stokes, H. Sanossian, M. Marinescu and A. Thomas, “Malware classification with recurrent networks,”2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp.1916-1920[22] G. Sun and Q. Qian, “Deep Learning and Visualization for Identifying Malware Families,” in IEEE Transactions on Dependable and Secure Computing,2021, vol. 18, no. 1, pp. 283-295,doi: 10.1109/TDSC.2018.2884928.[23] Malware Knowledge Base. Accessed: July 7, 2021. [Online]. Available: https://owl.nchc.org.tw/[24] W. J. Chiu, “Automated Malware Family Signature Generation based on Runtime API Call Sequence,” Unpublished master’s thesis, 2018, National Taiwan University, Taipei, Taiwan, doi: 10.6342/NTU201802357 zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202101327 en_US
