Publications-Theses
Article View/Open
Publication Export
Google ScholarTM
NCCU Library
Citation Infomation
Related Publications in TAIR
| Title | 監督式異常偵測與孿生神經網路 Supervised Anomaly Detection using Siamese Neural Network |
| Creator | 謝博丞 Hsieh, Bo-Cheng |
| Contributor | 周珮婷<br>陳怡如 Chou, Pei-Ting<br>Chen, Yi-Ju 謝博丞 Hsieh, Bo-Cheng |
| Key Words | 異常偵測 孿生神經網路 機器學習 監督式學習 結構化資料 Anomaly Detection Siamese Neural Network Machine Learning Supervised Learning Structured Data |
| Date | 2023 |
| Date Issued | 6-Jul-2023 17:05:40 (UTC+8) |
| Summary | 孿生神經網路是一種基於度量的小樣本學習 (Few-Shot Learning)， 其特殊的訓練方式與神經網路架構，於少量資料情境可有效進行特徵 提取，提升分類器的準確度。孿生神經網路在電腦視覺、自然語言處 理領域已被廣為使用作為非結構化資料的特徵提取方法，但鮮少有研 究將其應用於結構化資料，並與其他演算法進行比較。本研究利用孿 生神經網路與其他四種分類器以及重複採樣技術 SMOTE 組合，共 9 種演算法組合，對 5 筆資料進行監督式異常偵測，並測試異常比例對各種演算法的影響。研究結果發現孿生神經路於結構化資料表現優秀，且相對於其他演算法更不受資料異常比例影響。 The Siamese neural network is a metric-based few-shot learning method that can effectively extract features from a small amount of data and improve classifier ac- curacy through its special training method and neural network architecture. While it has been widely used as a feature extraction method for unstructured data in com- puter vision and natural language processing, few studies have compared it with other algorithms in structured data applications. In this study, we combined nine algo- rithm combinations using a siamese neural network, four different classifiers, and SMOTE for supervised anomaly detection with 5 datasets. We also tested the effect of anomaly ratios on various algorithms. The results showed that the Siamese neural network performed well in structured data and was less affected by data anomalies than other algorithms. |
| 參考文獻 | Belton, N., Lawlor, A., and Curran, K. M. (2021). Semi-supervised siamese network for identifying bad data in medical imaging datasets. arXiv preprint arXiv:2108.07130. Breiman, L. (2001). Random forests. Machine learning, 45:5–32. Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. (2000). Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international confer- ence on Management of data, pages 93–104. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R. (1993). Signature verifi- cation using a” siamese” time delay neural network. Advances in neural information processing systems, 6. Brümmer, N., Cumani, S., Glembek, O., Karafiát, M., Matějka, P., Pešán, J., Plchot, O., Soufifar, M., Villiers, E. d., and Černockỳ, J. H. (2012). Description and analysis of the brno276 system for lre2011. In Odyssey 2012-the speaker and language recognition workshop. Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3):1–58. Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: syn- thetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning, 20:273– 297. Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21–27. Durkota, K., Linda, M., Ludvik, M., and Tozicka, J. (2020). Neuron-net: Siamese network for anomaly detection. Technical report, Tech. Report in DCASE2020 Challenge Task 2. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2020). Generative adversarial networks. Communi- cations of the ACM, 63(11):139–144. Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE. Han, S., Hu, X., Huang, H., Jiang, M., and Zhao, Y. (2022). Adbench: Anomaly detection benchmark. arXiv preprint arXiv:2206.09426. Hilal, W., Gadsden, S. A., and Yawney, J. (2022). Financial fraud:: A review of anomaly detection techniques and recent advances. Khatri, S., Arora, A., and Agrawal, A. P. (2020). Supervised machine learning algorithms for credit card fraud detection: a comparison. In 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pages 680–683. IEEE. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90. LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436–444. LeNail, A. (2019). Nn-svg: Publication-ready neural network architecture schematics. Journal of Open Source Software, 4(33):747. Liu, F. T., Ting, K. M., and Zhou, Z.-H. (2008). Isolation forest. In 2008 eighth ieee international conference on data mining, pages 413–422. IEEE. Moustafa, N. and Slay, J. (2015). Unsw-nb15: a comprehensive data set for network intru- sion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE. Olszewski, R. T. (2001). Generalized feature extraction for structural pattern recognition in time-series data. Carnegie Mellon University. Pang, G., Shen, C., and van den Hengel, A. (2019). Deep anomaly detection with devia- tion networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 353–362. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. Scholkopf, B., Williamson, R., Smola, A., Shawe-Taylor, J., Platt, J., et al. (2000). Sup- port vector method for novelty detection. Advances in neural information processing systems, 12(3):582–588. Scott, A. J. and Knott, M. (1974). A cluster analysis method for grouping means in the analysis of variance. Biometrics, pages 507–512. Song, S. and Baek, J. G. (2020). New anomaly detection in semiconductor manufacturing process using oversampling method. In 12th International Conference on Agents and Artificial Intelligence, ICAART 2020, pages 926–932. SciTePress. Takimoto, H., Seki, J., F. Situju, S., and Kanagawa, A. (2022). Anomaly detection using siamese network with attention mechanism for few-shot learning. Applied Artificial Intelligence, 36(1):2094885. Tantithamthavorn, C., Hassan, A. E., and Matsumoto, K. (2018). The impact of class re- balancing techniques on the performance and interpretation of defect prediction models. IEEE Transactions on Software Engineering, 46(11):1200–1219. |
| Description | 碩士 國立政治大學 統計學系 110354023 |
| 資料來源 | http://thesis.lib.nccu.edu.tw/record/#G0110354023 |
| Type | thesis |
| dc.contributor.advisor | 周珮婷<br>陳怡如 | zh_TW |
| dc.contributor.advisor | Chou, Pei-Ting<br>Chen, Yi-Ju | en_US |
| dc.contributor.author (Authors) | 謝博丞 | zh_TW |
| dc.contributor.author (Authors) | Hsieh, Bo-Cheng | en_US |
| dc.creator (作者) | 謝博丞 | zh_TW |
| dc.creator (作者) | Hsieh, Bo-Cheng | en_US |
| dc.date (日期) | 2023 | en_US |
| dc.date.accessioned | 6-Jul-2023 17:05:40 (UTC+8) | - |
| dc.date.available | 6-Jul-2023 17:05:40 (UTC+8) | - |
| dc.date.issued (上傳時間) | 6-Jul-2023 17:05:40 (UTC+8) | - |
| dc.identifier (Other Identifiers) | G0110354023 | en_US |
| dc.identifier.uri (URI) | http://nccur.lib.nccu.edu.tw/handle/140.119/145946 | - |
| dc.description (描述) | 碩士 | zh_TW |
| dc.description (描述) | 國立政治大學 | zh_TW |
| dc.description (描述) | 統計學系 | zh_TW |
| dc.description (描述) | 110354023 | zh_TW |
| dc.description.abstract (摘要) | 孿生神經網路是一種基於度量的小樣本學習 (Few-Shot Learning)， 其特殊的訓練方式與神經網路架構，於少量資料情境可有效進行特徵 提取，提升分類器的準確度。孿生神經網路在電腦視覺、自然語言處 理領域已被廣為使用作為非結構化資料的特徵提取方法，但鮮少有研 究將其應用於結構化資料，並與其他演算法進行比較。本研究利用孿 生神經網路與其他四種分類器以及重複採樣技術 SMOTE 組合，共 9 種演算法組合，對 5 筆資料進行監督式異常偵測，並測試異常比例對各種演算法的影響。研究結果發現孿生神經路於結構化資料表現優秀，且相對於其他演算法更不受資料異常比例影響。 | zh_TW |
| dc.description.abstract (摘要) | The Siamese neural network is a metric-based few-shot learning method that can effectively extract features from a small amount of data and improve classifier ac- curacy through its special training method and neural network architecture. While it has been widely used as a feature extraction method for unstructured data in com- puter vision and natural language processing, few studies have compared it with other algorithms in structured data applications. In this study, we combined nine algo- rithm combinations using a siamese neural network, four different classifiers, and SMOTE for supervised anomaly detection with 5 datasets. We also tested the effect of anomaly ratios on various algorithms. The results showed that the Siamese neural network performed well in structured data and was less affected by data anomalies than other algorithms. | en_US |
| dc.description.tableofcontents | 摘要 i Abstract ii 目次 iii 圖目錄 v 表目錄 vi 第一章 緒論 1 1.1 異常偵測 1 1.2 孿生神經網路 2 第二章 文獻回顧 4 2.1 異常偵測 4 2.2 孿生神經網路 5 2.3 總結 6 第三章 研究方法 7 3.1 神經網路架構 7 3.1.1 全連接神經網路 7 3.1.2 孿生神經網路 8 3.1.3 對比損失 10 3.1.4 資料準備與模型訓練 11 3.2 其他演算法 12 3.2.1 SMOTE 12 3.2.2 支持向量機 13 3.2.3 隨機森林 13 3.2.4 單類別支持向量機 14 3.2.5 孤立森林 14 3.2.6 總結 15 3.3 評估指標 15 3.3.1 指標介紹 15 3.3.2 Scott Knott Analysis 17 3.4 資料集 18 3.4.1 異常比例調整：矽晶圓資料集 19 第四章 實證結果 21 4.1 全部資料之預測結果 21 4.2 異常比例調整之各模型表現變化：矽晶圓資料集 23 第五章 結論與建議 32 5.1 研究結論 32 5.2 未來方向與建議 33 參考文獻 34 | zh_TW |
| dc.format.extent | 5206896 bytes | - |
| dc.format.mimetype | application/pdf | - |
| dc.source.uri (資料來源) | http://thesis.lib.nccu.edu.tw/record/#G0110354023 | en_US |
| dc.subject (關鍵詞) | 異常偵測 | zh_TW |
| dc.subject (關鍵詞) | 孿生神經網路 | zh_TW |
| dc.subject (關鍵詞) | 機器學習 | zh_TW |
| dc.subject (關鍵詞) | 監督式學習 | zh_TW |
| dc.subject (關鍵詞) | 結構化資料 | zh_TW |
| dc.subject (關鍵詞) | Anomaly Detection | en_US |
| dc.subject (關鍵詞) | Siamese Neural Network | en_US |
| dc.subject (關鍵詞) | Machine Learning | en_US |
| dc.subject (關鍵詞) | Supervised Learning | en_US |
| dc.subject (關鍵詞) | Structured Data | en_US |
| dc.title (題名) | 監督式異常偵測與孿生神經網路 | zh_TW |
| dc.title (題名) | Supervised Anomaly Detection using Siamese Neural Network | en_US |
| dc.type (資料類型) | thesis | en_US |
| dc.relation.reference (參考文獻) | Belton, N., Lawlor, A., and Curran, K. M. (2021). Semi-supervised siamese network for identifying bad data in medical imaging datasets. arXiv preprint arXiv:2108.07130. Breiman, L. (2001). Random forests. Machine learning, 45:5–32. Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. (2000). Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international confer- ence on Management of data, pages 93–104. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R. (1993). Signature verifi- cation using a” siamese” time delay neural network. Advances in neural information processing systems, 6. Brümmer, N., Cumani, S., Glembek, O., Karafiát, M., Matějka, P., Pešán, J., Plchot, O., Soufifar, M., Villiers, E. d., and Černockỳ, J. H. (2012). Description and analysis of the brno276 system for lre2011. In Odyssey 2012-the speaker and language recognition workshop. Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3):1–58. Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: syn- thetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning, 20:273– 297. Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21–27. Durkota, K., Linda, M., Ludvik, M., and Tozicka, J. (2020). Neuron-net: Siamese network for anomaly detection. Technical report, Tech. Report in DCASE2020 Challenge Task 2. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2020). Generative adversarial networks. Communi- cations of the ACM, 63(11):139–144. Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE. Han, S., Hu, X., Huang, H., Jiang, M., and Zhao, Y. (2022). Adbench: Anomaly detection benchmark. arXiv preprint arXiv:2206.09426. Hilal, W., Gadsden, S. A., and Yawney, J. (2022). Financial fraud:: A review of anomaly detection techniques and recent advances. Khatri, S., Arora, A., and Agrawal, A. P. (2020). Supervised machine learning algorithms for credit card fraud detection: a comparison. In 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pages 680–683. IEEE. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90. LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436–444. LeNail, A. (2019). Nn-svg: Publication-ready neural network architecture schematics. Journal of Open Source Software, 4(33):747. Liu, F. T., Ting, K. M., and Zhou, Z.-H. (2008). Isolation forest. In 2008 eighth ieee international conference on data mining, pages 413–422. IEEE. Moustafa, N. and Slay, J. (2015). Unsw-nb15: a comprehensive data set for network intru- sion detection systems (unsw-nb15 network data set). In 2015 military communications and information systems conference (MilCIS), pages 1–6. IEEE. Olszewski, R. T. (2001). Generalized feature extraction for structural pattern recognition in time-series data. Carnegie Mellon University. Pang, G., Shen, C., and van den Hengel, A. (2019). Deep anomaly detection with devia- tion networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 353–362. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. Scholkopf, B., Williamson, R., Smola, A., Shawe-Taylor, J., Platt, J., et al. (2000). Sup- port vector method for novelty detection. Advances in neural information processing systems, 12(3):582–588. Scott, A. J. and Knott, M. (1974). A cluster analysis method for grouping means in the analysis of variance. Biometrics, pages 507–512. Song, S. and Baek, J. G. (2020). New anomaly detection in semiconductor manufacturing process using oversampling method. In 12th International Conference on Agents and Artificial Intelligence, ICAART 2020, pages 926–932. SciTePress. Takimoto, H., Seki, J., F. Situju, S., and Kanagawa, A. (2022). Anomaly detection using siamese network with attention mechanism for few-shot learning. Applied Artificial Intelligence, 36(1):2094885. Tantithamthavorn, C., Hassan, A. E., and Matsumoto, K. (2018). The impact of class re- balancing techniques on the performance and interpretation of defect prediction models. IEEE Transactions on Software Engineering, 46(11):1200–1219. | zh_TW |