學術產出-Theses
Article View/Open
Publication Export
-
題名 基於偏態排序最佳化探討圖形學習表示法之分佈於推薦系統
Exploring Distribution of Graph Embedding Based on Skewness Ranking Optimization for Recommender Systems作者 莊喻能
Chuang, Yu-Neng貢獻者 蔡銘峰
Tsai, Ming-Feng
莊喻能
Chuang, Yu-Neng關鍵詞 推薦系統
協同過濾法
圖形學習表示法
矩陣分解
偏態排序法
Recommender systems
Collaborative filitering
Graph embedding
Matrix factorization
Skewness optimization ranking日期 2020 上傳時間 1-Jul-2020 13:50:07 (UTC+8) 摘要 近年來大數據以及機器學習技術的蓬勃發展，推薦系統被廣泛應用於各種資訊系( Information Systems )上。如何有效地利用這些巨量的資料增進推薦系統效能，成為具有挑戰的工作。圖形學習表示法( Graph Embedding )便是一種特徵提取 ( Feature Extraction ) 的技術，此方法目的在於如何有效的將不同節點以及節點間的關係投射到低維度向量空間並賦予特徵向量。因此，如何有效率且精準的描述這些向量空間的概念，也被加入到圖形學習表示法的領域。本論文基於非對稱常態分佈( Skew Normal Distribution)之特性，提出以機率分佈重新檢視表示法向量空間，並針對使用者與喜好物品在非對稱常態分佈上會趨向正向偏態( Positive Skewness )的特性，將偏態之概念加入目標函式中進行優化。特別的是，本論文所提出之偏態項優化式為一通用優化項，能適用於過去各種 State of The Art 推薦演算法上，進而重塑各種推薦演算法所構建之向量空間。從理論面來論述，我們證明了如何在優化各種推薦演算法上之餘，同時優化基於非對稱常態分佈之 Shape 參數，此參數與分佈之偏態值為正相關。此外，針對所提出之演算法能同時最大化接收者操作特徵曲線( Receiver Operating Characteristic Curve ( ROC Curve ) )之論述，我們也提出一數學證述來解釋與分析。在數據實驗上，本文以將此偏態優化項主要實驗於矩陣分解類之推薦算法上，且為了展示方法的一致性，我們也將此偏態優化項實驗在基於圖形學習表示法的推薦演算法上，來做驗證本方法的可行性與正確性。而為了驗證此方法，本文實驗於五種不同的真實世 界巨量資料上，並且針對兩種常見的推薦任務: Top-N 推薦任務以及 Query-based 推薦任務上皆有所比較與操作。最後，在實驗結果的部分，結果呈現出我們所提之演算法與過去各種 State of The Art 之推薦演算法中實際比較後皆取得更優的表現。
In recent years, machine learning technology has drastically improved in adapting big data among various fields, including commercial streaming online service and recommendation systems. Especially in recommendation systems case, the user-based recommendation systems or personalized recommendation is one of the most challenging tasks. In this paper, hence, we propose a novel optimization criterion that leverages features of the skew normal distribution to better model the problem of personalized recommendation. Specifically, the developed criterion borrows the concept and the flexibility of the skew normal distribution and also based on three hyper-parameters to not only provide the degree of freedom in optimization and also highly attached to the optimization criterion. Moreover, we both provide the relation of optimization of the proposed criterion and the shape parameter in the skew normal distribution from theoretical point of view and provide the analogies and provide the theoretical proof on asymptotic analysis of the area under the ROC curve to our proposed method. Experimental results conducted on five large-scale real-world datasets reveal that our proposed optimization criterion significantly achieve the best performance of the state of the art and yields consistently on all tested datasets.參考文獻 [1] C.-M. Chen, M.-F. Tsai, Y.-C. Lin, and Y.-H. Yang. Query-based music recommendations via preference embedding. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 79–82. Association for Computing Machinery.[2] C.-M. Chen, C.-J. Wang, M.-F. Tsai, and Y.-H. Yang. Collaborative similarity em- bedding for recommender systems. In Proceedings of the 28th International Conference on World Wide Web, WWW ’19, page 2637–2643. Association for Computing Machinery.[3] A. Grover and J. Leskovec. Node2vec: Scalable feature learning for networks. In Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 855–864. Association for Computing Machinery, 2016.[4] M. A. Hasnat, J. Bohne, S. G. J. Milgram, and L. Chen. von mises-fisher mixture model-based deep learning: Application to face verification. In arXiv preprint arXiv:1706.04264, 2017.[5] R. He, W.-C. Kang, and J. McAuley. Translation-based recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems, RecSys ’17, page 161–169. Association for Computing Machinery.[6] C.-K. Hsieh, L. Yang, Y. Cui, T.-Y. Lin, S. Belongie, and D. Estrin. Collaborative metric learning. In Proceedings of the 26th International Conference on World Wide Web, WWW ’17, page 193–201. International World Wide Web Conferences Steering Committee.[7] Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In Proceedings of the 8th IEEE International Conference on Data Min- ing, ICDM ’08, page 263–272. IEEE Computer Society.[8] S. Kabbur, X. Ning, and G. Karypis. Fism: Factored item similarity models for top-N recommender systems. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, page 659–667. Association for Computing Machinery.[9] Y.Koren, R.Bell ,and C.Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37.[10] D. Liang, J. Altosaar, L. Charlin, and D. M. Blei. Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 59–66. Association for Computing Machinery.[11] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionally. In Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS ’13, pages 3111–3119. Curran Associates Inc., 2013.[12] X. Ning and G. Karypis. Slim: Sparse linear methods for top-n recommender systems. In Proceedings of the 11th IEEE International Conference on Data Mining, ICDM ’11, page 497–506. IEEE Computer Society.[13] F. Niu, B. Recht, C. Re, and S. J. Wright. Hogwild! a lock-free approach to parallelizing stochastic gradient descent. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, page 693–701. Curran Associates Inc.[14] R. Pan, Y. Zhou, B. Cao, N. N. Liu, R. Lukose, M. Scholz, and Q. Yang. One-class collaborative filtering. In Proceedings of the 8th IEEE International Conference on Data Mining, ICDM ’08, page 502–511. IEEE Computer Society.[15] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 29th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pages 701–710. Association for Computing Machinery, 2014.[16] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI ’09, page 452–461. AUAI Press.[17] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In Proceedings of the 19th International Conference on World Wide Web, WWW ’15, pages 1067–1077. International World Wide Web Conferences Steering Committee, 2015.[18] Y. Tian, X. Yu, B. Fan, F. Wu, H. Heijnen, and V. Balntas. Sosnet: Second order similarity regularization for local descriptor learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’19. Institute of Electrical and Electronics Engineers, 2019.[19] H. Wang, M. Zhao, X. Xie, W. Li, and M. Guo. Knowledge graph convolutional networks for recommender systems. In Proceedings of the 28th International Conference on World Wide Web, WWW ’19, page 3307–3313. Association for Computing Machinery.[20] X. Wang, X. He, Y. Cao, M. Liu, and T.-S. Chua. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’19, page 950–958. Association for Computing Machinery.[21] X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Re- search and Development in Information Retrieval, SIGIR’19, page 165–174. Association for Computing Machinery.[22] J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, IJCAI’11, page 2764–2770. AAAI Press.[23] J. Weston, H. Yee, and R. J. Weiss. Learning to rank recommendations with the k-order statistic loss. In Proceedings of the 7th ACM Conference on Recommender Systems, RecSys ’13, page 245–248. Association for Computing Machinery.[24] J.-H. Yang, C.-M. Chen, C.-J. Wang, and M.-F. Tsai. Hop-rec: High-order proximity for implicit recommendation. In Proceedings of the 12nd ACM Conference on Recommender Systems, RecSys ’18, page 140–144. Association for Computing Machinery.[25] X. Zhang, F.-X. Yu, K. Sanjiv, and S.-F. Chang. Learning spread-out local feature descriptors. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), ICCV ’17. Institute of Electrical and Electronics Engineers. 描述 碩士
國立政治大學
資訊科學系
107753011資料來源 http://thesis.lib.nccu.edu.tw/record/#G0107753011 資料類型 thesis dc.contributor.advisor 蔡銘峰 zh_TW dc.contributor.advisor Tsai, Ming-Feng en_US dc.contributor.author (Authors) 莊喻能 zh_TW dc.contributor.author (Authors) Chuang, Yu-Neng en_US dc.creator (作者) 莊喻能 zh_TW dc.creator (作者) Chuang, Yu-Neng en_US dc.date (日期) 2020 en_US dc.date.accessioned 1-Jul-2020 13:50:07 (UTC+8) - dc.date.available 1-Jul-2020 13:50:07 (UTC+8) - dc.date.issued (上傳時間) 1-Jul-2020 13:50:07 (UTC+8) - dc.identifier (Other Identifiers) G0107753011 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/130589 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 107753011 zh_TW dc.description.abstract (摘要) 近年來大數據以及機器學習技術的蓬勃發展，推薦系統被廣泛應用於各種資訊系( Information Systems )上。如何有效地利用這些巨量的資料增進推薦系統效能，成為具有挑戰的工作。圖形學習表示法( Graph Embedding )便是一種特徵提取 ( Feature Extraction ) 的技術，此方法目的在於如何有效的將不同節點以及節點間的關係投射到低維度向量空間並賦予特徵向量。因此，如何有效率且精準的描述這些向量空間的概念，也被加入到圖形學習表示法的領域。本論文基於非對稱常態分佈( Skew Normal Distribution)之特性，提出以機率分佈重新檢視表示法向量空間，並針對使用者與喜好物品在非對稱常態分佈上會趨向正向偏態( Positive Skewness )的特性，將偏態之概念加入目標函式中進行優化。特別的是，本論文所提出之偏態項優化式為一通用優化項，能適用於過去各種 State of The Art 推薦演算法上，進而重塑各種推薦演算法所構建之向量空間。從理論面來論述，我們證明了如何在優化各種推薦演算法上之餘，同時優化基於非對稱常態分佈之 Shape 參數，此參數與分佈之偏態值為正相關。此外，針對所提出之演算法能同時最大化接收者操作特徵曲線( Receiver Operating Characteristic Curve ( ROC Curve ) )之論述，我們也提出一數學證述來解釋與分析。在數據實驗上，本文以將此偏態優化項主要實驗於矩陣分解類之推薦算法上，且為了展示方法的一致性，我們也將此偏態優化項實驗在基於圖形學習表示法的推薦演算法上，來做驗證本方法的可行性與正確性。而為了驗證此方法，本文實驗於五種不同的真實世 界巨量資料上，並且針對兩種常見的推薦任務: Top-N 推薦任務以及 Query-based 推薦任務上皆有所比較與操作。最後，在實驗結果的部分，結果呈現出我們所提之演算法與過去各種 State of The Art 之推薦演算法中實際比較後皆取得更優的表現。 zh_TW dc.description.abstract (摘要) In recent years, machine learning technology has drastically improved in adapting big data among various fields, including commercial streaming online service and recommendation systems. Especially in recommendation systems case, the user-based recommendation systems or personalized recommendation is one of the most challenging tasks. In this paper, hence, we propose a novel optimization criterion that leverages features of the skew normal distribution to better model the problem of personalized recommendation. Specifically, the developed criterion borrows the concept and the flexibility of the skew normal distribution and also based on three hyper-parameters to not only provide the degree of freedom in optimization and also highly attached to the optimization criterion. Moreover, we both provide the relation of optimization of the proposed criterion and the shape parameter in the skew normal distribution from theoretical point of view and provide the analogies and provide the theoretical proof on asymptotic analysis of the area under the ROC curve to our proposed method. Experimental results conducted on five large-scale real-world datasets reveal that our proposed optimization criterion significantly achieve the best performance of the state of the art and yields consistently on all tested datasets. en_US dc.description.tableofcontents 致謝 1中文摘要 2Abstract 3第一章 緒論....................................... 11.1 前言..................................... 11.2 研究目的.................................. 31.3 研究動機詳述 ............................... 4第二章 相關文獻探討.................................. 52.1 圖形學習表示法.............................. 52.2 推薦系統.................................. 62.3 表示法空間調整.............................. 8第三章 研究方法..................................... 93.1 個人化推薦系統.............................. 93.1.1 問題定義.............................. 93.1.2 先備相關知識 ........................... 103.2 偏態排序優化(SkewnessRankingOptimization) ...... 123.2.1 觀察與動機 ............................ 133.2.2 優化準則.............................. 143.2.3 模型理論敘述與證明 ....................... 163.2.4 AUC分析.............................. 19第四章 實驗結果與討論................................ 214.1 資料集 ................................... 214.2 比較基準模型 ............................... 224.3 實驗設定與驗證標準 ........................... 234.3.1 實驗設定.............................. 234.3.2 驗證標準.............................. 234.4 實驗結果.................................. 244.4.1 Top-N推薦任務表現 ....................... 244.4.2 敏感度分析 ............................ 264.4.3 機率分佈之討論.......................... 304.4.4 估計量之差異分析 ........................ 32第五章 結論....................................... 34參考文獻.......................................... 35 zh_TW dc.format.extent 2949633 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0107753011 en_US dc.subject (關鍵詞) 推薦系統 zh_TW dc.subject (關鍵詞) 協同過濾法 zh_TW dc.subject (關鍵詞) 圖形學習表示法 zh_TW dc.subject (關鍵詞) 矩陣分解 zh_TW dc.subject (關鍵詞) 偏態排序法 zh_TW dc.subject (關鍵詞) Recommender systems en_US dc.subject (關鍵詞) Collaborative filitering en_US dc.subject (關鍵詞) Graph embedding en_US dc.subject (關鍵詞) Matrix factorization en_US dc.subject (關鍵詞) Skewness optimization ranking en_US dc.title (題名) 基於偏態排序最佳化探討圖形學習表示法之分佈於推薦系統 zh_TW dc.title (題名) Exploring Distribution of Graph Embedding Based on Skewness Ranking Optimization for Recommender Systems en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] C.-M. Chen, M.-F. Tsai, Y.-C. Lin, and Y.-H. Yang. Query-based music recommendations via preference embedding. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 79–82. Association for Computing Machinery.[2] C.-M. Chen, C.-J. Wang, M.-F. Tsai, and Y.-H. Yang. Collaborative similarity em- bedding for recommender systems. In Proceedings of the 28th International Conference on World Wide Web, WWW ’19, page 2637–2643. Association for Computing Machinery.[3] A. Grover and J. Leskovec. Node2vec: Scalable feature learning for networks. In Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 855–864. Association for Computing Machinery, 2016.[4] M. A. Hasnat, J. Bohne, S. G. J. Milgram, and L. Chen. von mises-fisher mixture model-based deep learning: Application to face verification. In arXiv preprint arXiv:1706.04264, 2017.[5] R. He, W.-C. Kang, and J. McAuley. Translation-based recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems, RecSys ’17, page 161–169. Association for Computing Machinery.[6] C.-K. Hsieh, L. Yang, Y. Cui, T.-Y. Lin, S. Belongie, and D. Estrin. Collaborative metric learning. In Proceedings of the 26th International Conference on World Wide Web, WWW ’17, page 193–201. International World Wide Web Conferences Steering Committee.[7] Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In Proceedings of the 8th IEEE International Conference on Data Min- ing, ICDM ’08, page 263–272. IEEE Computer Society.[8] S. Kabbur, X. Ning, and G. Karypis. Fism: Factored item similarity models for top-N recommender systems. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, page 659–667. Association for Computing Machinery.[9] Y.Koren, R.Bell ,and C.Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37.[10] D. Liang, J. Altosaar, L. Charlin, and D. M. Blei. Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 59–66. Association for Computing Machinery.[11] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionally. In Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS ’13, pages 3111–3119. Curran Associates Inc., 2013.[12] X. Ning and G. Karypis. Slim: Sparse linear methods for top-n recommender systems. In Proceedings of the 11th IEEE International Conference on Data Mining, ICDM ’11, page 497–506. IEEE Computer Society.[13] F. Niu, B. Recht, C. Re, and S. J. Wright. Hogwild! a lock-free approach to parallelizing stochastic gradient descent. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, page 693–701. Curran Associates Inc.[14] R. Pan, Y. Zhou, B. Cao, N. N. Liu, R. Lukose, M. Scholz, and Q. Yang. One-class collaborative filtering. In Proceedings of the 8th IEEE International Conference on Data Mining, ICDM ’08, page 502–511. IEEE Computer Society.[15] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 29th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pages 701–710. Association for Computing Machinery, 2014.[16] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI ’09, page 452–461. AUAI Press.[17] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In Proceedings of the 19th International Conference on World Wide Web, WWW ’15, pages 1067–1077. International World Wide Web Conferences Steering Committee, 2015.[18] Y. Tian, X. Yu, B. Fan, F. Wu, H. Heijnen, and V. Balntas. Sosnet: Second order similarity regularization for local descriptor learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’19. Institute of Electrical and Electronics Engineers, 2019.[19] H. Wang, M. Zhao, X. Xie, W. Li, and M. Guo. Knowledge graph convolutional networks for recommender systems. In Proceedings of the 28th International Conference on World Wide Web, WWW ’19, page 3307–3313. Association for Computing Machinery.[20] X. Wang, X. He, Y. Cao, M. Liu, and T.-S. Chua. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’19, page 950–958. Association for Computing Machinery.[21] X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Re- search and Development in Information Retrieval, SIGIR’19, page 165–174. Association for Computing Machinery.[22] J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, IJCAI’11, page 2764–2770. AAAI Press.[23] J. Weston, H. Yee, and R. J. Weiss. Learning to rank recommendations with the k-order statistic loss. In Proceedings of the 7th ACM Conference on Recommender Systems, RecSys ’13, page 245–248. Association for Computing Machinery.[24] J.-H. Yang, C.-M. Chen, C.-J. Wang, and M.-F. Tsai. Hop-rec: High-order proximity for implicit recommendation. In Proceedings of the 12nd ACM Conference on Recommender Systems, RecSys ’18, page 140–144. Association for Computing Machinery.[25] X. Zhang, F.-X. Yu, K. Sanjiv, and S.-F. Chang. Learning spread-out local feature descriptors. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), ICCV ’17. Institute of Electrical and Electronics Engineers. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202000474 en_US
