Publications-Theses
Article View/Open
Publication Export
-
題名 使用子主題建模的零樣本立場偵測
Zero-shot stance detection with subtopic modeling作者 穆永綸
Mu, Yung-Lun貢獻者 黃瀚萱
Huang, Hen-Hsen
穆永綸
Mu, Yung-Lun關鍵詞 立場偵測
遷移學習
零樣本學習
Stance detection
Transfer learning
Zero-shot learning日期 2022 上傳時間 2-Dec-2022 15:23:22 (UTC+8) 摘要 立場偵測任務有助我們初步了解發文者對特定主題的態度，但立場偵測模型的建立往往受限於標籤資料的不足，本研究針對可充分運用既有標籤資料的零樣本立場偵測任務進行研究，此外，由於網路社群媒體儼然是當前了解民眾意向的重要媒介，我們聚焦網路社群文本的立場偵測任務。我們的模型同時考量了「主題獨立」以及「主題相依」2 種可遷移學習的文本特徵，提升模型的泛化能力，利用資料集的立場標籤與主題資訊，進行對抗學習與監督對比學習：透過立場與主題分類的對抗學習，加強模型對「主題獨立」的特徵表示，另以 2 種監督對比學習設定，在強化模型對「主題獨立」特徵表示的同時，也能凸顯「主題相依」的特徵表示，實驗顯示此方法有助於立場偵測任務。另我們也嘗試透過LDA 子主題建模，以LDA 主題模型產生的子主題詞組增加輸入模型的語意資訊，然由於 LDA 主題模型是以非監督學習方法建模，其過程並未考量立場偵測任務需求，使得子主題詞組的方法未能在每個主題都發揮效果。
Stance detection(SD) tasks give us an initial look at the attitudes of author on a particular topic. Though fully supervised SD model can achieve favorable performance, the lack of labeled data limits its availability. In this work, we focus on zero-shot stance detection(ZSSD). Besides, as online social media get popular, we take textual content from online community as our research target.To generalize stance features for unseen target, we consider both ” topic-invariant” and ” topic-dependent” features that are transferable between source and target domain. Specifically, to make the use of ”topic-invariant” features, astance classifier and a topic discriminator is set for adversarial learning. To further generalize ” topic-invariant” and ” topic-dependent” stance features, contrastive learning strategy is deployed using the stance label and topic information from data-set. Experiments on benchmark data-set show that the proposed approach is feasible. Furthermore, we build LDA subtopic model for each target and augment the semantic information through the subtopic words. However, since our subtopic modeling task is unsupervised and independent from stance detection task, the beneficial of subtopic modeling turned out to be unstable.參考文獻 [1] Abeer ALDayel and Walid Magdy. Stance detection on social media: State of the art and trends. Information Processing Management, 58(4):102597, 2021.[2] Emily Allaway and Kathleen McKeown. Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In Proceedings of the 2020 Conferenceon Empirical Methods in Natural Language Processing (EMNLP), pages 8913–8931, Online, November 2020. Association for Computational Linguistics.[3] Emily Allaway, Malavika Srikanth, and Kathleen McKeown. Adversarial learning for zero-shot stance detection on social media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4756–4767, Online, June 2021. Association for Computational Linguistics.[4] Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. Stance detection with bidirectional conditional encoding. 2016.[5] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022, 2003.[6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprintarXiv:1810.04805, 2018.[7] Rui Dong, Yizhou Sun, Lu Wang, Yupeng Gu, and Yuan Zhong. Weakly-guided user stance prediction via joint modeling of content and social interaction. In Proceedings ofthe 2017 ACM on Conference on Information and Knowledge Management. ACM, nov 2017.[8] Matthew Hoffman, Francis Bach, and David Blei. Online learning for latent dirichlet allocation. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors,Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010.[9] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. 04 2020.[10] Dilek Küçük and Fazli Can. Stance detection. ACM Computing Surveys, 53(1):1–37, jan 2021.[11] Stephen Wai Hang Kwok, Sai Kumar Vadde, and Guanjin” Wang. Tweet topics and sentiments relating to covid-19 vaccination among australian twitter users: Machinelearning analysis. J Med Internet Res, 23(5):e26953, May 2021.[12] Bin Liang, Zixiao Chen, Lin Gui, Yulan He, Min Yang, and Ruifeng Xu. Zero-shot stance detection via contrastive learning. In Proceedings of the ACM Web Conference 2022. ACM, apr 2022.[13] Bin Liang, Yonghao Fu, Lin Gui, Min Yang, Jiachen Du, Yulan He, and Ruifeng Xu. Target-adaptive graph for cross-target stance detection. In Proceedings of the Web Conference 2021. ACM, apr 2021.[14] Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. JointCL: A joint contrastive learning framework for zero-shot stance detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 81–91, Dublin, Ireland, May 2022. Association for ComputationalLinguistics.[15] Rui Liu, Zheng Lin, Peng Fu, Yuanxin Liu, and Weiping Wang. Connecting targets via latent topics and contrastive learning: A unified framework for robust zero-shot and few-shot stance detection. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7812–7816, 2022.[16] Sandra Maria Correia Loureiro, João Guerreiro, and Iis Tussyadiah. Artificial intelligence in business: State of the art and future research agenda. Journal of Business Research,129:911–926, 2021.[17] Sara Mifrah. Topic modeling coherence: A comparative study between LDA and NMF models using COVID’19 corpus. International Journal of Advanced Trends in Computer Science and Engineering, 9(4):5756–5761, aug 2020.[18] Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41, San Diego, California, June 2016. Association for Computational Linguistics.[19] Chang Xu, Cecile Paris, Surya Nepal, and Ross Sparks. Cross-target stance classification with self-attention networks. 05 2018.[20] Guido Zarrella and Amy Marsh. MITRE at SemEval-2016 task 6: Transfer learning for stance detection. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 458–463, San Diego, California, June 2016. Association for Computational Linguistics.[21] Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. Aspect-augmented adversarial networks for domain adaptation. Transactions of the Association for Computational Linguistics, 5:515–528, dec 2017. 描述 碩士
國立政治大學
資訊科學系碩士在職專班
109971003資料來源 http://thesis.lib.nccu.edu.tw/record/#G0109971003 資料類型 thesis dc.contributor.advisor 黃瀚萱 zh_TW dc.contributor.advisor Huang, Hen-Hsen en_US dc.contributor.author (Authors) 穆永綸 zh_TW dc.contributor.author (Authors) Mu, Yung-Lun en_US dc.creator (作者) 穆永綸 zh_TW dc.creator (作者) Mu, Yung-Lun en_US dc.date (日期) 2022 en_US dc.date.accessioned 2-Dec-2022 15:23:22 (UTC+8) - dc.date.available 2-Dec-2022 15:23:22 (UTC+8) - dc.date.issued (上傳時間) 2-Dec-2022 15:23:22 (UTC+8) - dc.identifier (Other Identifiers) G0109971003 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/142662 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系碩士在職專班 zh_TW dc.description (描述) 109971003 zh_TW dc.description.abstract (摘要) 立場偵測任務有助我們初步了解發文者對特定主題的態度，但立場偵測模型的建立往往受限於標籤資料的不足，本研究針對可充分運用既有標籤資料的零樣本立場偵測任務進行研究，此外，由於網路社群媒體儼然是當前了解民眾意向的重要媒介，我們聚焦網路社群文本的立場偵測任務。我們的模型同時考量了「主題獨立」以及「主題相依」2 種可遷移學習的文本特徵，提升模型的泛化能力，利用資料集的立場標籤與主題資訊，進行對抗學習與監督對比學習：透過立場與主題分類的對抗學習，加強模型對「主題獨立」的特徵表示，另以 2 種監督對比學習設定，在強化模型對「主題獨立」特徵表示的同時，也能凸顯「主題相依」的特徵表示，實驗顯示此方法有助於立場偵測任務。另我們也嘗試透過LDA 子主題建模，以LDA 主題模型產生的子主題詞組增加輸入模型的語意資訊，然由於 LDA 主題模型是以非監督學習方法建模，其過程並未考量立場偵測任務需求，使得子主題詞組的方法未能在每個主題都發揮效果。 zh_TW dc.description.abstract (摘要) Stance detection(SD) tasks give us an initial look at the attitudes of author on a particular topic. Though fully supervised SD model can achieve favorable performance, the lack of labeled data limits its availability. In this work, we focus on zero-shot stance detection(ZSSD). Besides, as online social media get popular, we take textual content from online community as our research target.To generalize stance features for unseen target, we consider both ” topic-invariant” and ” topic-dependent” features that are transferable between source and target domain. Specifically, to make the use of ”topic-invariant” features, astance classifier and a topic discriminator is set for adversarial learning. To further generalize ” topic-invariant” and ” topic-dependent” stance features, contrastive learning strategy is deployed using the stance label and topic information from data-set. Experiments on benchmark data-set show that the proposed approach is feasible. Furthermore, we build LDA subtopic model for each target and augment the semantic information through the subtopic words. However, since our subtopic modeling task is unsupervised and independent from stance detection task, the beneficial of subtopic modeling turned out to be unstable. en_US dc.description.tableofcontents 致謝 i中文摘要 iiAbstract iii目錄 iv表目錄 vi圖目錄 vii第一章 緒論 1第一節 研究背景與動機 1第二節 研究目的 2第二章 文獻探討 4第一節 立場偵測任務 4第二節 監督學習的立場偵測 5第三節 零樣本學習的立場偵測 6第四節 LDA 主題模型 8第三章 研究方法 9第一節 立場偵測任務 9第二節 模型架構 10一、編碼器 11二、轉換層 11三、立場分類器 12四、主題鑑別器 13第三節 監督對比學習 13第四節 學習目標 14第五節 LDA 子主題建模 15第四章 實驗 17第一節 資料集 17一、SemEval 2016 立場偵測資料集 17二、無標籤資料 18三、資料切分與使用 19第二節 評估指標 19第三節 LDA 子主題 20第四節 立場偵測模型實作 20第五章 實驗結果 22第一節 LDA 子主題詞組 22一、各主題模型的一致性分數 22二、對 BERT 序列分類模型的效能影響 22第二節 模型效能 23一、對抗學習機制 23二、監督對比學習與 LDA 子主題 23三、子主題對主題鑑別的影響 24四、條件式編碼器的注意力權重 25第三節 小結 27第六章 結論與未來研究方向 28第一節 結論 28第二節 未來研究方向 29參考文獻 30 zh_TW dc.format.extent 1083100 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0109971003 en_US dc.subject (關鍵詞) 立場偵測 zh_TW dc.subject (關鍵詞) 遷移學習 zh_TW dc.subject (關鍵詞) 零樣本學習 zh_TW dc.subject (關鍵詞) Stance detection en_US dc.subject (關鍵詞) Transfer learning en_US dc.subject (關鍵詞) Zero-shot learning en_US dc.title (題名) 使用子主題建模的零樣本立場偵測 zh_TW dc.title (題名) Zero-shot stance detection with subtopic modeling en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Abeer ALDayel and Walid Magdy. Stance detection on social media: State of the art and trends. Information Processing Management, 58(4):102597, 2021.[2] Emily Allaway and Kathleen McKeown. Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. In Proceedings of the 2020 Conferenceon Empirical Methods in Natural Language Processing (EMNLP), pages 8913–8931, Online, November 2020. Association for Computational Linguistics.[3] Emily Allaway, Malavika Srikanth, and Kathleen McKeown. Adversarial learning for zero-shot stance detection on social media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4756–4767, Online, June 2021. Association for Computational Linguistics.[4] Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. Stance detection with bidirectional conditional encoding. 2016.[5] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022, 2003.[6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprintarXiv:1810.04805, 2018.[7] Rui Dong, Yizhou Sun, Lu Wang, Yupeng Gu, and Yuan Zhong. Weakly-guided user stance prediction via joint modeling of content and social interaction. In Proceedings ofthe 2017 ACM on Conference on Information and Knowledge Management. ACM, nov 2017.[8] Matthew Hoffman, Francis Bach, and David Blei. Online learning for latent dirichlet allocation. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors,Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010.[9] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. 04 2020.[10] Dilek Küçük and Fazli Can. Stance detection. ACM Computing Surveys, 53(1):1–37, jan 2021.[11] Stephen Wai Hang Kwok, Sai Kumar Vadde, and Guanjin” Wang. Tweet topics and sentiments relating to covid-19 vaccination among australian twitter users: Machinelearning analysis. J Med Internet Res, 23(5):e26953, May 2021.[12] Bin Liang, Zixiao Chen, Lin Gui, Yulan He, Min Yang, and Ruifeng Xu. Zero-shot stance detection via contrastive learning. In Proceedings of the ACM Web Conference 2022. ACM, apr 2022.[13] Bin Liang, Yonghao Fu, Lin Gui, Min Yang, Jiachen Du, Yulan He, and Ruifeng Xu. Target-adaptive graph for cross-target stance detection. In Proceedings of the Web Conference 2021. ACM, apr 2021.[14] Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. JointCL: A joint contrastive learning framework for zero-shot stance detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 81–91, Dublin, Ireland, May 2022. Association for ComputationalLinguistics.[15] Rui Liu, Zheng Lin, Peng Fu, Yuanxin Liu, and Weiping Wang. Connecting targets via latent topics and contrastive learning: A unified framework for robust zero-shot and few-shot stance detection. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7812–7816, 2022.[16] Sandra Maria Correia Loureiro, João Guerreiro, and Iis Tussyadiah. Artificial intelligence in business: State of the art and future research agenda. Journal of Business Research,129:911–926, 2021.[17] Sara Mifrah. Topic modeling coherence: A comparative study between LDA and NMF models using COVID’19 corpus. International Journal of Advanced Trends in Computer Science and Engineering, 9(4):5756–5761, aug 2020.[18] Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41, San Diego, California, June 2016. Association for Computational Linguistics.[19] Chang Xu, Cecile Paris, Surya Nepal, and Ross Sparks. Cross-target stance classification with self-attention networks. 05 2018.[20] Guido Zarrella and Amy Marsh. MITRE at SemEval-2016 task 6: Transfer learning for stance detection. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 458–463, San Diego, California, June 2016. Association for Computational Linguistics.[21] Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. Aspect-augmented adversarial networks for domain adaptation. Transactions of the Association for Computational Linguistics, 5:515–528, dec 2017. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202201678 en_US
