1. NCCU Academic Hub
  2. Publications
  3. College of Informatics
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 新聞事件觸發之全文知識更新
Knowledge Update in Full Text Triggered by a News Event
作者 李昱廷
Lee, Yu-Ting
貢獻者 李蔡彥<br>黃瀚萱
Li, Tsai-Yen<br>Huang, Hen-Hsen
李昱廷
Lee, Yu-Ting
關鍵詞 文本生成
時序知識建模
知識更新
自然語言生成
大型語言模型
內容改寫
新聞事件
Text Generation
Temporal Knowledge Modeling
Update Summarization
Natural Language Generation
Knowledge Update
Large Language Model
Text Revision
News Event
日期 2023
上傳時間 1-Dec-2023 10:34:04 (UTC+8)
摘要 在網路資訊的快速發展下，每日的新聞事件更迭與知識獲取已成為人們主要獲取資訊的管道，新的知識內容每分每秒都不斷地在發生，而保持資訊的更新也需要極大的人力和時間成本。在本研究中，我們規劃了一項新的自然語言生成任務，即新聞事件觸發之知識更新。研究目標以現有關於某主題的文章或舊版本的內容和一個關於該主題的新聞事件，根據該新聞事件的資訊生成一篇更新後的文章。在資料蒐集的過程，我們建立一個多粒度的新聞資料集以適用於研究目標。蒐集主要源自於維基百科的文章，經由爬取並與多種語言的新聞事件對齊，而資料集包含有引文、文章之首段和文章的全文。我們提出改良後的模型設計用於自動化更新全文知識，並以多個大型語言模型驗證模型架構之有效性。
With the rapid development of internet information, daily news events and knowledge acquisition have become the primary channels for people to access information. New knowledge content is constantly emerging every second, requiring significant human and time resources to ensure knowledge updates. In this research, we propose a new natural language generation task, namely ”Knowledge Update in Full Text Triggered by a News Event”. Our objective is to generate an updated article based on a given news event and existing articles or old versions of content on a specific topic. To facilitate our research objective, we construct a multi-granularity news dataset suitable for our task. The dataset is primarily sourced from Wikipedia articles, crawled and aligned with news events in multiple language units. Dataset includes citations, first paragraphs, and full-text articles. We present an improved model architecture tailored specifically for the task of updating knowledge in full-content articles and validate the effectiveness of our framework with multiple large language models.
參考文獻 [1] OpenAI. OpenAI: Introducing ChatGPT. https://openai.com/blog/chatgpt, 2022. [2] OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774, 2023. [3] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm.stanford.edu/2023/03/13/alpaca.html, 3(6):7, 2023. [4] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. https://vicuna.lmsys.org (accessed 14 April 2023), 2023. [5] Demian Gholipour Ghalandari, Chris Hokamp, Nghia The Pham, John Glover, and Georgiana Ifrim. A large-scale multi-document summarization dataset from the wikipedia current events portal. arXiv preprint arXiv:2005.10070, 2020. [6] Yu-Ting Lee, Ying-Jhe Tang, Yu-Chung Cheng, Pai-Lin Chen, Tsai-YenLi, and Hen-Hsen Huang. A multi-grained dataset for news event triggered knowledge update. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4158–4162, 2022. [7] Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed H Awadallah, and Dragomir Radev. Leveraging locality in abstractive text summarization. arXiv preprint arXiv:2205.12476, 2022. [8] Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, and Furu Wei. Attention temperature matters in abstractive summarization distillation. arXiv preprint arXiv:2106.03441, 2021. [9] Ashish Vaswani, Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones, AidanN Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [10] Sansiri Tarnpradab, Fereshteh Jafariakinabad, and Kien A Hua. Improving online forums summarization via hierarchical unified deep neural network. arXiv preprint arXiv:2103.13587, 2021. [11] Wei Li, Xinyan Xiao, Jiachen Liu, Hua Wu, Haifeng Wang, and Junping Du. Leveraging graph to improve abstractive multi-document summarization. arXiv preprint arXiv:2005.10043, 2020. [12] Hussam Alkaissi and Samy I McFarlane. Artificial hallucinations in chatgpt: implications in scientific writing. Cureus, 15(2), 2023. [13] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. [14] Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. Primera: Pyramid-based masked sentence pre-training for multi-document summarization. arXiv preprint arXiv:2110.08499, 2021. [15] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv e-prints, page arXiv:1910.10683, October 2019. [16] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965, 2022. [17] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [18] Sheena Panthaplackel, Adrian Benton, and Mark Dredze. Updated headline generation: Creating updated summaries for evolving news stories. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6438–6461, 2022. [19] Hoa Trang Dang, Karolina Owczarzak, et al. Overview of the tac 2008 update summarization task. In TAC, 2008. [20] Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020. [21] Sho Takase, JunSuzuki, Naoaki Okazaki, Tsutomu Hirao,and Masaaki Nagata. Neural headline generation on abstract meaning representation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 1054–1059, 2016. [22] David Zajic,Bonnie Dorr,and Richard Schwartz. Automatic headline generation for newspaper stories. In Workshop on Text Summarization (ACL 2002 and DUC 2002 meeting on Text Summarization). Philadelphia, page 65, 2002. [23] Alexander Spangher, Xiang Ren, Jonathan May, and Nanyun Peng. Newsedits: A news article revision dataset and a novel document-level reasoning challenge. In Proceedings of the 2022 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, pages 127–157, 2022. [24] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. [25] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [26] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [27] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. [28] Alexander R Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv preprint arXiv:1906.01749, 2019. [29] Nachshon Cohen, Oren Kalinsky, Yftah Ziser, and Alessandro Moschitti. Wikisum: Coherent summarization dataset for efficient human evaluation. 2021. [30] Paul Over and James Yen. An introduction to duc-2004. National Institute of Standards and Technology, 2004. [31] Kristina Toutanova, Chris Brockett, Michael Gamon, Jagadeesh Jagarlamudi, Hisami Suzuki, and Lucy Vanderwende. The pythy summarization system: Microsoft research at duc 2007. In Proc. of DUC, volume 2007. Citeseer, 2007. [32] Colin B Clement, Matthew Bierbaum, Kevin P O’Keeffe, and Alexander A Alemi. On the use of arxiv as a dataset. arXiv preprint arXiv:1905.00075, 2019. [33] Peter J. Denning. The locality principle. Commun. ACM, 48(7):19–24, jul 2005. [34] Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489, San Diego, California, June 2016. Association for Computational Linguistics. [35] Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierar- chical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074–1084, Florence, Italy, July 2019. Association for Computational Linguistics. [36] Yang Liu and Mirella Lapata. Hierarchical transformers for multi-document summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5070–5081, Florence, Italy, July 2019. Association for Computational Linguistics. [37] Jianmin Zhang, Jiwei Tan,and Xiaojun Wan.Adapting neural single-document summarization model for abstractive multi-document summarization: A pilot study. In Proceedings of the 11th International Conference on Natural Language Generation, pages 381–390, Tilburg University, The Netherlands, November 2018. Association for Computational Linguistics. [38] Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed H Awadallah, and Dragomir Radev. Dyle: Dynamic latent extraction for abstractive long-input summarization. arXiv preprint arXiv:2110.08168, 2021. [39] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014. [40] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. [41] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. [42] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005. [43] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. [44] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
描述 碩士
國立政治大學
資訊科學系
110753204
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0110753204
資料類型 thesis
dc.contributor.advisor 李蔡彥<br>黃瀚萱zh_TW
dc.contributor.advisor Li, Tsai-Yen<br>Huang, Hen-Hsenen_US
dc.contributor.author (Authors) 李昱廷zh_TW
dc.contributor.author (Authors) Lee, Yu-Tingen_US
dc.creator (作者) 李昱廷zh_TW
dc.creator (作者) Lee, Yu-Tingen_US
dc.date (日期) 2023en_US
dc.date.accessioned 1-Dec-2023 10:34:04 (UTC+8)-
dc.date.available 1-Dec-2023 10:34:04 (UTC+8)-
dc.date.issued (上傳時間) 1-Dec-2023 10:34:04 (UTC+8)-
dc.identifier (Other Identifiers) G0110753204en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/148475-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學系zh_TW
dc.description (描述) 110753204zh_TW
dc.description.abstract (摘要) 在網路資訊的快速發展下，每日的新聞事件更迭與知識獲取已成為人們主要獲取資訊的管道，新的知識內容每分每秒都不斷地在發生，而保持資訊的更新也需要極大的人力和時間成本。在本研究中，我們規劃了一項新的自然語言生成任務，即新聞事件觸發之知識更新。研究目標以現有關於某主題的文章或舊版本的內容和一個關於該主題的新聞事件，根據該新聞事件的資訊生成一篇更新後的文章。在資料蒐集的過程，我們建立一個多粒度的新聞資料集以適用於研究目標。蒐集主要源自於維基百科的文章，經由爬取並與多種語言的新聞事件對齊，而資料集包含有引文、文章之首段和文章的全文。我們提出改良後的模型設計用於自動化更新全文知識，並以多個大型語言模型驗證模型架構之有效性。zh_TW
dc.description.abstract (摘要) With the rapid development of internet information, daily news events and knowledge acquisition have become the primary channels for people to access information. New knowledge content is constantly emerging every second, requiring significant human and time resources to ensure knowledge updates. In this research, we propose a new natural language generation task, namely ”Knowledge Update in Full Text Triggered by a News Event”. Our objective is to generate an updated article based on a given news event and existing articles or old versions of content on a specific topic. To facilitate our research objective, we construct a multi-granularity news dataset suitable for our task. The dataset is primarily sourced from Wikipedia articles, crawled and aligned with news events in multiple language units. Dataset includes citations, first paragraphs, and full-text articles. We present an improved model architecture tailored specifically for the task of updating knowledge in full-content articles and validate the effectiveness of our framework with multiple large language models.en_US
dc.description.tableofcontents 第一章 緒論 1 第二章 文獻探討 7 第一節 長文本輸入研究 7 第二節 標題更新生成 10 第三節 文本跨度與注意力研究 12 第四節 文章編輯行為研究 14 第五節 小結 16 第三章 研究方法 19 第一節 方法架構 19 第二節 資料蒐集 20 第三節 資料集組成 21 第四節 資料測試 23 第五節 模型設計 25 第六節 核心演算法 27 第七節 非對齊式句子標記演算法 28 第八節 雙向句子編輯行為標記演算法 30 第九節 內容篩選與生成 31 第十節 系統建置與演示 37 第四章 實驗結果 42 第一節 實驗說明 42 第二節 評估指標 42 第三節 以BART驗證模型架構 45 第四節 Decoding: 與 prompt-based 模型協作 48 第五節 zero-shot 環境與 fine-tuning 開源 prompt-based 模型 49 第五章 結論與未來展望 52 第一節 研究結論 52 第二節 實際應用 53 第三節 未來發展與改進 54 參考文獻 56zh_TW
dc.format.extent 6983146 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0110753204en_US
dc.subject (關鍵詞) 文本生成zh_TW
dc.subject (關鍵詞) 時序知識建模zh_TW
dc.subject (關鍵詞) 知識更新zh_TW
dc.subject (關鍵詞) 自然語言生成zh_TW
dc.subject (關鍵詞) 大型語言模型zh_TW
dc.subject (關鍵詞) 內容改寫zh_TW
dc.subject (關鍵詞) 新聞事件zh_TW
dc.subject (關鍵詞) Text Generationen_US
dc.subject (關鍵詞) Temporal Knowledge Modelingen_US
dc.subject (關鍵詞) Update Summarizationen_US
dc.subject (關鍵詞) Natural Language Generationen_US
dc.subject (關鍵詞) Knowledge Updateen_US
dc.subject (關鍵詞) Large Language Modelen_US
dc.subject (關鍵詞) Text Revisionen_US
dc.subject (關鍵詞) News Eventen_US
dc.title (題名) 新聞事件觸發之全文知識更新zh_TW
dc.title (題名) Knowledge Update in Full Text Triggered by a News Eventen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) [1] OpenAI. OpenAI: Introducing ChatGPT. https://openai.com/blog/chatgpt, 2022. [2] OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774, 2023. [3] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm.stanford.edu/2023/03/13/alpaca.html, 3(6):7, 2023. [4] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. https://vicuna.lmsys.org (accessed 14 April 2023), 2023. [5] Demian Gholipour Ghalandari, Chris Hokamp, Nghia The Pham, John Glover, and Georgiana Ifrim. A large-scale multi-document summarization dataset from the wikipedia current events portal. arXiv preprint arXiv:2005.10070, 2020. [6] Yu-Ting Lee, Ying-Jhe Tang, Yu-Chung Cheng, Pai-Lin Chen, Tsai-YenLi, and Hen-Hsen Huang. A multi-grained dataset for news event triggered knowledge update. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4158–4162, 2022. [7] Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed H Awadallah, and Dragomir Radev. Leveraging locality in abstractive text summarization. arXiv preprint arXiv:2205.12476, 2022. [8] Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, and Furu Wei. Attention temperature matters in abstractive summarization distillation. arXiv preprint arXiv:2106.03441, 2021. [9] Ashish Vaswani, Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones, AidanN Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [10] Sansiri Tarnpradab, Fereshteh Jafariakinabad, and Kien A Hua. Improving online forums summarization via hierarchical unified deep neural network. arXiv preprint arXiv:2103.13587, 2021. [11] Wei Li, Xinyan Xiao, Jiachen Liu, Hua Wu, Haifeng Wang, and Junping Du. Leveraging graph to improve abstractive multi-document summarization. arXiv preprint arXiv:2005.10043, 2020. [12] Hussam Alkaissi and Samy I McFarlane. Artificial hallucinations in chatgpt: implications in scientific writing. Cureus, 15(2), 2023. [13] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. [14] Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. Primera: Pyramid-based masked sentence pre-training for multi-document summarization. arXiv preprint arXiv:2110.08499, 2021. [15] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv e-prints, page arXiv:1910.10683, October 2019. [16] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965, 2022. [17] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [18] Sheena Panthaplackel, Adrian Benton, and Mark Dredze. Updated headline generation: Creating updated summaries for evolving news stories. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6438–6461, 2022. [19] Hoa Trang Dang, Karolina Owczarzak, et al. Overview of the tac 2008 update summarization task. In TAC, 2008. [20] Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020. [21] Sho Takase, JunSuzuki, Naoaki Okazaki, Tsutomu Hirao,and Masaaki Nagata. Neural headline generation on abstract meaning representation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 1054–1059, 2016. [22] David Zajic,Bonnie Dorr,and Richard Schwartz. Automatic headline generation for newspaper stories. In Workshop on Text Summarization (ACL 2002 and DUC 2002 meeting on Text Summarization). Philadelphia, page 65, 2002. [23] Alexander Spangher, Xiang Ren, Jonathan May, and Nanyun Peng. Newsedits: A news article revision dataset and a novel document-level reasoning challenge. In Proceedings of the 2022 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, pages 127–157, 2022. [24] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. [25] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [26] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [27] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. [28] Alexander R Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv preprint arXiv:1906.01749, 2019. [29] Nachshon Cohen, Oren Kalinsky, Yftah Ziser, and Alessandro Moschitti. Wikisum: Coherent summarization dataset for efficient human evaluation. 2021. [30] Paul Over and James Yen. An introduction to duc-2004. National Institute of Standards and Technology, 2004. [31] Kristina Toutanova, Chris Brockett, Michael Gamon, Jagadeesh Jagarlamudi, Hisami Suzuki, and Lucy Vanderwende. The pythy summarization system: Microsoft research at duc 2007. In Proc. of DUC, volume 2007. Citeseer, 2007. [32] Colin B Clement, Matthew Bierbaum, Kevin P O’Keeffe, and Alexander A Alemi. On the use of arxiv as a dataset. arXiv preprint arXiv:1905.00075, 2019. [33] Peter J. Denning. The locality principle. Commun. ACM, 48(7):19–24, jul 2005. [34] Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489, San Diego, California, June 2016. Association for Computational Linguistics. [35] Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierar- chical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074–1084, Florence, Italy, July 2019. Association for Computational Linguistics. [36] Yang Liu and Mirella Lapata. Hierarchical transformers for multi-document summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5070–5081, Florence, Italy, July 2019. Association for Computational Linguistics. [37] Jianmin Zhang, Jiwei Tan,and Xiaojun Wan.Adapting neural single-document summarization model for abstractive multi-document summarization: A pilot study. In Proceedings of the 11th International Conference on Natural Language Generation, pages 381–390, Tilburg University, The Netherlands, November 2018. Association for Computational Linguistics. [38] Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed H Awadallah, and Dragomir Radev. Dyle: Dynamic latent extraction for abstractive long-input summarization. arXiv preprint arXiv:2110.08168, 2021. [39] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014. [40] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. [41] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. [42] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005. [43] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. [44] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.zh_TW