高壓縮比超長文本抽象式摘要生成

Abstract

本研究探討的是文本摘要的新任務：生成具有高壓縮比的長文本抽象化摘要。\n自動摘要是自然語言處理領域中廣泛研究的主題，目前以 Transformer 架構為基礎的神經網路模型在新聞的抽象式摘要上，展現了一定的成效。\n本研究則針對更具挑戰性的輸入類型，書籍，作為摘要的對象進行探討。\n與長度僅數百字的新聞相比，書籍長達上萬字或更多，而對超長輸出入進行建模，是當前神經網路模型的一大挑戰。\n除此之外，書籍摘要需要將大量文字，改用少許概括性的文字重新表述。\n然而目前不論萃取式或抽象式摘要，主要的原理均是對輸入中的文句進行選擇與排序，故不易將大量詳細瑣碎的文句濃縮成概括性的宏觀概念。\n因此，書籍摘要的高壓縮率，構成現有摘要生成技術的另一挑戰。\n\n為解決上述的兩個挑戰，我們提出了一個基於 Transformer 神經網路的多層處理架構，適用於非常長的文本摘要，而且可以在監督式與非監督式兩種模式下運作。\n為了訓練我們的模型，我們提出了偽標記的策略，在不需額外人工標記的情況下訓練生成模型，進一步提出了一種自監督學習任務，利用多任務學習的方式，促進抽象摘要模型選用廣泛的宏觀表達方式，將具體詳細的措辭重新表述。\n實驗結果顯示，與現有方法相比，本篇論文提出的方法可以生成更好的摘要。

Text summarization is a topic widely-studied in the area of natural language processing. Most works of summarization focus on news or document summarization, where the length of input text is usually limited to hundreds of words. This work shows an attempt to deal with a much more challenging case, book summarization. Compared with news article, the length of a book usually exceeds ten thousands or even more, making a barrier to current neural network models, which have a shorter input limitation. The high compression ratio of book summarization forms another challenge for most current extractive and abstractive summarization models, which generate the summary by selecting and reordering sentences or words in the input, failing to condense details into broad, macro concepts. To address these two issues, we present a novel hierarchical model for very long text summarization in two ways, unsupervised and supervised. We train the Transformer-based generation model with pseudo-labeled data in the hierarchical manner for handling very long input text. A self-supervised learning task is further proposed for improving the ability of the abstractive summarization model for rephrasing specific, detailed wording with broad, macro expressions. Experimental results show our approach can generate better summaries compared with existing methods.