基於注意力機制語言模型之財務風險文章偵測與實體辨識

Abstract

本研究利用注意力機制模型偵測財務文章之風險事件及抽取潛在金融犯罪名單，建構自動化模型以降低人力標記成本及提升預測速度。我們分析不同模型架構及訓練方法之優缺點，並比較傳統神經網路方法與 Transformer Based 模型的差異。模型架構分為兩階段，第一階段判斷目標文章是否包含金融風險事件，而第二階段則在這些文章中抽取高危險的名單。我們提出聯合訓練方法同時訓練兩階段的模型，透過實驗證明可在不損失正確性的情況提升訓練及預測速度，並得以提升模型穩定性。我們亦針對注意力機制模型內部的 Attention Weight 做視覺化分析，顯示模型能在不提供標注的情況自動關注金融風險詞彙。另外我們針對缺乏風險人名標記的訓練資料之情況，利用以上 Attention Weight 分析設計特殊的規則，達到一定程度的效果提升。最後我們額外在一個 Wikipedia 上的英文資料集做測試，說明此研究結果亦可應用於不同領域及不同語言的任務。

This thesis uses transformer-based models to detect risk events from financial articles and extract potential financial criminals. With such automated models, we can reduce human costs on labeling and increase prediction performance. In this thesis, we analyze the advantages and disadvantages of different approaches and compare the differences between traditional neural networks and\nTransformer-based models. The proposed method contains two stages: the first stage determines whether the target news contains financial risk events, and the second stage extracts high-risk entities from the news. We propose a\njoint-training method to train these two stages at the same time. Experimental results show that the proposed joint-training method improves prediction accuracy and enhances the stability of the training process. We also visualize\nthe attention weights of the attention mechanism model, showing that the model automatically pays attention to financial risk vocabularies without providing annotations. In addition, we use the above attention weight scheme to design special rules, achieving a certain degree of effect improvement for the case that lacks risk-name-annotation. Finally, further experiments conducted on a dataset from English Wikipedia confirm that the proposed method can also apply to different domains and languages.