Sentimental Analysis on Big Data – On case of Financial Document Text Mining to Predict Sub-Index Trend

Abstract

This research analyzed the potential emotion by sentimental analysis in large volume of financial text documents about Taiwan electronic industry to predict the stock trend. In recent researches about sentimental analysis, supervised method was proven to be able to reach high accuracy, but the training set of supervised method should be classified by manpower and couldn’t discover the unknown category. So this research put forward a solution which mixed supervised and unsupervised methods. First, we introduce unsupervised method to find out the topics of documents. Then we calculated the sentimental index to judge the document’s emotional direction. After that, we find out which theme documents’ sentiment index are leading indicators in Taiwan electronic sub-index (TE). Finally, we used supervised method by integrating the sentimental index of leading indicators with other 24 indirect sentimental indexes to build the prediction model of TE. By result, we found that LDA model has better cluster performance than TFIDF-Kmeans model, and also has higher accuracy than NPMI-Concor model on classification. By comparing sentimental index with MACD, we proved that the trend of sentimental index and TE to each other is more similar than MACD line and TE to each other. We also discovered that the sentiment indexes from enterprise operation and macro-economics topics are leading indicators and found that the prediction model of TE which includes the sentiment index is better than which only includes the technical indicators.