Black-Litterman 模型結合強化學習之投資組合配置

Abstract

本研究嘗試將強化學習 (Reinforcement Learning) 應用於預測金融資產價格走勢，並結合Black-Litterman模型建構全球性多元投資組合。本研究使用近端策略優化演算法 (Proximal Policy Optimization, PPO)，以資產價量資料預測資產價格漲跌及漲跌幅度，並將預測結果作為Black-Litterman模型中的投資者觀點進行資產配置，比較投資組合在不同獎勵設定及不同更新次數下的績效表現。本研究以美國五檔不同資產類別ETF作為基礎資產，研究結果顯示強化學習在一定更新次數上具有預測力，本研究建立之投資組合績效在更新次數600000皆能贏過其餘基準模型。另外，對於強化學習而言，以不同獎勵設定訓練模型比起增加更新次數對績效有著較大的影響。

In this thesis, we try to apply reinforcement learning to forecasting price trends of finance assets. We combine the forecasts with the Black-litterman model and construct various globally diversified portfolios. In our paper, we use the Proximal Policy Optimization algorithm to forecast assets’ price trends by historical price and volume. The prediction of results are used as the investor’s views in the Black-Litterman model for asset allocation. This study compares the performance of the portfolios under different reward setting and number of updates. The empirical results show that reinforcement learning has predictive power at a certain number of updates. The portfolios performance in this study outperform the benchmark portfolios at 600,000 updates. In addition, for the reinforcement learning, training the model with different reward setting has a greater impact on performance than increasing the number of updates.