學術產出-Theses
Article View/Open
Publication Export
-
題名 基於深度強化學習來探索市場與資產因子於資產配置策略優化
Exploring Portfolio Optimization Strategy with Market and Asset Factors in Deep Reinforcement Learning作者 張柏詠
Chang, Pai-Yung貢獻者 胡毓忠
Hu, Yuh-Jong
張柏詠
Chang, Pai-Yung關鍵詞 機器學習
深度強化學習
資產配置
投資組合
DDPG
LSTM
Machine Learning
Deep Reinforcement Learning
Asset Allocation
Investment portfolio
DDPG
LSTM日期 2023 上傳時間 9-Mar-2023 18:37:46 (UTC+8) 摘要 本文共有三個實驗命題，命題一，研究深度強化學習模型在不同市場情境下如何應變。命題二，比較深度強化學習資產配置模型與加入市場分數之資產配置模型之風險報酬。命題三，探討加入市場分數之資產配置模型與加入無風險資產之資產配置模型差異。以三命題探索不同市場趨勢與不同資產池對於深度強化學習資產配置模型之各類比較，命題一研究顯示深度強化學習資產配置模型在市場趨勢屬於恐慌時期與熊市時皆能有效改善風險報酬率；命題二研究成果顯示將資產配置模型拆成資產分數與市場分數兩部分，能在有效降低風險同時保有一定的獲利能力，風險報酬率更勝單純資產配置模型。命題三研究成果顯示加入市場分數與於資產池中加入無風險資產之資產模型各有優缺點，然長期來看加入市場分數較能在承受相同風險條件下追求更優的獲利率。三命題皆以風險與報酬指標來比較不同資產配置模型優劣，期望能建構出穩定獲利資產配置模型。
There are three experimental purposes in this paper. First, how will the DRL asset management model respond to different market trends？ Second, compare the risk-reward of the DRL asset management model with the DRL asset management model added market score. Third, discuss the difference between the asset management model adding market scores and the asset management model adding risk-free assets. Explore various comparisons of different market trends and different asset pools for the DRL asset management model with three propositions. The results revealed that the DRL model can effectively improve the risk-reward ratio during the great depression and bear market. The research results of proposition 2 show that splitting the asset management model into two parts, the asset score and the market score, can effectively reduce risks while maintaining certain profitability, and the risk-reward ratio is better than the traditional asset management model. The research results of proposition 3 show that adding market scores and adding risk-free assets to the asset pool have its own advantages and disadvantages. However, in the long term, adding market scores can pursue better profitability under the same risk conditions.參考文獻 [1] Chen, C., Zhao, L., Bian, J., Xing, C., & Liu, T. Y. (2019). Investment behaviors can tell what inside: Exploring stock intrinsic properties for stock trend prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 2376-2384)[2] Darapaneni, N., Basu, A., Savla, S., Gururajan, R., Saquib, N., Singhavi, S., ... & Paduri, A. R. (2020). Automated portfolio rebalancing using Q-learning. In 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (pp. 0596-0602).[3] Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2016). Deep direct reinforcement learning for financial signal representation and trading. IEEE transactions on neural networks and learning systems, 28(3) (pp.653-664).[4] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8) (pp. 1735-1780).[5] Hoseinzade, E., & Haratizadeh, S. (2019). CNNpred: CNN-based stock market prediction using a diverse set of variables. Expert Systems with Applications, 129 (pp. 273-285).[6] Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.[7] Jin, O., & El-Saawy, H. (2016). Portfolio management using reinforcement learning. Stanford University.[8] Lintner, J. (1975). The valuation of risk assets and the selection of risky investments in stock portfolios and capital budgets. In Stochastic optimization models in finance (pp. 131-155).[9] Liu, F., Li, Y., Li, B., Li, J., & Xie, H. (2021). Bitcoin transaction strategy construction based on deep reinforcement learning. Applied Soft Computing, 113, 107952.[10] Liu, X. Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. D. (2020). FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance. arXiv preprint arXiv:2011.09607.[11] Nelson, D. M., Pereira, A. C., & De Oliveira, R. A. (2017). Stock market`s price movement prediction with LSTM neural networks. In 2017 International joint conference on neural networks (IJCNN) (pp. 1419-1426).[12] Ozbayoglu, A. M., Gudelek, M. U., & Sezer, O. B. (2020). Deep learning for financial applications: A survey. Applied Soft Computing, 93, 106384.[13] Roondiwala, M., Patel, H., & Varma, S. (2017). Predicting stock prices using LSTM. International Journal of Science and Research (IJSR), 6(4) (pp. 1754-1756).[14] Solorio-Fernández, S., Carrasco-Ochoa, J. A., & Martínez-Trinidad, J. F. (2020). A review of unsupervised feature selection methods. Artificial Intelligence Review, 53(2) (pp. 907-948).[15] Statman, M. (1987). How many stocks make a diversified portfolio?. Journal of financial and quantitative analysis, 22(3) (pp. 353-363).[16] Understanding LSTM Networks. http://colah.github.io/posts/2015-08-Understanding-LSTMs. [Online; accessed 9-NOV-2022][17] Vargas, M. R., De Lima, B. S., & Evsukoff, A. G. (2017). Deep learning for stock market prediction from financial news articles. In 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA) (pp. 60-65)[18]Wang, B., & Zhang, X. Deep Learning Applying on Stock Trading. Stanford University.[19] Wang, J., Zhang, Y., Tang, K., Wu, J., & Xiong, Z. (2019). Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1900-1908).[20] Wang, Z., Huang, B., Tu, S., Zhang, K., & Xu, L. (2021). DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(1) (pp. 643-650).[21] What are neural networks? https://www.ibm.com/cloud/learn/neural-networks. [Online; accessed 9-NOV-2022][22] Wu, X., Chen, H., Wang, J., Troiano, L., Loia, V., & Fujita, H. (2020). Adaptive stock trading strategies with deep reinforcement learning methods. Information Sciences, 538 (pp. 142-158). 描述 碩士
國立政治大學
資訊科學系
110753114資料來源 http://thesis.lib.nccu.edu.tw/record/#G0110753114 資料類型 thesis dc.contributor.advisor 胡毓忠 zh_TW dc.contributor.advisor Hu, Yuh-Jong en_US dc.contributor.author (Authors) 張柏詠 zh_TW dc.contributor.author (Authors) Chang, Pai-Yung en_US dc.creator (作者) 張柏詠 zh_TW dc.creator (作者) Chang, Pai-Yung en_US dc.date (日期) 2023 en_US dc.date.accessioned 9-Mar-2023 18:37:46 (UTC+8) - dc.date.available 9-Mar-2023 18:37:46 (UTC+8) - dc.date.issued (上傳時間) 9-Mar-2023 18:37:46 (UTC+8) - dc.identifier (Other Identifiers) G0110753114 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/143837 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 110753114 zh_TW dc.description.abstract (摘要) 本文共有三個實驗命題，命題一，研究深度強化學習模型在不同市場情境下如何應變。命題二，比較深度強化學習資產配置模型與加入市場分數之資產配置模型之風險報酬。命題三，探討加入市場分數之資產配置模型與加入無風險資產之資產配置模型差異。以三命題探索不同市場趨勢與不同資產池對於深度強化學習資產配置模型之各類比較，命題一研究顯示深度強化學習資產配置模型在市場趨勢屬於恐慌時期與熊市時皆能有效改善風險報酬率；命題二研究成果顯示將資產配置模型拆成資產分數與市場分數兩部分，能在有效降低風險同時保有一定的獲利能力，風險報酬率更勝單純資產配置模型。命題三研究成果顯示加入市場分數與於資產池中加入無風險資產之資產模型各有優缺點，然長期來看加入市場分數較能在承受相同風險條件下追求更優的獲利率。三命題皆以風險與報酬指標來比較不同資產配置模型優劣，期望能建構出穩定獲利資產配置模型。 zh_TW dc.description.abstract (摘要) There are three experimental purposes in this paper. First, how will the DRL asset management model respond to different market trends？ Second, compare the risk-reward of the DRL asset management model with the DRL asset management model added market score. Third, discuss the difference between the asset management model adding market scores and the asset management model adding risk-free assets. Explore various comparisons of different market trends and different asset pools for the DRL asset management model with three propositions. The results revealed that the DRL model can effectively improve the risk-reward ratio during the great depression and bear market. The research results of proposition 2 show that splitting the asset management model into two parts, the asset score and the market score, can effectively reduce risks while maintaining certain profitability, and the risk-reward ratio is better than the traditional asset management model. The research results of proposition 3 show that adding market scores and adding risk-free assets to the asset pool have its own advantages and disadvantages. However, in the long term, adding market scores can pursue better profitability under the same risk conditions. en_US dc.description.tableofcontents 第一章 前言 1第一節 研究動機 1第二節 研究目的 1第三節 研究架構 2第二章 文獻探討 4第一節 時間序列預測文獻回顧 4第二節 資產配置文獻回顧 5第三章 研究方法論 8第一節 長短期記憶(LSTM)演算法介紹 8第二節 深度確定策略梯度(DDPG)演算法介紹 11第四章 系統設計 17第一節 系統概述 17第二節 資產分數系統 18第三節 市場分數系統 21第五章 研究實作 24第一節 資料集來源與週期 24第二節 特徵工程 24第三節 交易參數設定 27第四節 模型訓練 28第五節 模型成果評量 30第六章 結論與未來展望 41第一節 研究結論 41第二節 未來展望 43參考文獻 48 zh_TW dc.format.extent 3171378 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0110753114 en_US dc.subject (關鍵詞) 機器學習 zh_TW dc.subject (關鍵詞) 深度強化學習 zh_TW dc.subject (關鍵詞) 資產配置 zh_TW dc.subject (關鍵詞) 投資組合 zh_TW dc.subject (關鍵詞) DDPG zh_TW dc.subject (關鍵詞) LSTM zh_TW dc.subject (關鍵詞) Machine Learning en_US dc.subject (關鍵詞) Deep Reinforcement Learning en_US dc.subject (關鍵詞) Asset Allocation en_US dc.subject (關鍵詞) Investment portfolio en_US dc.subject (關鍵詞) DDPG en_US dc.subject (關鍵詞) LSTM en_US dc.title (題名) 基於深度強化學習來探索市場與資產因子於資產配置策略優化 zh_TW dc.title (題名) Exploring Portfolio Optimization Strategy with Market and Asset Factors in Deep Reinforcement Learning en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Chen, C., Zhao, L., Bian, J., Xing, C., & Liu, T. Y. (2019). Investment behaviors can tell what inside: Exploring stock intrinsic properties for stock trend prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 2376-2384)[2] Darapaneni, N., Basu, A., Savla, S., Gururajan, R., Saquib, N., Singhavi, S., ... & Paduri, A. R. (2020). Automated portfolio rebalancing using Q-learning. In 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (pp. 0596-0602).[3] Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2016). Deep direct reinforcement learning for financial signal representation and trading. IEEE transactions on neural networks and learning systems, 28(3) (pp.653-664).[4] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8) (pp. 1735-1780).[5] Hoseinzade, E., & Haratizadeh, S. (2019). CNNpred: CNN-based stock market prediction using a diverse set of variables. Expert Systems with Applications, 129 (pp. 273-285).[6] Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.[7] Jin, O., & El-Saawy, H. (2016). Portfolio management using reinforcement learning. Stanford University.[8] Lintner, J. (1975). The valuation of risk assets and the selection of risky investments in stock portfolios and capital budgets. In Stochastic optimization models in finance (pp. 131-155).[9] Liu, F., Li, Y., Li, B., Li, J., & Xie, H. (2021). Bitcoin transaction strategy construction based on deep reinforcement learning. Applied Soft Computing, 113, 107952.[10] Liu, X. Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. D. (2020). FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance. arXiv preprint arXiv:2011.09607.[11] Nelson, D. M., Pereira, A. C., & De Oliveira, R. A. (2017). Stock market`s price movement prediction with LSTM neural networks. In 2017 International joint conference on neural networks (IJCNN) (pp. 1419-1426).[12] Ozbayoglu, A. M., Gudelek, M. U., & Sezer, O. B. (2020). Deep learning for financial applications: A survey. Applied Soft Computing, 93, 106384.[13] Roondiwala, M., Patel, H., & Varma, S. (2017). Predicting stock prices using LSTM. International Journal of Science and Research (IJSR), 6(4) (pp. 1754-1756).[14] Solorio-Fernández, S., Carrasco-Ochoa, J. A., & Martínez-Trinidad, J. F. (2020). A review of unsupervised feature selection methods. Artificial Intelligence Review, 53(2) (pp. 907-948).[15] Statman, M. (1987). How many stocks make a diversified portfolio?. Journal of financial and quantitative analysis, 22(3) (pp. 353-363).[16] Understanding LSTM Networks. http://colah.github.io/posts/2015-08-Understanding-LSTMs. [Online; accessed 9-NOV-2022][17] Vargas, M. R., De Lima, B. S., & Evsukoff, A. G. (2017). Deep learning for stock market prediction from financial news articles. In 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA) (pp. 60-65)[18]Wang, B., & Zhang, X. Deep Learning Applying on Stock Trading. Stanford University.[19] Wang, J., Zhang, Y., Tang, K., Wu, J., & Xiong, Z. (2019). Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1900-1908).[20] Wang, Z., Huang, B., Tu, S., Zhang, K., & Xu, L. (2021). DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(1) (pp. 643-650).[21] What are neural networks? https://www.ibm.com/cloud/learn/neural-networks. [Online; accessed 9-NOV-2022][22] Wu, X., Chen, H., Wang, J., Troiano, L., Loia, V., & Fujita, H. (2020). Adaptive stock trading strategies with deep reinforcement learning methods. Information Sciences, 538 (pp. 142-158). zh_TW
