學術產出-Theses
Article View/Open
Publication Export
-
題名 應用深度強化學習將加密貨幣納入資產配置組合優化之探索
Applying Deep Reinforcement Learning to Explore Adding Cryptocurrencies in Asset Allocation Optimization作者 李昂縣
Lee, Ang-Hsien貢獻者 胡毓忠
Hu, Yuh-Jong
李昂縣
Lee, Ang-Hsien關鍵詞 深度強化學習
Soft Actor-Critic
加密貨幣
資產配置
現代投資組合理論
均值-變異數最佳化
Deep reinforcement learning
Soft Actor-Critic
Cryptocurrency
Asset portfolio
Modern Portfolio Theory
Mean-Variance Optimization日期 2023 上傳時間 1-Dec-2023 14:09:06 (UTC+8) 摘要 隨著近年來加密貨幣市場的蓬勃發展，越來越多的投資者將加密貨幣納入其投資組合，以期望獲得更高的回報率。相較於傳統交易市場，加密貨幣市場提供更透明的交易資訊，更靈活地反應市場變化，且不受休市時間的限制。然而，加密貨幣市場的高波動及流動性不足等特性，使得投資者通常對納入其投資組合持保守態度，或者將其與其他傳統資產結合以對沖風險。 深度強化學習在複雜環境中的自我學習和適應能力、對高維度數據的有效處理，以及良好的泛化性能，有望在加密貨幣投資決策中發揮顯著的優勢。因此本研究的目的在於應用Soft Actor Critic深度強化學習演算法，整合加密貨幣風險評估並探討加密貨幣納入資產配置組合的效益。透過實證分析，我們比較了加密貨幣資產配置組合與傳統金融資產配置組合的成效，以及DRL模型與現代投資組合理論的均值-變異數最佳化模型在加密貨幣資產配置優化的效益表現，為投資者提供具參考價值且系統性的投資建議。
With the rapid growth of the cryptocurrency market, an increasing number of investors are incorporating cryptocurrencies into their portfolios in pursuit of potential high returns. Compared to traditional markets, cryptocurrencies provide heightened transparency and flexibility, operating continuously around the clock. However, due to their unpredictable nature, some investors may approach them cautiously or combine them with traditional assets for risk management. Deep Reinforcement Learning (DRL) excels in navigating complex environments, handling high-dimensional data, and demonstrating robust adaptability. This study aims to integrate cryptocurrency risk assessment using the Soft Actor-Critic algorithm and explore the benefits of incorporating cryptocurrencies into asset allocation while determining suitable allocation ratios. By comparing the performance of cryptocurrency and traditional asset portfolios and evaluating the effectiveness of this research model, we aim to provide valuable and systematic investment guidance.參考文獻 [1] Hamid Ali, Hammad Majeed, Imran Usman, and Khaled A Almejalli. Reducing entropy overestimation in soft actor critic using dual policy network. Wireless Communications and Mobile Computing, 2021:1– 13, 2021. [2] Lennart Ante. How elon musk’s twitter activity moves cryptocurrency markets. Technological Forecasting and Social Change, 186:122112, 2023. [3] Amirreza Attarzadeh and Mehmet Balcilar. On the dynamic return and volatility connectedness of cryptocurrency, crude oil, clean energy, and stock markets: a time-varying analysis. Environmental Science and Pollution Research, 29(43):65185–65196, 2022. [4] Golnoosh Babaei, Paolo Giudici, and Emanuela Raffinetti. Explain- able artificial intelligence for crypto asset allocation. Finance Research Letters, 47:102941, 2022. [5] Manjula BC, Shilpa BS, et al. Analysis of cryptocurrency, bitcoin and the future. East Asian Journal of Multidisciplinary Research, 1(7):1293– 1302, 2022. [6] Eric Benhamou, David Saltiel, Jean Jacques Ohana, Jamal Atif, and Rida Laraki. Deep reinforcement learning (drl) for portfolio allocation. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part V, pages 527–531. Springer, 2021. [7] Eric Benhamou, David Saltiel, Sandrine Ungari, and Abhishek Mukhopadhyay. Bridging the gap between markowitz planning and deep reinforcement learning. arXiv preprint arXiv:2010.09108, 2020. [8] Elie Bouri, Peter Molnár, Georges Azzi, David Roubaud, and Lars Ivar Hagfors. On the hedge and safe haven properties of bitcoin: Is it really more than a diversifier? Finance Research Letters, 20:192–198, 2017. [9] Gerard Cornuejols and Reha Tütüncü. Optimization methods in finance, volume 5. Cambridge University Press, 2006. [10] Kwamie Dunbar and Johnson Owusu-Amoako. Cryptocurrency re- turns under empirical asset pricing. International Review of Financial Analysis, 82:102216, 2022. [11] Ricard Durall. Asset allocation: From markowitz to deep reinforcement learning. arXiv preprint arXiv:2208.07158, 2022. [12] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018. [13] Miquel Noguer i Alonso, Sonam Srivastava, et al. Deep reinforcement learning for asset allocation in us equities. Technical report, 2020. [14] Damian Kisiel and Denise Gorse. Portfolio transformer for attention- based asset allocation. In International Conference on Artificial Intelligence and Soft Computing, pages 61–71. Springer, 2022. [15] Petter N Kolm, Reha Tütüncü, and Frank J Fabozzi. 60 years of portfolio optimization: Practical challenges and current trends. European Journal of Operational Research, 234(2):356–371, 2014. [16] Edmond Lezmi and Jiali Xu. Time series forecasting with transformer models and application to asset management. Available at SSRN 4375798, 2023. [17] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Con- tinuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015. [18] Xiao-Yang Liu, Jingyang Rui, Jiechao Gao, Liuqing Yang, Hongyang Yang, Zhaoran Wang, Christina Dan Wang, and Guo Jian. FinRL- Meta: Data-driven deep reinforcementlearning in quantitative finance. Data-Centric AI Workshop, NeurIPS, 2021. [19] Xiao-Yang Liu, Ziyi Xia, Jingyang Rui, Jiechao Gao, Hongyang Yang, Ming Zhu, Christina Dan Wang, Zhaoran Wang, and Jian Guo. FinRL- Meta: Market environments and benchmarks for data-driven financial reinforcement learning. NeurIPS, 2022. [20] Xiao-Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. Finrl: Deep reinforcement learning framework to automate trading in quantitative finance. In Proceedings of the second ACM international conference on AI in finance, pages 1–9, 2021. [21] Yukun Liu and Aleh Tsyvinski. Risks and returns of cryptocurrency. The Review of Financial Studies, 34(6):2689–2727, 2021. [22] Luis Lorenzo and Javier Arroyo. Online risk-based portfolio allocation on subsets of crypto assets applying a prototype-based clustering algo- rithm. Financial Innovation, 9(1):25, 2023. [23] Yue Ma, Ziping Liu, and Chuck McAllister. Deep reinforcement learn- ing for portfolio management. In Yan Shi, Gongzhu Hu, Krishna Kamb- hampaty, and Takaaki Goto, editors, Proceedings of 35th International Conference on Computer Applications in Industry and Engineering, vol- ume 89 of EPiC Series in Computing, pages 41–51. EasyChair, 2022. [24] Leonard C MacLean, Edward O Thorp, and William T Ziemba. The Kelly capital growth investment criterion: Theory and practice, vol- ume 3. world scientific, 2011. [25] Iwao Maeda, David DeGraw, Michiharu Kitano, Hiroyasu Matsushima, Hiroki Sakaji, Kiyoshi Izumi, and Atsuo Kato. Deep reinforcement learning in agent based financial market simulation. Journal of Risk and Financial Management, 13(4):71, 2020. [26] H Markowitz. The utility of wealth, journal of political economy, vol. 60. 1952. [27] Akib Mashrur, Wei Luo, Nayyar A Zaidi, and Antonio Robles-Kelly. Machine learning for financial risk management: A survey. IEEE Access, 8:203203–203223, 2020. [28] Adebayo Oshingbesan, Eniola Ajiboye, Peruth Kamashazi, and Tim- othy Mbaka. Model-free reinforcement learning for asset allocation. arXiv preprint arXiv:2209.10458, 2022. [29] Sebastian Palmquist and Janis Mednis. Portfolio optimization with crypto assets:analyzing the impact of the investors’ subjective views on portfolio risk, 2022. [30] Alla Petukhina and Erin Sprünken. Evaluation of multi-asset investment strategies with digital assets. Digital Finance, 3:45–79, 2021. [31] Emmanouil Platanakis and Andrew Urquhart. Portfolio management with cryptocurrencies: The role of estimation risk. Economics Letters, 177:76–80, 2019. [32] Emmanouil Platanakis and Andrew Urquhart. Should investors include bitcoin in their portfolios? a portfolio theory approach. The British accounting review, 52(4):100837, 2020. [33] Refk Selmi, Walid Mensi, Shawkat Hammoudeh, and Jamal Bouoiyour. Is bitcoin a hedge, a safe haven or a diversifier for oil price movements? a comparison with gold. Energy Economics, 74:787–801, 2018. [34] Efthymia Symitsi and Konstantinos J Chalvatzis. The economic value of bitcoin: A portfolio analysis of currencies, gold, oil and stocks. Research in International Business and Finance, 48:97–110, 2019. [35] United States Department of the Treasury. Crypto-assets: Implications for consumers, investors, and businesses. Publication by the United States Department of the Treasury, September 2022. 描述 碩士
國立政治大學
資訊科學系碩士在職專班
110971018資料來源 http://thesis.lib.nccu.edu.tw/record/#G0110971018 資料類型 thesis dc.contributor.advisor 胡毓忠 zh_TW dc.contributor.advisor Hu, Yuh-Jong en_US dc.contributor.author (Authors) 李昂縣 zh_TW dc.contributor.author (Authors) Lee, Ang-Hsien en_US dc.creator (作者) 李昂縣 zh_TW dc.creator (作者) Lee, Ang-Hsien en_US dc.date (日期) 2023 en_US dc.date.accessioned 1-Dec-2023 14:09:06 (UTC+8) - dc.date.available 1-Dec-2023 14:09:06 (UTC+8) - dc.date.issued (上傳時間) 1-Dec-2023 14:09:06 (UTC+8) - dc.identifier (Other Identifiers) G0110971018 en_US dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/148551 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系碩士在職專班 zh_TW dc.description (描述) 110971018 zh_TW dc.description.abstract (摘要) 隨著近年來加密貨幣市場的蓬勃發展，越來越多的投資者將加密貨幣納入其投資組合，以期望獲得更高的回報率。相較於傳統交易市場，加密貨幣市場提供更透明的交易資訊，更靈活地反應市場變化，且不受休市時間的限制。然而，加密貨幣市場的高波動及流動性不足等特性，使得投資者通常對納入其投資組合持保守態度，或者將其與其他傳統資產結合以對沖風險。 深度強化學習在複雜環境中的自我學習和適應能力、對高維度數據的有效處理，以及良好的泛化性能，有望在加密貨幣投資決策中發揮顯著的優勢。因此本研究的目的在於應用Soft Actor Critic深度強化學習演算法，整合加密貨幣風險評估並探討加密貨幣納入資產配置組合的效益。透過實證分析，我們比較了加密貨幣資產配置組合與傳統金融資產配置組合的成效，以及DRL模型與現代投資組合理論的均值-變異數最佳化模型在加密貨幣資產配置優化的效益表現，為投資者提供具參考價值且系統性的投資建議。 zh_TW dc.description.abstract (摘要) With the rapid growth of the cryptocurrency market, an increasing number of investors are incorporating cryptocurrencies into their portfolios in pursuit of potential high returns. Compared to traditional markets, cryptocurrencies provide heightened transparency and flexibility, operating continuously around the clock. However, due to their unpredictable nature, some investors may approach them cautiously or combine them with traditional assets for risk management. Deep Reinforcement Learning (DRL) excels in navigating complex environments, handling high-dimensional data, and demonstrating robust adaptability. This study aims to integrate cryptocurrency risk assessment using the Soft Actor-Critic algorithm and explore the benefits of incorporating cryptocurrencies into asset allocation while determining suitable allocation ratios. By comparing the performance of cryptocurrency and traditional asset portfolios and evaluating the effectiveness of this research model, we aim to provide valuable and systematic investment guidance. en_US dc.description.tableofcontents 第一章 緒論 1 1.1 研究動機 1 1.2 研究目的 2 1.3 研究架構 2 第二章 文獻探討 4 2.1 現代投資組合理論 5 2.1.1 均值-變異數最佳化 6 2.2 深度強化學習 7 2.2.1 Soft Actor-Critic 9 2.3 加密貨幣 11 第三章 相關研究 13 3.1 DRL 模型應用在金融領域資產配置 13 3.2 加密貨幣加入資產配置 15 3.3 研究貢獻 17 第四章 研究方法 19 4.1 實驗設計 19 4.2 實驗定義 21 4.3 實驗流程 28 第五章 實驗結果 33 5.1 資料蒐集 33 5.2 模型建立 36 5.3 模型測試 38 5.4 成果評量 41 第六章 結論及未來展望 44 6.1 結論 44 6.2 未來展望 47 參考文獻 48 zh_TW dc.format.extent 5862447 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0110971018 en_US dc.subject (關鍵詞) 深度強化學習 zh_TW dc.subject (關鍵詞) Soft Actor-Critic zh_TW dc.subject (關鍵詞) 加密貨幣 zh_TW dc.subject (關鍵詞) 資產配置 zh_TW dc.subject (關鍵詞) 現代投資組合理論 zh_TW dc.subject (關鍵詞) 均值-變異數最佳化 zh_TW dc.subject (關鍵詞) Deep reinforcement learning en_US dc.subject (關鍵詞) Soft Actor-Critic en_US dc.subject (關鍵詞) Cryptocurrency en_US dc.subject (關鍵詞) Asset portfolio en_US dc.subject (關鍵詞) Modern Portfolio Theory en_US dc.subject (關鍵詞) Mean-Variance Optimization en_US dc.title (題名) 應用深度強化學習將加密貨幣納入資產配置組合優化之探索 zh_TW dc.title (題名) Applying Deep Reinforcement Learning to Explore Adding Cryptocurrencies in Asset Allocation Optimization en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Hamid Ali, Hammad Majeed, Imran Usman, and Khaled A Almejalli. Reducing entropy overestimation in soft actor critic using dual policy network. Wireless Communications and Mobile Computing, 2021:1– 13, 2021. [2] Lennart Ante. How elon musk’s twitter activity moves cryptocurrency markets. Technological Forecasting and Social Change, 186:122112, 2023. [3] Amirreza Attarzadeh and Mehmet Balcilar. On the dynamic return and volatility connectedness of cryptocurrency, crude oil, clean energy, and stock markets: a time-varying analysis. Environmental Science and Pollution Research, 29(43):65185–65196, 2022. [4] Golnoosh Babaei, Paolo Giudici, and Emanuela Raffinetti. Explain- able artificial intelligence for crypto asset allocation. Finance Research Letters, 47:102941, 2022. [5] Manjula BC, Shilpa BS, et al. Analysis of cryptocurrency, bitcoin and the future. East Asian Journal of Multidisciplinary Research, 1(7):1293– 1302, 2022. [6] Eric Benhamou, David Saltiel, Jean Jacques Ohana, Jamal Atif, and Rida Laraki. Deep reinforcement learning (drl) for portfolio allocation. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part V, pages 527–531. Springer, 2021. [7] Eric Benhamou, David Saltiel, Sandrine Ungari, and Abhishek Mukhopadhyay. Bridging the gap between markowitz planning and deep reinforcement learning. arXiv preprint arXiv:2010.09108, 2020. [8] Elie Bouri, Peter Molnár, Georges Azzi, David Roubaud, and Lars Ivar Hagfors. On the hedge and safe haven properties of bitcoin: Is it really more than a diversifier? Finance Research Letters, 20:192–198, 2017. [9] Gerard Cornuejols and Reha Tütüncü. Optimization methods in finance, volume 5. Cambridge University Press, 2006. [10] Kwamie Dunbar and Johnson Owusu-Amoako. Cryptocurrency re- turns under empirical asset pricing. International Review of Financial Analysis, 82:102216, 2022. [11] Ricard Durall. Asset allocation: From markowitz to deep reinforcement learning. arXiv preprint arXiv:2208.07158, 2022. [12] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018. [13] Miquel Noguer i Alonso, Sonam Srivastava, et al. Deep reinforcement learning for asset allocation in us equities. Technical report, 2020. [14] Damian Kisiel and Denise Gorse. Portfolio transformer for attention- based asset allocation. In International Conference on Artificial Intelligence and Soft Computing, pages 61–71. Springer, 2022. [15] Petter N Kolm, Reha Tütüncü, and Frank J Fabozzi. 60 years of portfolio optimization: Practical challenges and current trends. European Journal of Operational Research, 234(2):356–371, 2014. [16] Edmond Lezmi and Jiali Xu. Time series forecasting with transformer models and application to asset management. Available at SSRN 4375798, 2023. [17] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Con- tinuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015. [18] Xiao-Yang Liu, Jingyang Rui, Jiechao Gao, Liuqing Yang, Hongyang Yang, Zhaoran Wang, Christina Dan Wang, and Guo Jian. FinRL- Meta: Data-driven deep reinforcementlearning in quantitative finance. Data-Centric AI Workshop, NeurIPS, 2021. [19] Xiao-Yang Liu, Ziyi Xia, Jingyang Rui, Jiechao Gao, Hongyang Yang, Ming Zhu, Christina Dan Wang, Zhaoran Wang, and Jian Guo. FinRL- Meta: Market environments and benchmarks for data-driven financial reinforcement learning. NeurIPS, 2022. [20] Xiao-Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. Finrl: Deep reinforcement learning framework to automate trading in quantitative finance. In Proceedings of the second ACM international conference on AI in finance, pages 1–9, 2021. [21] Yukun Liu and Aleh Tsyvinski. Risks and returns of cryptocurrency. The Review of Financial Studies, 34(6):2689–2727, 2021. [22] Luis Lorenzo and Javier Arroyo. Online risk-based portfolio allocation on subsets of crypto assets applying a prototype-based clustering algo- rithm. Financial Innovation, 9(1):25, 2023. [23] Yue Ma, Ziping Liu, and Chuck McAllister. Deep reinforcement learn- ing for portfolio management. In Yan Shi, Gongzhu Hu, Krishna Kamb- hampaty, and Takaaki Goto, editors, Proceedings of 35th International Conference on Computer Applications in Industry and Engineering, vol- ume 89 of EPiC Series in Computing, pages 41–51. EasyChair, 2022. [24] Leonard C MacLean, Edward O Thorp, and William T Ziemba. The Kelly capital growth investment criterion: Theory and practice, vol- ume 3. world scientific, 2011. [25] Iwao Maeda, David DeGraw, Michiharu Kitano, Hiroyasu Matsushima, Hiroki Sakaji, Kiyoshi Izumi, and Atsuo Kato. Deep reinforcement learning in agent based financial market simulation. Journal of Risk and Financial Management, 13(4):71, 2020. [26] H Markowitz. The utility of wealth, journal of political economy, vol. 60. 1952. [27] Akib Mashrur, Wei Luo, Nayyar A Zaidi, and Antonio Robles-Kelly. Machine learning for financial risk management: A survey. IEEE Access, 8:203203–203223, 2020. [28] Adebayo Oshingbesan, Eniola Ajiboye, Peruth Kamashazi, and Tim- othy Mbaka. Model-free reinforcement learning for asset allocation. arXiv preprint arXiv:2209.10458, 2022. [29] Sebastian Palmquist and Janis Mednis. Portfolio optimization with crypto assets:analyzing the impact of the investors’ subjective views on portfolio risk, 2022. [30] Alla Petukhina and Erin Sprünken. Evaluation of multi-asset investment strategies with digital assets. Digital Finance, 3:45–79, 2021. [31] Emmanouil Platanakis and Andrew Urquhart. Portfolio management with cryptocurrencies: The role of estimation risk. Economics Letters, 177:76–80, 2019. [32] Emmanouil Platanakis and Andrew Urquhart. Should investors include bitcoin in their portfolios? a portfolio theory approach. The British accounting review, 52(4):100837, 2020. [33] Refk Selmi, Walid Mensi, Shawkat Hammoudeh, and Jamal Bouoiyour. Is bitcoin a hedge, a safe haven or a diversifier for oil price movements? a comparison with gold. Energy Economics, 74:787–801, 2018. [34] Efthymia Symitsi and Konstantinos J Chalvatzis. The economic value of bitcoin: A portfolio analysis of currencies, gold, oil and stocks. Research in International Business and Finance, 48:97–110, 2019. [35] United States Department of the Treasury. Crypto-assets: Implications for consumers, investors, and businesses. Publication by the United States Department of the Treasury, September 2022. zh_TW
