Publications-Theses
Article View/Open
Publication Export
-
題名 利用深度強化式學習建構價差交易策略:以台指期與摩台期為例
The Construction of TX-TW Pair Trading Strategies based on Deep Reinforcement Learning作者 許晏寧
Hsu, Yen-Ning貢獻者 江彌修
Chiang, Mi-Hsiu
許晏寧
Hsu, Yen-Ning關鍵詞 價差交易
強化式學習
類神經網路
台股期貨
摩台期貨
Pairs trading
Reinforcement learning
Neural network
Taiwan Stock Index Futures
MSCI Taiwan Index Futures日期 2019 上傳時間 1-Jul-2019 10:46:58 (UTC+8) 摘要 本研究使用三種基於模型的深度強化式學習DQN、Double DQN和Dueling DQN來建構價差交易策略，本研究會選擇開發此類交易策略，主要是因為深度強化式學習的獎勵機制和建構交易策略有很好的對應性且價差交易策略能夠有效的減少市場風險。本研究採用2006/01/01至2018/11/16的台股期貨和摩台期貨進行回測，並設計隨機策略、固定策略當作基準策略，實證結果發現深度強化式學習均可以獲得比基準策略更好的表現，而整體上DQN表現勝過Double DQN和Dueling DQN。但細看可以發現，在不同的回測期間，三種深度強化式學習分別有其表現最好的時候，代表此三種模型分別學到不一樣的規則，此規則在不同的時期有不一樣的適用性。
In this paper, we implement three model-based reinforcement learning algorithms with deep learning, Deep Q-Learning Network (DQN), Double Deep Q-learning Network (Double DQN) and Dueling Deep Q-Learning Network (Dueling DQN) in pair trading strategy. In addition, deep reinforcement learning (DRL) has appealing theoretical properties which are hopefully potential since the reward mechanism in DRL with pair trading rules is able to significantly reduce the market risk. We conduct experiments in TX and TW historical data (2006/01/01-2018/11-16) and design the random strategy and fixed strategy to be the benchmark. The empirical results show that three DRL strategies can achieve better performance than the benchmark strategies overall and DQN is more desirable than Double DQN and Dueling DQN. However, during different back-testing period, we observe that they have the best performance respectively. It means that three models learn different rules separately and the rules have different applicability in different periods.參考文獻 [1] Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003.[2] Binh H. D. & Robert W. F. (2012). Are Pairs Trading Profits Robust to Trading Costs? The Journal of Financial Research, 35(2), 261-287.[3] Chien Y. H. (2018). Financial Trading as a Game: A Deep Reinforcement Learning Approach. arXiv preprint arXiv:1807.02787[4] Evan G. , William N. G., & K. G. R. (2006). Pairs Trading: Performance of a Relative-Value Arbitrage Rule. The Review of Financial Studies, 19(3), 797-827.[5] Gold C. (2003), FX trading via recurrent Reinforcement Learning, Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, 363-370.[6] Hado van H., Arthur G., & David S. (2015). Deep Reinforcement Learning with Double Q-learning. arXiv preprint arXiv:1509.06461[7] Jae W. L., Jonghun P., O J., Jongwoo L., & Euyseok H. (2007). A Multiagent Approach to Q-Learning for Daily Stock Trading. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 37(6), 864 – 877.[8] Kearns M., Nevmyvaka Y. (2013) Machine learning for market microstructure and high frequency trading. In: Easley D., López de Prado M., O’Hara M. (Eds.) High-Frequency Trading – New Realities for Traders, Markets and Regulators, 91-124.[9] Moody J., Saffel M. (2001), Learning to trade via Direct Reinforcement, IEEE Transactions on Neural Network, 12, 875-889.[10] Moody J., Wu L., Liao Y., Saffel M. (1998), Performance functions and Reinforcement Learning for trading systems and portfolios, Journal of Forecasting, 17 (56), 441-470.[11] Moody, J. & Wu, L. (1997), Optimization of trading systems and portfolios, in Y. Abu-Mostafa, A. N. Refenes & A. S. Weigend, eds, `Decision Technologies for Financial Engineering`, World Scientific, London, 23-35.[12] O J., Lee J., Lee, J.W., Zhang, B.-T. (2006) Adaptive stock trading with dynamic asset allocation using reinforcement learning, Information Sciences, 176 (15), 2121-2147.[13] Richard S. S. and Andrew G. B. (1998) Reinforcement Learning: An Introduction. MIT Press.[14] Sergey I., Christian S. (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv preprint arXiv:1502.03167[15] Vidyamurthy, G. (2004). Pairs Trading: quantitative methods and analysis (Vol. 217). John Wiley & Sons.[16] Volodymyr M., Koray K., David S., Alex G., Ioannis A., Daan W., Martin R., (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.[17] Volodymyr M., Koray K., David S., Andrei A. R., Joel V., Marc G. B., Alex G., Martin R., Andreas K. F., Georg O., et al. Human-level control through deep reinforcement learning. Nature 518(7540): 529–533, 201.[18] Yuriy N., Yi F., & Michael K. (2006) Reinforcement Learning for Optimized Trade Execution. In Proceedings of the 23rd international conference on Machine learning. 673-680.[19] Ziyu W., Tom S., Matteo H., Hado van H., Marc L., & Nando de F.(2015). Dueling Network Architectures for Deep Reinforcement Learning. arXiv preprint arXiv:1511.06581 描述 碩士
國立政治大學
金融學系
106352004資料來源 http://thesis.lib.nccu.edu.tw/record/#G0106352004 資料類型 thesis dc.contributor.advisor 江彌修 zh_TW dc.contributor.advisor Chiang, Mi-Hsiu en_US dc.contributor.author (Authors) 許晏寧 zh_TW dc.contributor.author (Authors) Hsu, Yen-Ning en_US dc.creator (作者) 許晏寧 zh_TW dc.creator (作者) Hsu, Yen-Ning en_US dc.date (日期) 2019 en_US dc.date.accessioned 1-Jul-2019 10:46:58 (UTC+8) - dc.date.available 1-Jul-2019 10:46:58 (UTC+8) - dc.date.issued (上傳時間) 1-Jul-2019 10:46:58 (UTC+8) - dc.identifier (Other Identifiers) G0106352004 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/124137 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 金融學系 zh_TW dc.description (描述) 106352004 zh_TW dc.description.abstract (摘要) 本研究使用三種基於模型的深度強化式學習DQN、Double DQN和Dueling DQN來建構價差交易策略，本研究會選擇開發此類交易策略，主要是因為深度強化式學習的獎勵機制和建構交易策略有很好的對應性且價差交易策略能夠有效的減少市場風險。本研究採用2006/01/01至2018/11/16的台股期貨和摩台期貨進行回測，並設計隨機策略、固定策略當作基準策略，實證結果發現深度強化式學習均可以獲得比基準策略更好的表現，而整體上DQN表現勝過Double DQN和Dueling DQN。但細看可以發現，在不同的回測期間，三種深度強化式學習分別有其表現最好的時候，代表此三種模型分別學到不一樣的規則，此規則在不同的時期有不一樣的適用性。 zh_TW dc.description.abstract (摘要) In this paper, we implement three model-based reinforcement learning algorithms with deep learning, Deep Q-Learning Network (DQN), Double Deep Q-learning Network (Double DQN) and Dueling Deep Q-Learning Network (Dueling DQN) in pair trading strategy. In addition, deep reinforcement learning (DRL) has appealing theoretical properties which are hopefully potential since the reward mechanism in DRL with pair trading rules is able to significantly reduce the market risk. We conduct experiments in TX and TW historical data (2006/01/01-2018/11-16) and design the random strategy and fixed strategy to be the benchmark. The empirical results show that three DRL strategies can achieve better performance than the benchmark strategies overall and DQN is more desirable than Double DQN and Dueling DQN. However, during different back-testing period, we observe that they have the best performance respectively. It means that three models learn different rules separately and the rules have different applicability in different periods. en_US dc.description.tableofcontents 第一章 緒論 1第二章 文獻探討 4第一節 強化式學習的相關應用 4第二節 價差交易策略 4第三章 研究方法 6第一節 研究對象與選用資料 6第二節 強化式學習 13第三節 深度強化式學習 17第四節 模型與回測設計 21第四章 實證結果 28第一節 隨機策略和固定策略 28第二節 深度強化學習策略 38第三節 績效總結 47第五章 結論與建議 51第一節 結論 51第二節 未來策略建議 51參考文獻 53 zh_TW dc.format.extent 2909136 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0106352004 en_US dc.subject (關鍵詞) 價差交易 zh_TW dc.subject (關鍵詞) 強化式學習 zh_TW dc.subject (關鍵詞) 類神經網路 zh_TW dc.subject (關鍵詞) 台股期貨 zh_TW dc.subject (關鍵詞) 摩台期貨 zh_TW dc.subject (關鍵詞) Pairs trading en_US dc.subject (關鍵詞) Reinforcement learning en_US dc.subject (關鍵詞) Neural network en_US dc.subject (關鍵詞) Taiwan Stock Index Futures en_US dc.subject (關鍵詞) MSCI Taiwan Index Futures en_US dc.title (題名) 利用深度強化式學習建構價差交易策略:以台指期與摩台期為例 zh_TW dc.title (題名) The Construction of TX-TW Pair Trading Strategies based on Deep Reinforcement Learning en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003.[2] Binh H. D. & Robert W. F. (2012). Are Pairs Trading Profits Robust to Trading Costs? The Journal of Financial Research, 35(2), 261-287.[3] Chien Y. H. (2018). Financial Trading as a Game: A Deep Reinforcement Learning Approach. arXiv preprint arXiv:1807.02787[4] Evan G. , William N. G., & K. G. R. (2006). Pairs Trading: Performance of a Relative-Value Arbitrage Rule. The Review of Financial Studies, 19(3), 797-827.[5] Gold C. (2003), FX trading via recurrent Reinforcement Learning, Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, 363-370.[6] Hado van H., Arthur G., & David S. (2015). Deep Reinforcement Learning with Double Q-learning. arXiv preprint arXiv:1509.06461[7] Jae W. L., Jonghun P., O J., Jongwoo L., & Euyseok H. (2007). A Multiagent Approach to Q-Learning for Daily Stock Trading. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 37(6), 864 – 877.[8] Kearns M., Nevmyvaka Y. (2013) Machine learning for market microstructure and high frequency trading. In: Easley D., López de Prado M., O’Hara M. (Eds.) High-Frequency Trading – New Realities for Traders, Markets and Regulators, 91-124.[9] Moody J., Saffel M. (2001), Learning to trade via Direct Reinforcement, IEEE Transactions on Neural Network, 12, 875-889.[10] Moody J., Wu L., Liao Y., Saffel M. (1998), Performance functions and Reinforcement Learning for trading systems and portfolios, Journal of Forecasting, 17 (56), 441-470.[11] Moody, J. & Wu, L. (1997), Optimization of trading systems and portfolios, in Y. Abu-Mostafa, A. N. Refenes & A. S. Weigend, eds, `Decision Technologies for Financial Engineering`, World Scientific, London, 23-35.[12] O J., Lee J., Lee, J.W., Zhang, B.-T. (2006) Adaptive stock trading with dynamic asset allocation using reinforcement learning, Information Sciences, 176 (15), 2121-2147.[13] Richard S. S. and Andrew G. B. (1998) Reinforcement Learning: An Introduction. MIT Press.[14] Sergey I., Christian S. (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv preprint arXiv:1502.03167[15] Vidyamurthy, G. (2004). Pairs Trading: quantitative methods and analysis (Vol. 217). John Wiley & Sons.[16] Volodymyr M., Koray K., David S., Alex G., Ioannis A., Daan W., Martin R., (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.[17] Volodymyr M., Koray K., David S., Andrei A. R., Joel V., Marc G. B., Alex G., Martin R., Andreas K. F., Georg O., et al. Human-level control through deep reinforcement learning. Nature 518(7540): 529–533, 201.[18] Yuriy N., Yi F., & Michael K. (2006) Reinforcement Learning for Optimized Trade Execution. In Proceedings of the 23rd international conference on Machine learning. 673-680.[19] Ziyu W., Tom S., Matteo H., Hado van H., Marc L., & Nando de F.(2015). Dueling Network Architectures for Deep Reinforcement Learning. arXiv preprint arXiv:1511.06581 zh_TW dc.identifier.doi (DOI) 10.6814/NCCU201900104 en_US
