學術產出-Theses
Article View/Open
Publication Export
-
題名 深度強化學習之模型比較: 以股票自動交易系統為例
A Comparison of Deep Reinforcement Learning Models: The Case of Stock Automated Trading System作者 黃瑜萍
Huang, Yu-Ping貢獻者 蔡炎龍<br>蕭明福
Tsai, Yen-Lung<br>Shaw, Ming-Fu
黃瑜萍
Huang, Yu-Ping關鍵詞 深度學習
強化學習
深度 Q 學習
匯率
股票交易
Deep learning
Reinforcement learning
Deep Q learning
Exchange rate
Stock trading日期 2021 上傳時間 4-Aug-2021 15:58:49 (UTC+8) 摘要 本研究引入深度 Q 學習方法，建構一個自動化股票交易系統，研究範圍包含台灣股票市場 14 家科技業公司。研究期間為 2016 年 1 月 4 日至 2020年 12 月 31 日。本研究數據資料有兩種型態 (1) 股票資訊，(2) 股票資訊加上匯率參數。我們將深度 Q 學習的模型，與不同模型和其他策略相比較，以檢測深度 Q 學習是否更適用於股票交易。實證結果發現支持向量機與神經網路在實務面上難以進行股票交易操作，而深度 Q 學習的模型則具有相對好的成效。尤其，加入匯率參數的深度 Q 學習，獲得的報酬皆優於買入持有策略和台灣加權股價指數。
This research introduces the Deep Q learning model to construct an automated stock trading system. Our samples are 14 Taiwanese technology companies. Specifically, we include two types of data, (1) stock information and (2) stock information and exchange rate parameters, which are collected from the Taiwan stock market. The sampling period is from Jan 4, 2016 to Dec 31, 2020. We compare our main model, Deep Q learning, with different models and strategies to examine whether Deep Q learning is more applicable to stock trading. The empirical results show that it is difficult for Support vector machines and Neural networks to operate stock trading; however, Deep Q learning demonstrates better performance. In particular, the return rate of the Deep Q learning model is higher than the Buy-and-hold strategy and Taiwan Weighted Stock Index if considering exchange rate parameters.參考文獻 王泓仁. 台幣匯率對我國經濟金融活動之影響. 中央銀行季刊》,(),–, Wang, Hung-Jen (),“e Impacts of NT Dollar Exchange Rates on Taiwan＇s Economy＂, Quarterly Reviews, Central Bank of the Republic of China (Taiwan),(), 2005.徐千婷. 匯率與總體經濟變數之關係: 台灣實證分析, 2006.Wei Bao, Jun Yue, and Yulei Rao. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PloS one, 12(7):e0180944, 2017.Alessio Brini and Daniele Tantari. Deep reinforcement trading with predictable returns. arXiv preprint arXiv:2104.14683, 2021.Arthur Charpentier, Romuald Elie, and Carl Remlinger.Reinforcement learning in economics and finance. Computational Economics, pages 1–38, 2021.Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3): 273–297, 1995.Li Deng and Dong Yu. Deep learning: methods and applications. Foundations and trends in signal processing, 7(3–4):197–387, 2014.Eugene F Fama. The behavior of stock-market prices. The journal of Business, 38(1): 34–105, 1965.Eugene F Fama. Random walks in stock market prices. Financial analysts journal, 51(1): 75–80, 1995.Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.Chien Yi Huang. Financial trading as a game: A deep reinforcement learning approach. arXiv preprint arXiv:1807.02787, 2018.Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285, 1996.Konstantina Kourou, Themis P Exarchos, Konstantinos P Exarchos, Michalis V Karamouzis, and Dimitrios I Fotiadis. Machine learning applications in cancer prognosis and prediction. Computational and structural biotechnology journal, 13:8–17, 2015.Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553): 436–444, 2015.Jae Won Lee. Stock price prediction using reinforcement learning. In ISIE 2001. 2001 IEEE International Symposium on Industrial Electronics Proceedings (Cat. No. 01TH8570), volume 1, pages 690–695. IEEE, 2001.Jae Won Lee, Jonghun Park, O Jangmin, Jongwoo Lee, and Euyseok Hong. A multiagentapproach to q-learning for daily stock trading. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 37(6):864–877, 2007.Pierre Menard, Omar Darwiche Domingues, Xuedong Shang, and Michal Valko. Ucb momentum q-learning: Correcting the bias without forgetting. arXiv preprint arXiv: 2103.01312, 2021.Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv: 1507.04296, 2015.Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. Multimodal deep learning. In ICML, 2011.Kate Phylaktis and Fabiola Ravazzolo. Stock prices and exchange rate dynamics. Journal of international Money and Finance, 24(7):1031–1053, 2005.G William Schwert. Business cycles, financial crises, and stock volatility. In CarnegieRochester Conference series on public policy, volume 31, pages 83–125. ERichard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522, 2018.Paul D Yoo, Maria H Kim, and Tony Jan. Machine learning techniques and use of event information for stock market prediction: A survey and evaluation. In International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06), volume 2, pages 835–841. IEEE, 2005. 描述 碩士
國立政治大學
經濟學系
108258021資料來源 http://thesis.lib.nccu.edu.tw/record/#G0108258021 資料類型 thesis dc.contributor.advisor 蔡炎龍<br>蕭明福 zh_TW dc.contributor.advisor Tsai, Yen-Lung<br>Shaw, Ming-Fu en_US dc.contributor.author (Authors) 黃瑜萍 zh_TW dc.contributor.author (Authors) Huang, Yu-Ping en_US dc.creator (作者) 黃瑜萍 zh_TW dc.creator (作者) Huang, Yu-Ping en_US dc.date (日期) 2021 en_US dc.date.accessioned 4-Aug-2021 15:58:49 (UTC+8) - dc.date.available 4-Aug-2021 15:58:49 (UTC+8) - dc.date.issued (上傳時間) 4-Aug-2021 15:58:49 (UTC+8) - dc.identifier (Other Identifiers) G0108258021 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/136559 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 經濟學系 zh_TW dc.description (描述) 108258021 zh_TW dc.description.abstract (摘要) 本研究引入深度 Q 學習方法，建構一個自動化股票交易系統，研究範圍包含台灣股票市場 14 家科技業公司。研究期間為 2016 年 1 月 4 日至 2020年 12 月 31 日。本研究數據資料有兩種型態 (1) 股票資訊，(2) 股票資訊加上匯率參數。我們將深度 Q 學習的模型，與不同模型和其他策略相比較，以檢測深度 Q 學習是否更適用於股票交易。實證結果發現支持向量機與神經網路在實務面上難以進行股票交易操作，而深度 Q 學習的模型則具有相對好的成效。尤其，加入匯率參數的深度 Q 學習，獲得的報酬皆優於買入持有策略和台灣加權股價指數。 zh_TW dc.description.abstract (摘要) This research introduces the Deep Q learning model to construct an automated stock trading system. Our samples are 14 Taiwanese technology companies. Specifically, we include two types of data, (1) stock information and (2) stock information and exchange rate parameters, which are collected from the Taiwan stock market. The sampling period is from Jan 4, 2016 to Dec 31, 2020. We compare our main model, Deep Q learning, with different models and strategies to examine whether Deep Q learning is more applicable to stock trading. The empirical results show that it is difficult for Support vector machines and Neural networks to operate stock trading; however, Deep Q learning demonstrates better performance. In particular, the return rate of the Deep Q learning model is higher than the Buy-and-hold strategy and Taiwan Weighted Stock Index if considering exchange rate parameters. en_US dc.description.tableofcontents 致謝 i中文摘要 iiAbstract iii目錄 iv表目錄 vi圖目錄 vii第一章 緒論 1第一節 研究背景與動機 1第二節 強化學習簡述 1第三節 問題設定 2第二章 文獻回顧 3第一節 匯率與股市關聯性 3第二節 強化學習在金融領域之應用 3第三章 機器學習與深度學習模型探討 5第一節 支持向量機 5第二節 神經網路 7一、神經網路架構 9二、激發函數 10三、損失函數 12第三節 強化學習 14一、馬可夫決策過程 15二、價值函數 16三、蒙地卡羅法及時差學習法 17四、深度強化學習 19第四章 資料來源及說明 21第一節 股票資訊 21第二節 匯率參數 22第三節 模型使用類型輸入資料 22第五章 實證結果 24第一節 模型結構介紹 24第二節 支持向量機、神經網路對於股票資訊預測 24第三節 深度強化學習模型之結構變化與過程 28第四節 深度強化學習模型實證結果 34一、深度強化學習與買入持有策略 34二、深度強化學習與台灣加權股價指數 36三、深度 Q 學習綜合比較 38四、預測模型 (支持向量機、神經網路) 與深度強化學習模型之比較 42第六章 結論與未來展望 43參考文獻 44 zh_TW dc.format.extent 2358372 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0108258021 en_US dc.subject (關鍵詞) 深度學習 zh_TW dc.subject (關鍵詞) 強化學習 zh_TW dc.subject (關鍵詞) 深度 Q 學習 zh_TW dc.subject (關鍵詞) 匯率 zh_TW dc.subject (關鍵詞) 股票交易 zh_TW dc.subject (關鍵詞) Deep learning en_US dc.subject (關鍵詞) Reinforcement learning en_US dc.subject (關鍵詞) Deep Q learning en_US dc.subject (關鍵詞) Exchange rate en_US dc.subject (關鍵詞) Stock trading en_US dc.title (題名) 深度強化學習之模型比較: 以股票自動交易系統為例 zh_TW dc.title (題名) A Comparison of Deep Reinforcement Learning Models: The Case of Stock Automated Trading System en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) 王泓仁. 台幣匯率對我國經濟金融活動之影響. 中央銀行季刊》,(),–, Wang, Hung-Jen (),“e Impacts of NT Dollar Exchange Rates on Taiwan＇s Economy＂, Quarterly Reviews, Central Bank of the Republic of China (Taiwan),(), 2005.徐千婷. 匯率與總體經濟變數之關係: 台灣實證分析, 2006.Wei Bao, Jun Yue, and Yulei Rao. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PloS one, 12(7):e0180944, 2017.Alessio Brini and Daniele Tantari. Deep reinforcement trading with predictable returns. arXiv preprint arXiv:2104.14683, 2021.Arthur Charpentier, Romuald Elie, and Carl Remlinger.Reinforcement learning in economics and finance. Computational Economics, pages 1–38, 2021.Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3): 273–297, 1995.Li Deng and Dong Yu. Deep learning: methods and applications. Foundations and trends in signal processing, 7(3–4):197–387, 2014.Eugene F Fama. The behavior of stock-market prices. The journal of Business, 38(1): 34–105, 1965.Eugene F Fama. Random walks in stock market prices. Financial analysts journal, 51(1): 75–80, 1995.Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.Chien Yi Huang. Financial trading as a game: A deep reinforcement learning approach. arXiv preprint arXiv:1807.02787, 2018.Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285, 1996.Konstantina Kourou, Themis P Exarchos, Konstantinos P Exarchos, Michalis V Karamouzis, and Dimitrios I Fotiadis. Machine learning applications in cancer prognosis and prediction. Computational and structural biotechnology journal, 13:8–17, 2015.Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553): 436–444, 2015.Jae Won Lee. Stock price prediction using reinforcement learning. In ISIE 2001. 2001 IEEE International Symposium on Industrial Electronics Proceedings (Cat. No. 01TH8570), volume 1, pages 690–695. IEEE, 2001.Jae Won Lee, Jonghun Park, O Jangmin, Jongwoo Lee, and Euyseok Hong. A multiagentapproach to q-learning for daily stock trading. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 37(6):864–877, 2007.Pierre Menard, Omar Darwiche Domingues, Xuedong Shang, and Michal Valko. Ucb momentum q-learning: Correcting the bias without forgetting. arXiv preprint arXiv: 2103.01312, 2021.Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv: 1507.04296, 2015.Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. Multimodal deep learning. In ICML, 2011.Kate Phylaktis and Fabiola Ravazzolo. Stock prices and exchange rate dynamics. Journal of international Money and Finance, 24(7):1031–1053, 2005.G William Schwert. Business cycles, financial crises, and stock volatility. In CarnegieRochester Conference series on public policy, volume 31, pages 83–125. ERichard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522, 2018.Paul D Yoo, Maria H Kim, and Tony Jan. Machine learning techniques and use of event information for stock market prediction: A survey and evaluation. In International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06), volume 2, pages 835–841. IEEE, 2005. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202100671 en_US
