1. NCCU Academic Hub
  2. Publications
  3. College of Science
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 卷積深度Ｑ-學習之ETF自動交易系統
Convolutional Deep Q-learning for ETF Automated Trading System
作者 陳非霆
Chen, Fei-Ting
貢獻者 蔡炎龍
陳非霆
Chen, Fei-Ting
關鍵詞 深度學習
增強學習
卷積神經網路
Q-learning
DQN
ETF
Deep learning
Neural network
CNN
Q-leanring
DQN
ETF
日期 2017
上傳時間 1-Dec-2017 12:07:52 (UTC+8)
摘要 本篇文章使用了增強學習與捲積深度學習結合的DQCN模型製作交易系統，希望藉由此交易系統能自行判斷是否買賣ETF，由於ETF屬於穩定性高且手續費高的衍生性金融商品，所以該系統不即時性的做買賣，採用每二十個開盤日進行一次買賣，並由這20個開盤日進行買賣的預測，希望該系統能最大化我們未來的報酬。
DQN是一種增強學習的模型，並在其中使用深度學習進行動作價值的預測，利用增強學習的自我更新動作價值的機制，再用深度學習強大的學習能力成就了人工智慧，並在其取得良好的成效。
In this paper, we used DCQN model, which is combined with reinforcement learning and CNN to train a trading system and hope the trading system could judge whether buy or sell ETFs. Since ETFs is a derivative financial good with high stability and related fee, the system does not perform real-time trading and it performs every 20 trading day. The system predicts value of action based on data in the last 20 opening days to maximize our future rewards.
DQN is a reinforcement learning model, using deep learning to predict value of actions in model. Combined with the RL`s mechanism, which updates value of actions, and deep learning, which has a strong ability of learning, to finish an artificial intelligence. We got a perfect effect.
參考文獻 [1] Anastasia Borovykh, Sander Bohte, and Cornelis W Oosterlee. Conditional time se- ries forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691, 2017.
[2] Guglielmo Maria Caporale, Juncal Cuñado, and Luis A Gil-Alana. Modelling long- run trends and cycles in financial time series data. Journal of Time Series Analysis, 34(3):405–421, 2013.
[3] Thira Chavarnakul and David Enke. Intelligent technical analysis based equivolume charting for stock trading using neural networks. Expert Systems with Applications, 34(2):1004–1017, 2008.
[4] Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. The importance of experience replay database composition in deep reinforcement learning. In Deep Reinforcement Learning Workshop, NIPS, 2015.
[5] John Cristian Borges Gamboa. Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887, 2017.
[6] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.
[7] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.

[8] Ramon Lawrence. Using neural networks to forecast stock market prices. University of Manitoba, 1997.
[9] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553): 436–444, 2015.
[10] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep rein- forcement learning. arXiv preprint arXiv:1509.02971, 2015.
[11] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, pages 1928–1937, 2016.
[12] Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beat- tie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296, 2015.
[13] Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229, 2013.
[14] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.
[15] Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581, 2015.
[16] Yudong Zhang and Lenan Wu. Stock market prediction of s&p 500 via combination of improved bco approach and bp neural network. Expert systems with applications, 36(5):8849–8854, 2009.
描述 碩士
國立政治大學
應用數學系
103751002
資料來源 http://thesis.lib.nccu.edu.tw/record/#G1037510021
資料類型 thesis
dc.contributor.advisor 蔡炎龍zh_TW
dc.contributor.author (Authors) 陳非霆zh_TW
dc.contributor.author (Authors) Chen, Fei-Tingen_US
dc.creator (作者) 陳非霆zh_TW
dc.creator (作者) Chen, Fei-Tingen_US
dc.date (日期) 2017en_US
dc.date.accessioned 1-Dec-2017 12:07:52 (UTC+8)-
dc.date.available 1-Dec-2017 12:07:52 (UTC+8)-
dc.date.issued (上傳時間) 1-Dec-2017 12:07:52 (UTC+8)-
dc.identifier (Other Identifiers) G1037510021en_US
dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/114966-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 應用數學系zh_TW
dc.description (描述) 103751002zh_TW
dc.description.abstract (摘要) 本篇文章使用了增強學習與捲積深度學習結合的DQCN模型製作交易系統，希望藉由此交易系統能自行判斷是否買賣ETF，由於ETF屬於穩定性高且手續費高的衍生性金融商品，所以該系統不即時性的做買賣，採用每二十個開盤日進行一次買賣，並由這20個開盤日進行買賣的預測，希望該系統能最大化我們未來的報酬。
DQN是一種增強學習的模型，並在其中使用深度學習進行動作價值的預測，利用增強學習的自我更新動作價值的機制，再用深度學習強大的學習能力成就了人工智慧，並在其取得良好的成效。		zh_TW
dc.description.abstract (摘要) In this paper, we used DCQN model, which is combined with reinforcement learning and CNN to train a trading system and hope the trading system could judge whether buy or sell ETFs. Since ETFs is a derivative financial good with high stability and related fee, the system does not perform real-time trading and it performs every 20 trading day. The system predicts value of action based on data in the last 20 opening days to maximize our future rewards.
DQN is a reinforcement learning model, using deep learning to predict value of actions in model. Combined with the RL`s mechanism, which updates value of actions, and deep learning, which has a strong ability of learning, to finish an artificial intelligence. We got a perfect effect.		en_US
dc.description.tableofcontents 1 Introduction 1
2 Deep Learning 3
2.1 NeuralNetworkandNeuron.......................... 5
2.1.1 ActivationFunction .......................... 7
2.1.2 LossFunction.............................. 9
2.1.3 GradientDescentMethod ....................... 10
2.2 ConvolutionalNeuralNetwork...................... 11
3 Reinforcement Learning 14
3.1 Introduction................................... 15
3.2 Temporal-DifferencePrediction .................. 18
3.3 Q-Learning ................................... 20
3.4 DeepQ-Learning ................................ 21
3.5 Policy-BasedMethod............................. 24
3.6 Actor-Critic................................... 26
3.7 AsynchronousAdvantageActor-Critic(A3C). . . . . . . .. . . . . 28
4 Exchange-Traded Fund 30
4.1 ExchangeTradedFunds ........................... 31
4.2 AdvantageofETF ............................... 32
4.3 ExampleofETF ................................ 33
5 Automated Trading System 35
5.1 ETFdata.................................... 35
5.2 AutomatedTradingSystem ....................... 37
5.2.1 Introduction .............................. 37
5.2.2 Definition................................ 38
5.2.3 InitialParameterSettlement................ 40
5.2.4 NeuralNetwork............................. 41
5.2.5 DCQN.................................. 42
5.3 Result..................................... 43
6 Conclusion.................................. 45
Bibliography.................................. 47		zh_TW
dc.format.extent 972894 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G1037510021en_US
dc.subject (關鍵詞) 深度學習zh_TW
dc.subject (關鍵詞) 增強學習zh_TW
dc.subject (關鍵詞) 卷積神經網路zh_TW
dc.subject (關鍵詞) Q-learningzh_TW
dc.subject (關鍵詞) DQNzh_TW
dc.subject (關鍵詞) ETFzh_TW
dc.subject (關鍵詞) Deep learningen_US
dc.subject (關鍵詞) Neural networken_US
dc.subject (關鍵詞) CNNen_US
dc.subject (關鍵詞) Q-leanringen_US
dc.subject (關鍵詞) DQNen_US
dc.subject (關鍵詞) ETFen_US
dc.title (題名) 卷積深度Ｑ-學習之ETF自動交易系統zh_TW
dc.title (題名) Convolutional Deep Q-learning for ETF Automated Trading Systemen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) [1] Anastasia Borovykh, Sander Bohte, and Cornelis W Oosterlee. Conditional time se- ries forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691, 2017.
[2] Guglielmo Maria Caporale, Juncal Cuñado, and Luis A Gil-Alana. Modelling long- run trends and cycles in financial time series data. Journal of Time Series Analysis, 34(3):405–421, 2013.
[3] Thira Chavarnakul and David Enke. Intelligent technical analysis based equivolume charting for stock trading using neural networks. Expert Systems with Applications, 34(2):1004–1017, 2008.
[4] Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. The importance of experience replay database composition in deep reinforcement learning. In Deep Reinforcement Learning Workshop, NIPS, 2015.
[5] John Cristian Borges Gamboa. Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887, 2017.
[6] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.
[7] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.

[8] Ramon Lawrence. Using neural networks to forecast stock market prices. University of Manitoba, 1997.
[9] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553): 436–444, 2015.
[10] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep rein- forcement learning. arXiv preprint arXiv:1509.02971, 2015.
[11] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, pages 1928–1937, 2016.
[12] Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beat- tie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296, 2015.
[13] Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229, 2013.
[14] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.
[15] Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581, 2015.
[16] Yudong Zhang and Lenan Wu. Stock market prediction of s&p 500 via combination of improved bco approach and bp neural network. Expert systems with applications, 36(5):8849–8854, 2009.		zh_TW