學術產出-學位論文
文章檢視/開啟
書目匯出
Google ScholarTM
政大圖書館
引文資訊
-
No data in Web of Science(Wrong one)No data in Scopus (Wrong two)Server returned HTTP response code: 429 for URL: https://api.elsevier.com/content/search/scopus?apiKey=80c8c594bfc1533f4784bcba5a0cd6dd&query=DOI(10.6814%2FNCCU202000375)+OR+TITLE-ABS-KEY%28%22%E5%9F%BA%E6%96%BC%E6%B7%B1%E5%BA%A6%E5%BC%B7%E5%8C%96%E5%AD%B8%E7%BF%92%E7%9A%84%E6%99%BA%E8%83%BD%E5%AD%98%E8%B2%A8%E6%8E%A7%E5%88%B6%EF%BC%9A%E4%BB%A5%E9%AB%98%E7%A7%91%E6%8A%80%E4%BE%9B%E6%87%89%E9%8F%88%E7%82%BA%E4%BE%8B%22%29
TAIR相關學術產出
| 題名 | 基於深度強化學習的智能存貨控制：以高科技供應鏈為例 A Deep Reinforcement Learning approach for Intelligent Inventory Control in high-tech supply chains |
| 作者 | 廖信堯 Liao, Hsin-Yao |
| 貢獻者 | 莊皓鈞 Chuang, Hao-Chun 廖信堯 Liao, Hsin-Yao |
| 關鍵詞 | 強化學習 深度學習 庫存最佳化 模擬 作業管理 Reinforcement learning Deep learning Inventory optimization Simulation Operations management |
| 日期 | 2020 |
| 上傳時間 | 6-四月-2020 14:44:04 (UTC+8) |
| 摘要 | Machine learning is revolutionizing business operations across industry sectors. Among different learning techniques, deep reinforcement learning (DRL) has received broad attention in recent years due to the salient performance of AlphaGo, an artificial intelligence (AI) system empowered by DRL. DRL is a model-free and data-driven approach to develop near-optimal policies for sequential decision-making problems. Intrigued by the success of DRL in various fields, we, in this study, assess the applicability of DRL to multi-period inventory control under stochastic demand, which is a classical Markov Decision Process problem. Working with the largest distributor of electronics manufacturing services (EMS) in the world, we propose deep Q-networks (DQN) for intelligent inventory control (IIC). Facing erratic and non-stationary demand for electronic components with limited market life cycle, the distributor could not infer the exact demand distribution and solve the inventory optimization problem analytically in a finite-horizon with lost sales setting. Hence, we develop DQN by specifying relevant state and decision inputs, and then designing a data-driven simulation environment, in which the agent is trained over thousands of episodes. For trained items, DQN outperforms the benchmark in a few ways. First, DQN can reduce the total inventory by at least 40% while achieving better service level. Second, when penalty parameter increases, DQN can effectively reduce the amount of out-of-stock. While we transfer trained DQN into testing sets, within the same item, the out-of-sample performance is excellent. For other unseen items, we use the Maximum Entropy Bootstrap to train ensemble DDQN and make our DRL agent more robust. Given the promising results in our experiments, we discuss implications, limitations, and further directions for applying DRL/DQN to business decision-making problems. |
| 參考文獻 | Arulkumaran, K., Deisenroth, P. M., Brundage, M., & Bharath, A. A. (2017) A brief survey of deep reinforcement learning. ArXiv: 1708.05866v2. Chaharsooghi, S., Heydari, J., & Zegordi, S. (2008) A reinforcement learning model for supply chain ordering management: An application to the beer game. Decision Support Systems, 45(4): 949-959. Chollet, F. (2017) Deep Learning with Python. Manning Publications. Giannoccaro, I., & Pontrandolfo, P. (2002) Inventory management in supply chains: A reinforcement learning approach. International Journal of Production Economics, 78(2): 153-161. Gijsbrechts, J., Boute, R., Zhang, D., & Van Mieghem, J. (2019) Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems. Available at SSRN https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3302881. Gosavi, A. (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing, 21(3): 177-345. Kingma, D. P., & Ba, J. (2017). Adam: A Method for Stochastic Optimization. ArXiv:1412.6980v9. Lin, L. J. (1992) Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3-4): 293-321. Mnih V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) Playing Atari with deep reinforcement learning. ArXiv:1312.5602v1. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., & Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015) Human-level control through deep reinforcement learning. Nature, 518: 529-533. Oroojlooyjadid, A., Snyder, L., & Takáč, M. (2019) Applying deep learning to the newsvendor problem. IISE Transactions, in press. Oroojlooyjadid, A., Nazari, M., Snyder, L., & Takáč, M. (2019b) A deep Q-Network for the beer game: A deep reinforcement learning algorithm to solve inventory optimization problems. ArXiv:1708.05924v3. Porteus, E. L. (2002) Foundations of Stochastic Inventory Theory. Stanford University Press, California. Qi, X., Wu, G., Boriboonsomsin, K., Barth, M. J., & Gonder, J. (2016) Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles. Transportation Research Record, 2572(1), 1-8. Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. ArXiv: 1511.05952v4. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., Hassabis, D. (2017) Mastering the game of Go without human knowledge. Nature 550: 354-359. Sutton, R. S. and Barto, A. G. (2018) Reinforcement Learning: An Introduction. Second edition, MIT Press, Cambridge. Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-Learning. ArXiv:1509.06461. Vinod, H.D. and Lòpez-de-Lacalle, J. (2009). Maximum entropy bootstrap for time series. The meboot R Package. J. Stat. Softw., 29 (2009), pp. 1-19 |
| 描述 | 碩士 國立政治大學 資訊管理學系 107356003 |
| 資料來源 | http://thesis.lib.nccu.edu.tw/record/#G0107356003 |
| 資料類型 | thesis |
| dc.contributor.advisor | 莊皓鈞 | zh_TW |
| dc.contributor.advisor | Chuang, Hao-Chun | en_US |
| dc.contributor.author (作者) | 廖信堯 | zh_TW |
| dc.contributor.author (作者) | Liao, Hsin-Yao | en_US |
| dc.creator (作者) | 廖信堯 | zh_TW |
| dc.creator (作者) | Liao, Hsin-Yao | en_US |
| dc.date (日期) | 2020 | en_US |
| dc.date.accessioned | 6-四月-2020 14:44:04 (UTC+8) | - |
| dc.date.available | 6-四月-2020 14:44:04 (UTC+8) | - |
| dc.date.issued (上傳時間) | 6-四月-2020 14:44:04 (UTC+8) | - |
| dc.identifier (其他 識別碼) | G0107356003 | en_US |
| dc.identifier.uri (URI) | http://nccur.lib.nccu.edu.tw/handle/140.119/129213 | - |
| dc.description (描述) | 碩士 | zh_TW |
| dc.description (描述) | 國立政治大學 | zh_TW |
| dc.description (描述) | 資訊管理學系 | zh_TW |
| dc.description (描述) | 107356003 | zh_TW |
| dc.description.abstract (摘要) | Machine learning is revolutionizing business operations across industry sectors. Among different learning techniques, deep reinforcement learning (DRL) has received broad attention in recent years due to the salient performance of AlphaGo, an artificial intelligence (AI) system empowered by DRL. DRL is a model-free and data-driven approach to develop near-optimal policies for sequential decision-making problems. Intrigued by the success of DRL in various fields, we, in this study, assess the applicability of DRL to multi-period inventory control under stochastic demand, which is a classical Markov Decision Process problem. Working with the largest distributor of electronics manufacturing services (EMS) in the world, we propose deep Q-networks (DQN) for intelligent inventory control (IIC). Facing erratic and non-stationary demand for electronic components with limited market life cycle, the distributor could not infer the exact demand distribution and solve the inventory optimization problem analytically in a finite-horizon with lost sales setting. Hence, we develop DQN by specifying relevant state and decision inputs, and then designing a data-driven simulation environment, in which the agent is trained over thousands of episodes. For trained items, DQN outperforms the benchmark in a few ways. First, DQN can reduce the total inventory by at least 40% while achieving better service level. Second, when penalty parameter increases, DQN can effectively reduce the amount of out-of-stock. While we transfer trained DQN into testing sets, within the same item, the out-of-sample performance is excellent. For other unseen items, we use the Maximum Entropy Bootstrap to train ensemble DDQN and make our DRL agent more robust. Given the promising results in our experiments, we discuss implications, limitations, and further directions for applying DRL/DQN to business decision-making problems. | en_US |
| dc.description.tableofcontents | Introduction 1 Section2. Literature Review 3 Section2.1 Reinforcement Learning 3 Section2.2 Deep Reinforcement Learning for Inventory Control 6 Section3. A Deep Reinforcement Learning Agent 8 Section3.1 Problem Situation and Simulation Environment 8 Section3.2 Deep Q-Networks for Inventory Control 10 Section4. Training Performance and Comparisons 15 Section4.1 RL Simulation Design and Neural Net Architecture Tuning 15 Section4.2 Comparisons Between DRL and Benchmark 18 Section5. The Applicability of Transferring Trained Agents 21 Section5.1 Transfer Learning Performance on Latter Period of the Same Item 21 Section5.2 Transfer Learning Performance on New Unseen Items 22 Section5.3 DDQN with Ensemble Learning Method 24 Section6. Conclusion and Discussion 25 References 27 | zh_TW |
| dc.format.extent | 1802849 bytes | - |
| dc.format.mimetype | application/pdf | - |
| dc.source.uri (資料來源) | http://thesis.lib.nccu.edu.tw/record/#G0107356003 | en_US |
| dc.subject (關鍵詞) | 強化學習 | zh_TW |
| dc.subject (關鍵詞) | 深度學習 | zh_TW |
| dc.subject (關鍵詞) | 庫存最佳化 | zh_TW |
| dc.subject (關鍵詞) | 模擬 | zh_TW |
| dc.subject (關鍵詞) | 作業管理 | zh_TW |
| dc.subject (關鍵詞) | Reinforcement learning | en_US |
| dc.subject (關鍵詞) | Deep learning | en_US |
| dc.subject (關鍵詞) | Inventory optimization | en_US |
| dc.subject (關鍵詞) | Simulation | en_US |
| dc.subject (關鍵詞) | Operations management | en_US |
| dc.title (題名) | 基於深度強化學習的智能存貨控制：以高科技供應鏈為例 | zh_TW |
| dc.title (題名) | A Deep Reinforcement Learning approach for Intelligent Inventory Control in high-tech supply chains | en_US |
| dc.type (資料類型) | thesis | en_US |
| dc.relation.reference (參考文獻) | Arulkumaran, K., Deisenroth, P. M., Brundage, M., & Bharath, A. A. (2017) A brief survey of deep reinforcement learning. ArXiv: 1708.05866v2. Chaharsooghi, S., Heydari, J., & Zegordi, S. (2008) A reinforcement learning model for supply chain ordering management: An application to the beer game. Decision Support Systems, 45(4): 949-959. Chollet, F. (2017) Deep Learning with Python. Manning Publications. Giannoccaro, I., & Pontrandolfo, P. (2002) Inventory management in supply chains: A reinforcement learning approach. International Journal of Production Economics, 78(2): 153-161. Gijsbrechts, J., Boute, R., Zhang, D., & Van Mieghem, J. (2019) Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems. Available at SSRN https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3302881. Gosavi, A. (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing, 21(3): 177-345. Kingma, D. P., & Ba, J. (2017). Adam: A Method for Stochastic Optimization. ArXiv:1412.6980v9. Lin, L. J. (1992) Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3-4): 293-321. Mnih V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) Playing Atari with deep reinforcement learning. ArXiv:1312.5602v1. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., & Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015) Human-level control through deep reinforcement learning. Nature, 518: 529-533. Oroojlooyjadid, A., Snyder, L., & Takáč, M. (2019) Applying deep learning to the newsvendor problem. IISE Transactions, in press. Oroojlooyjadid, A., Nazari, M., Snyder, L., & Takáč, M. (2019b) A deep Q-Network for the beer game: A deep reinforcement learning algorithm to solve inventory optimization problems. ArXiv:1708.05924v3. Porteus, E. L. (2002) Foundations of Stochastic Inventory Theory. Stanford University Press, California. Qi, X., Wu, G., Boriboonsomsin, K., Barth, M. J., & Gonder, J. (2016) Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles. Transportation Research Record, 2572(1), 1-8. Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. ArXiv: 1511.05952v4. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., Hassabis, D. (2017) Mastering the game of Go without human knowledge. Nature 550: 354-359. Sutton, R. S. and Barto, A. G. (2018) Reinforcement Learning: An Introduction. Second edition, MIT Press, Cambridge. Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-Learning. ArXiv:1509.06461. Vinod, H.D. and Lòpez-de-Lacalle, J. (2009). Maximum entropy bootstrap for time series. The meboot R Package. J. Stat. Softw., 29 (2009), pp. 1-19 | zh_TW |
| dc.identifier.doi (DOI) | 10.6814/NCCU202000375 | en_US |