學術產出-Theses
Article View/Open
Publication Export
-
題名 Q-Learning 與 Deep Q-Learning 於都市路邊停車位搜尋之研究
Study on Q-Learning and Deep Q-Learning in Urban Roadside Parking Space Search作者 李俊毅
Lee, Chun-Yee貢獻者 張宏慶
Jang, Hung-Chin
李俊毅
Lee, Chun-Yee關鍵詞 深度學習
強化學習
停車位搜尋日期 2022 上傳時間 2-Sep-2022 15:04:38 (UTC+8) 摘要 隨著近年物聯網、5G網路應用及深度學習等技術的發展。智慧城市的發展也逐漸受到重視，其中智慧交通更是政府主要的推廣的領域。對於停車位搜尋問題，目前最常見的做法是在每個停車位設置感測器，透過提供駕駛即時的停車位資訊令駕駛能找到合適的停車位，也因此近年停車位搜尋問題的相關研究多半著重在改善對於停車空位的偵測能力。雖然直接提供每個固定停車位的即時資訊，可以為駕駛選擇行駛路線提供幫助，但在使用者驅車前往該停車空位的過程，停車空位仍有先被其他駕駛使用的可能性。這會導致使用者必須在途中重新判斷並選擇行駛路線，而無法在原先估計的地點順利停車，形成欲透過提供停車位資訊間接解決停車問題時的盲點。為解決此問題，停車位搜尋問題的另一種研究方向是將重心放在找到合適的停車路徑而非鎖定特定的停車空位上。透過預測停車空位存在的機率，為使用者導航較高機率的停車行駛路線，進而減少車輛因尋找停車位導致徘徊的行車時間。由於目前台灣的實體環境適用於停車空位統計的相關資料較少，且設置大量相關感測器的成本高昂，短期難以實現大規模的建設。本研究擬以需要設置較少感測器的深度強化學習方法，Deep Q-Learning解決停車位搜尋問題，並加入及LSTM(Long Short-Term Memory)及GRU(Gated Recurrent Unit)神經網路模型提升深度強化學習模型中對於Q值估算的精確度。最終透過深度強化學習模型引導車輛行駛停車路線，達到降低停車所需行駛時間的目的。而缺乏適用於研究問題統計資料的問題，本研究將透過SUMO模擬器(Simulation of Urban MObility)，根據整體的停車頻率及車流量產生擬真的車流環境，以此作為學習模型的訓練資料及實驗資料。模擬資料將隨機產生一般車流與停車車流。一般車流是指目的地在停車範圍外不會停留於停車範圍內的車流，這類車流將以最短路徑在模擬環境中行經停車範圍。停車車流則是目的地設定在停車範圍內的車流，用以模擬實際車流環境中可能會在目標車輛搜尋停車路徑中停車的其他車輛。本研究旨在以模擬車流資料驗證使用深度強化學習解決停車位搜尋問題的有效性，並比較強化學習及深度學習方法在解決停車位搜尋問題的表現，評估不同深度學習方法在進一步解決停車問題上的效益。 參考文獻 1. Chase Dowling, Tanner Fiez, Lillian Ratliff, Baosen Zhang, “How Much Urban Traffic is Searching for Parking”, arXiv:1702.06156, Feb. 20172. Asma Houissa, Dominique Barth, Nadège Faul, Thierry Mautor, “A Learning Algorithm to Minimize the Expectation Time of Finding a Parking Place In Urban Area”, 22nd IEEE Symposium on Computers and Communication: Workshops – ISUT, 20173. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra Martin Riedmiller, “Playing Atari with Deep Reinforcement Learning”, DeepMind Technologies, Dec. 20134. Shuguan Yang, Wei Ma, Xidong Pi, Sean Qian , “A Deep Learning Approach to Real-Time Parking Occupancy Prediction in Spatio-Temporal Networks Incorporating Multiple Spatio-Temporal Data Sources”, arXiv:1901.06758v5, May 20195. B. Xu, O. Wolfson, J. Yang, L. Stenneth, P. S. Yu and P. C. Nelson, “Real-Time Street Parking Availability Estimation,” 2013 IEEE 14th International Conference on Mobile Data Management, 20136. Zeng, C. Ma, K. Wang and Z. Cui, “Parking Occupancy Prediction Method Based on Multi Factors and Stacked GRU-LSTM”, in IEEE Access, vol. 10, pp. 47361-47370, 20227. “HISTORY OF INTELLIGENT TRANSPORTATION SYSTEM”, U.S. Department of Transportation, 20218. 台北市交通政策白皮書, 台北市政府交通局, 20189. X. Fang, R. Xiang, L. Peng, H. Li and Y. Sun, “SAW: A Hybrid Prediction Model for Parking Occupancy Under the Environment of Lacking Real-Time Data”, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society, 201810. K. Kashihara, “Deep Q learning for traffic simulation in autonomous driving at a highway junction”, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 201711. M. Liu, J. Naoum-Sawaya, Y. Gu, F. Lecue and R. Shorten, “A Distributed Markovian Parking Assist System,” in IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2230-2240, June 201912. Gould Michael, Montoliu Raul, Torres-Sospedra Joaquín, Huerta Joaquín, “An Occupancy Simulator for a Smart Parking System: Developmental Design and Experimental Considerations”, ISPRS International Journal of Geo-Information; Basel Vol. 8, Iss. 5, 201913. 自用小客車使用狀況調查報告, 中華民國交通部統計處, 201914. T. Rajabioun and P. A. Ioannou, “On-Street and Off-Street Parking Availability Prediction Using Multivariate Spatiotemporal Models”, in IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 5, pp. 2913-2924, Oct. 201515. F. Bock, S. Di Martino and A. Origlia, “Smart Parking: Using a Crowd of Taxis to Sense On-Street Parking Space Availability”, in IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 2, pp. 496-508, Feb. 202016. I. Aydin, M. Karakose and E. Karakose, “A navigation and reservation based smart parking platform using genetic optimization for smart cities”, 2017 5th International Istanbul Smart Grid and Cities Congress and Fair (ICSG), 201717. 鄔德傳, 臺北市推動智慧停車之挑戰與對策, 國家文官學院, T&D飛訊第266期, 2020 May18. 臺北市路邊停車格位圖層, 政府資料開放平臺, 臺北市停車管理工程處, 202219. L. Xiangdong, C. Yuefeng, C. Gang and X. Zengwei, “Prediction of short-term available parking space using LSTM model”, 2019 14th International Conference on Computer Science & Education (ICCSE), 201920. Clare Chen, Vincent Ying, Dillon Laird, “Deep Q-Learning with Recurrent Neural Networks”, 201621. J. Yu, K. Zhang and L. Peng, “Integrated Prediction of Regional Traffic Situation Based on Multi-Task Spatial-Temporal Network”, IECON 2021 – 47th Annual Conference of the IEEE Industrial Electronics Society, 202122. L. Zheng, X. Xiao, B. Sun, D. Mei and B. Peng, “Short-Term Parking Demand Prediction Method Based on Variable Prediction Interval”, in IEEE Access, vol. 8, pp. 58594-58602, 202023. Shudong Yang, Xueying Yu, Ying Zhou, “LSTM and GRU neural network performance comparison study”, International Workshop on Electronic Communication and Artificial Intelligence (IWECAI), 202024. J. Lin, S. -Y. Chen, C. -Y. Chang and G. Chen, “SPA: Smart Parking Algorithm Based on Driver Behavior and Parking Traffic Predictions”, in IEEE Access, vol. 7, pp. 34275-34288, 201925. X. Ye, J. Wang, T. Wang, X. Yan, Q. Ye and J. Chen, “Short-Term Prediction of Available Parking Space Based on Machine Learning Approaches”, in IEEE Access, vol. 8, pp. 174530-174541, 2020 描述 碩士
國立政治大學
資訊科學系
106753020資料來源 http://thesis.lib.nccu.edu.tw/record/#G0106753020 資料類型 thesis dc.contributor.advisor 張宏慶 zh_TW dc.contributor.advisor Jang, Hung-Chin en_US dc.contributor.author (Authors) 李俊毅 zh_TW dc.contributor.author (Authors) Lee, Chun-Yee en_US dc.creator (作者) 李俊毅 zh_TW dc.creator (作者) Lee, Chun-Yee en_US dc.date (日期) 2022 en_US dc.date.accessioned 2-Sep-2022 15:04:38 (UTC+8) - dc.date.available 2-Sep-2022 15:04:38 (UTC+8) - dc.date.issued (上傳時間) 2-Sep-2022 15:04:38 (UTC+8) - dc.identifier (Other Identifiers) G0106753020 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/141637 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 106753020 zh_TW dc.description.abstract (摘要) 隨著近年物聯網、5G網路應用及深度學習等技術的發展。智慧城市的發展也逐漸受到重視，其中智慧交通更是政府主要的推廣的領域。對於停車位搜尋問題，目前最常見的做法是在每個停車位設置感測器，透過提供駕駛即時的停車位資訊令駕駛能找到合適的停車位，也因此近年停車位搜尋問題的相關研究多半著重在改善對於停車空位的偵測能力。雖然直接提供每個固定停車位的即時資訊，可以為駕駛選擇行駛路線提供幫助，但在使用者驅車前往該停車空位的過程，停車空位仍有先被其他駕駛使用的可能性。這會導致使用者必須在途中重新判斷並選擇行駛路線，而無法在原先估計的地點順利停車，形成欲透過提供停車位資訊間接解決停車問題時的盲點。為解決此問題，停車位搜尋問題的另一種研究方向是將重心放在找到合適的停車路徑而非鎖定特定的停車空位上。透過預測停車空位存在的機率，為使用者導航較高機率的停車行駛路線，進而減少車輛因尋找停車位導致徘徊的行車時間。由於目前台灣的實體環境適用於停車空位統計的相關資料較少，且設置大量相關感測器的成本高昂，短期難以實現大規模的建設。本研究擬以需要設置較少感測器的深度強化學習方法，Deep Q-Learning解決停車位搜尋問題，並加入及LSTM(Long Short-Term Memory)及GRU(Gated Recurrent Unit)神經網路模型提升深度強化學習模型中對於Q值估算的精確度。最終透過深度強化學習模型引導車輛行駛停車路線，達到降低停車所需行駛時間的目的。而缺乏適用於研究問題統計資料的問題，本研究將透過SUMO模擬器(Simulation of Urban MObility)，根據整體的停車頻率及車流量產生擬真的車流環境，以此作為學習模型的訓練資料及實驗資料。模擬資料將隨機產生一般車流與停車車流。一般車流是指目的地在停車範圍外不會停留於停車範圍內的車流，這類車流將以最短路徑在模擬環境中行經停車範圍。停車車流則是目的地設定在停車範圍內的車流，用以模擬實際車流環境中可能會在目標車輛搜尋停車路徑中停車的其他車輛。本研究旨在以模擬車流資料驗證使用深度強化學習解決停車位搜尋問題的有效性，並比較強化學習及深度學習方法在解決停車位搜尋問題的表現，評估不同深度學習方法在進一步解決停車問題上的效益。 zh_TW dc.description.tableofcontents 目錄第一章 緒論 101.1 背景簡介 101.1.1 智慧交通 101.1.2 停車位搜尋問題 121.1.2.1 停車空位機率估算 131.1.2.2 交通統計資料參考與交通環境模擬 141.1.3 強化學習與深度學習 141.1.3.1 強化學習和深度強化學習 141.1.3.2 Q-Learning 161.1.3.3 Deep Q-Learning 171.1.3.4 LSTM 181.1.3.5 GRU 181.2 研究背景與動機 191.3 論文架構 21第二章 相關研究 232.1 How Much Urban Traffic is Searching for Parking [1] 232.2 A Learning Algorithm to Minimize the Expectation Time of Finding a Parking Place In Urban Area [2] 252.3 Playing Atari with Deep Reinforcement Learning [3] 262.4 A Deep Learning Approach to Real-Time Parking Occupancy Prediction in Spatio-Temporal Networks Incorporating Multiple Spatio-Temporal Data Sources [4] 272.5 Real-time Street Parking Availability Estimation [5] 292.6 Parking Occupancy Prediction Method Based on Multi Factors and Stacked GRU-LSTM [6] 31第三章 研究方法 323.1 問題分析與設計 323.1.1 問題定義與系統設計 323.1.2 資料來源與模擬車流環境 343.1.3 模擬實驗與停車狀況分析 353.2 方法論 373.3 流程圖 46第四章 模擬實驗與結果分析 484.1 模擬實驗環境與設定 484.2 模擬實驗 504.3 實驗結果 514.3.1 實驗一（模擬環境設定比較） 514.3.2 實驗二（Q-Learning與Deep Q-Learning比較） 594.3.3 實驗三（LSTM與GRU比較） 67第五章 結論與未來研究 73參考文獻 75 zh_TW dc.format.extent 3445193 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0106753020 en_US dc.subject (關鍵詞) 深度學習 zh_TW dc.subject (關鍵詞) 強化學習 zh_TW dc.subject (關鍵詞) 停車位搜尋 zh_TW dc.title (題名) Q-Learning 與 Deep Q-Learning 於都市路邊停車位搜尋之研究 zh_TW dc.title (題名) Study on Q-Learning and Deep Q-Learning in Urban Roadside Parking Space Search en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) 1. Chase Dowling, Tanner Fiez, Lillian Ratliff, Baosen Zhang, “How Much Urban Traffic is Searching for Parking”, arXiv:1702.06156, Feb. 20172. Asma Houissa, Dominique Barth, Nadège Faul, Thierry Mautor, “A Learning Algorithm to Minimize the Expectation Time of Finding a Parking Place In Urban Area”, 22nd IEEE Symposium on Computers and Communication: Workshops – ISUT, 20173. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra Martin Riedmiller, “Playing Atari with Deep Reinforcement Learning”, DeepMind Technologies, Dec. 20134. Shuguan Yang, Wei Ma, Xidong Pi, Sean Qian , “A Deep Learning Approach to Real-Time Parking Occupancy Prediction in Spatio-Temporal Networks Incorporating Multiple Spatio-Temporal Data Sources”, arXiv:1901.06758v5, May 20195. B. Xu, O. Wolfson, J. Yang, L. Stenneth, P. S. Yu and P. C. Nelson, “Real-Time Street Parking Availability Estimation,” 2013 IEEE 14th International Conference on Mobile Data Management, 20136. Zeng, C. Ma, K. Wang and Z. Cui, “Parking Occupancy Prediction Method Based on Multi Factors and Stacked GRU-LSTM”, in IEEE Access, vol. 10, pp. 47361-47370, 20227. “HISTORY OF INTELLIGENT TRANSPORTATION SYSTEM”, U.S. Department of Transportation, 20218. 台北市交通政策白皮書, 台北市政府交通局, 20189. X. Fang, R. Xiang, L. Peng, H. Li and Y. Sun, “SAW: A Hybrid Prediction Model for Parking Occupancy Under the Environment of Lacking Real-Time Data”, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society, 201810. K. Kashihara, “Deep Q learning for traffic simulation in autonomous driving at a highway junction”, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 201711. M. Liu, J. Naoum-Sawaya, Y. Gu, F. Lecue and R. Shorten, “A Distributed Markovian Parking Assist System,” in IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2230-2240, June 201912. Gould Michael, Montoliu Raul, Torres-Sospedra Joaquín, Huerta Joaquín, “An Occupancy Simulator for a Smart Parking System: Developmental Design and Experimental Considerations”, ISPRS International Journal of Geo-Information; Basel Vol. 8, Iss. 5, 201913. 自用小客車使用狀況調查報告, 中華民國交通部統計處, 201914. T. Rajabioun and P. A. Ioannou, “On-Street and Off-Street Parking Availability Prediction Using Multivariate Spatiotemporal Models”, in IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 5, pp. 2913-2924, Oct. 201515. F. Bock, S. Di Martino and A. Origlia, “Smart Parking: Using a Crowd of Taxis to Sense On-Street Parking Space Availability”, in IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 2, pp. 496-508, Feb. 202016. I. Aydin, M. Karakose and E. Karakose, “A navigation and reservation based smart parking platform using genetic optimization for smart cities”, 2017 5th International Istanbul Smart Grid and Cities Congress and Fair (ICSG), 201717. 鄔德傳, 臺北市推動智慧停車之挑戰與對策, 國家文官學院, T&D飛訊第266期, 2020 May18. 臺北市路邊停車格位圖層, 政府資料開放平臺, 臺北市停車管理工程處, 202219. L. Xiangdong, C. Yuefeng, C. Gang and X. Zengwei, “Prediction of short-term available parking space using LSTM model”, 2019 14th International Conference on Computer Science & Education (ICCSE), 201920. Clare Chen, Vincent Ying, Dillon Laird, “Deep Q-Learning with Recurrent Neural Networks”, 201621. J. Yu, K. Zhang and L. Peng, “Integrated Prediction of Regional Traffic Situation Based on Multi-Task Spatial-Temporal Network”, IECON 2021 – 47th Annual Conference of the IEEE Industrial Electronics Society, 202122. L. Zheng, X. Xiao, B. Sun, D. Mei and B. Peng, “Short-Term Parking Demand Prediction Method Based on Variable Prediction Interval”, in IEEE Access, vol. 8, pp. 58594-58602, 202023. Shudong Yang, Xueying Yu, Ying Zhou, “LSTM and GRU neural network performance comparison study”, International Workshop on Electronic Communication and Artificial Intelligence (IWECAI), 202024. J. Lin, S. -Y. Chen, C. -Y. Chang and G. Chen, “SPA: Smart Parking Algorithm Based on Driver Behavior and Parking Traffic Predictions”, in IEEE Access, vol. 7, pp. 34275-34288, 201925. X. Ye, J. Wang, T. Wang, X. Yan, Q. Ye and J. Chen, “Short-Term Prediction of Available Parking Space Based on Machine Learning Approaches”, in IEEE Access, vol. 8, pp. 174530-174541, 2020 zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202201502 en_US
