1. NCCU Academic Hub
  2. Publications
  3. College of Informatics
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 工業物聯網中基於深度強化學習之服務功能鏈最佳資源配置機制
A DRL-based Scheme for Optimal Resource Allocation of Service Function Chain in IIoT Networks
作者 林婕
Lin, Chieh
貢獻者 孫士勝
Sun, Shi-Sheng
林婕
Lin, Chieh
關鍵詞 工業物聯網
深度強化學習
服務功能鏈
虛擬網路功能
Industrial Internet of Thing (IIoT)
Deep Reinforcement Learning (DRL)
Service Function Chain (SFC)
Virtual Network Function (VNF)
日期 2025
上傳時間 4-Aug-2025 13:58:54 (UTC+8)
摘要 工業物聯網（Industrial Internet of Things, IIoT）是將物聯網（Internet of Things, IoT）技術運用於工業環境，並透過網路串聯各項設備，實現即時資料的收集、分析與交換，以此有效提升工廠自動化生產與營運流程的效率。IIoT 中的應用相對多元，服務請求亦不盡相同，如何設計並優化資源分配的策略尤其重要，若未妥善處理，可能導致資源使用效率不彰而影響系統整體的效能。為了解決上述問題，本研究導入服務功能鏈（Service Function Chain, SFC），使資料流依序流經一系列預先定義執行順序的虛擬網路功能（Virtual Network Functions, VNFs），提供具彈性的部署方法，支援不同服務的資源需求。我們提出了一種基於深度強化學習之動態服務功能鏈部署的資源分配架構（DRL-based Resource Allocation for Dynamic SFC Embedding, DRL-RADSE），並整合部署及遷移兩種不同的決策模型，以降低系統在處理服務請求過程中所產生的部署延遲。經模擬結果顯示，本文所提出的方法能有效處理不同長度的服務請求，部署及遷移延遲均能收斂，且收斂穩定後的延遲表現優於既有文獻。
The Industrial Internet of Things (IIoT) combines Internet of Things (IoT) technologies into industrial environments. By interconnecting devices through networks, IIoT facilitates real-time data collection, analysis, and exchange, thereby significantly enhancing the efficiency of automated production and operational workflows. The diversity of IIoT applications and the differences of service requests make the design and optimization of resource allocation strategies particularly critical. The unsuitability of allocation strategies may lead to reduced resource utilization and degraded system performance. This thesis introduces Service Function Chaining (SFC), in which data flow sequentially traverses a series of Virtual Network Functions (VNFs) defined in a predetermined execution order. This approach improves the flexibility of function deployment and supports resource requirements across various service types. We propose a DRL-based Resource Allocation framework for Dynamic SFC Embedding, referred to as DRL-RADSE, which integrates two distinct decision models for placement and migration, with the objective of minimizing the average embedding time of total service requests. Simulation results show that the proposed method can effectively handle service requests of varying lengths, achieve convergence in placement and migration embedding time, and outperform existing work with improved time performance.
參考文獻 [1] J. Halpern and C. Pignataro, “Service function chaining (sfc) architecture,” Internet Engineering Task Force (IETF), Tech. Rep., 2015. [2] H. Hantouti, N. Benamar, and T. Taleb, “Service function chaining in 5g & beyond networks: Challenges and open research issues,” IEEE Network, vol. 34, no. 4, pp. 320–327, 2020. [3] H. Hantouti, N. Benamar, T. Taleb, and A. Laghrissi, “Traffic steering for service function chaining,” IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 487–507, 2019. [4] S. Sezer, S. Scott-Hayward, P. K. Chouhan, B. Fraser, D. Lake, J. Finnegan, N. Viljoen, M. Miller, and N. Rao, “Are we ready for sdn? implementation challenges for software-defined networks,” IEEE Communications Magazine, vol. 51, no. 7, pp. 36–43, 2013. [5] B. A. A. Nunes, M. Mendonca, X.-N. Nguyen, K. Obraczka, and T. Turletti, “A survey of software-defined networking: Past, present, and future of programmable networks,” IEEE Communications Surveys & Tutorials, vol. 16, no. 3, pp. 1617–1634, 2014. [6] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. The MIT Press, 2015. [7] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015. [8] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” 2019. [Online]. Available: https://arxiv.org/abs/1509.02971 [9] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller., “Deterministic policy gradient algorithms,” International Conference on Machine Learning, vol. 9, 2014. [10] T.-W. Kuo, B.-H. Liou, K. C.-J. Lin, and M.-J. Tsai, “Deploying chains of virtual network functions: On the relation between link and server usage,” IEEE/ACM Transactions on Networking, vol. 26, no. 4, pp. 1562–1576, 2018. [11] Y. Wu, Z. Jia, Q. Wu, and Z. Lu, “Adaptive qoe-aware sfc orchestration in uav networks: A deep reinforcement learning approach,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 6, pp. 6052–6065, 2024. [12] Y.-H. Hsu, T.-R. Tsai, T.-C. Yeh, and Y.-L. Wang, “Deep reinforcement learning based mobility-aware sfc embedding for mec in 5g and beyond,” in 2023 IEEE Wireless Communications and Networking Conference (WCNC), 2023, pp. 1–6. [13] X. Yu, R. Wang, J. Hao, Q. Wu, C. Yi, P. Wang, and D. Niyato, “Priority-aware deployment of autoscaling service function chains based on deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol. 10, no. 3, pp. 1050–1062, 2024. [14] J. Li, R. Wang, and K. Wang, “Service function chaining in industrial internet of things with edge intelligence: A natural actor-critic approach,” IEEE Transactions on Industrial Informatics, vol. 19, no. 1, pp. 491–502, 2023. [15] R. Behravesh, D. Harutyunyan, E. Coronado, and R. Riggio, “Time-sensitive mobile user association and sfc placement in mec-enabled 5g networks,” IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3006–3020, 2021. [16] T. Jing, Z. Liu, M. Zhu, X. Li, B. Gao, Q. Gao, and Y. Huo, “P-drr: Ppo-based efficient dynamic resource reallocation scheme in industrial internet of things,” in 2023 IEEE 98th Vehicular Technology Conference (VTC2023-Fall), 2023, pp. 1–5. [17] L. Stahlbock, J. Weber, and F. Köster, “An optimization approach of container startup times for time-sensitive embedded systems,” in 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), 2022, pp. 2019–2026. [18] S. He, X. Lyu, W. Ni, H. Tian, R. P. Liu, and E. Hossain, “Virtual service placement for edge computing under finite memory and bandwidth,” IEEE Transactions on Communications, vol. 68, no. 12, pp. 7702–7718, 2020. [19] T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” CoRR, 09 2015. [20] L. Zhang, S. Zhuge, Y. Wang, H. Xu, and E. Sun, “Energy-delay tradeoff for virtual machine placement in virtualized multi-access edge computing: A two-sided matching approach,” TechRxiv, 12 2020.
描述 碩士
國立政治大學
資訊科學系
112753135
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0112753135
資料類型 thesis
dc.contributor.advisor 孫士勝zh_TW
dc.contributor.advisor Sun, Shi-Shengen_US
dc.contributor.author (Authors) 林婕zh_TW
dc.contributor.author (Authors) Lin, Chiehen_US
dc.creator (作者) 林婕zh_TW
dc.creator (作者) Lin, Chiehen_US
dc.date (日期) 2025en_US
dc.date.accessioned 4-Aug-2025 13:58:54 (UTC+8)-
dc.date.available 4-Aug-2025 13:58:54 (UTC+8)-
dc.date.issued (上傳時間) 4-Aug-2025 13:58:54 (UTC+8)-
dc.identifier (Other Identifiers) G0112753135en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/158481-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學系zh_TW
dc.description (描述) 112753135zh_TW
dc.description.abstract (摘要) 工業物聯網（Industrial Internet of Things, IIoT）是將物聯網（Internet of Things, IoT）技術運用於工業環境，並透過網路串聯各項設備，實現即時資料的收集、分析與交換，以此有效提升工廠自動化生產與營運流程的效率。IIoT 中的應用相對多元，服務請求亦不盡相同，如何設計並優化資源分配的策略尤其重要，若未妥善處理，可能導致資源使用效率不彰而影響系統整體的效能。為了解決上述問題，本研究導入服務功能鏈（Service Function Chain, SFC），使資料流依序流經一系列預先定義執行順序的虛擬網路功能（Virtual Network Functions, VNFs），提供具彈性的部署方法，支援不同服務的資源需求。我們提出了一種基於深度強化學習之動態服務功能鏈部署的資源分配架構（DRL-based Resource Allocation for Dynamic SFC Embedding, DRL-RADSE），並整合部署及遷移兩種不同的決策模型，以降低系統在處理服務請求過程中所產生的部署延遲。經模擬結果顯示，本文所提出的方法能有效處理不同長度的服務請求，部署及遷移延遲均能收斂，且收斂穩定後的延遲表現優於既有文獻。zh_TW
dc.description.abstract (摘要) The Industrial Internet of Things (IIoT) combines Internet of Things (IoT) technologies into industrial environments. By interconnecting devices through networks, IIoT facilitates real-time data collection, analysis, and exchange, thereby significantly enhancing the efficiency of automated production and operational workflows. The diversity of IIoT applications and the differences of service requests make the design and optimization of resource allocation strategies particularly critical. The unsuitability of allocation strategies may lead to reduced resource utilization and degraded system performance. This thesis introduces Service Function Chaining (SFC), in which data flow sequentially traverses a series of Virtual Network Functions (VNFs) defined in a predetermined execution order. This approach improves the flexibility of function deployment and supports resource requirements across various service types. We propose a DRL-based Resource Allocation framework for Dynamic SFC Embedding, referred to as DRL-RADSE, which integrates two distinct decision models for placement and migration, with the objective of minimizing the average embedding time of total service requests. Simulation results show that the proposed method can effectively handle service requests of varying lengths, achieve convergence in placement and migration embedding time, and outperform existing work with improved time performance.en_US
dc.description.tableofcontents 1 Introduction 1 1.1 Background 1 1.2 Motivation 2 1.3 Contribution 3 1.4 Thesis Organization 3 2 Related Work 5 2.1 Background Knowledge 5 2.2 Related Work 11 3 System Model 15 3.1 Definition of Time Component 15 3.2 System Architecture 17 3.3 Problem Formulation 22 4 DRL-based Resource Allocation for Dynamic SFC Embedding (DRL-RADSE) 25 4.1 Overview of DRL-RADSE 25 4.2 MDP module 30 4.3 Algorithm Design and Implementation 32 5 Performance Evaluation 35 5.1 Experimental Setup 35 5.2 Comparison Methods 37 5.3 Simulation Results 37 6 Conclusion and Future Work 49 6.1 Conclusion 49 6.2 Future Work 50 Bibliography 51zh_TW
dc.format.extent 12927012 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0112753135en_US
dc.subject (關鍵詞) 工業物聯網zh_TW
dc.subject (關鍵詞) 深度強化學習zh_TW
dc.subject (關鍵詞) 服務功能鏈zh_TW
dc.subject (關鍵詞) 虛擬網路功能zh_TW
dc.subject (關鍵詞) Industrial Internet of Thing (IIoT)en_US
dc.subject (關鍵詞) Deep Reinforcement Learning (DRL)en_US
dc.subject (關鍵詞) Service Function Chain (SFC)en_US
dc.subject (關鍵詞) Virtual Network Function (VNF)en_US
dc.title (題名) 工業物聯網中基於深度強化學習之服務功能鏈最佳資源配置機制zh_TW
dc.title (題名) A DRL-based Scheme for Optimal Resource Allocation of Service Function Chain in IIoT Networksen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) [1] J. Halpern and C. Pignataro, “Service function chaining (sfc) architecture,” Internet Engineering Task Force (IETF), Tech. Rep., 2015. [2] H. Hantouti, N. Benamar, and T. Taleb, “Service function chaining in 5g & beyond networks: Challenges and open research issues,” IEEE Network, vol. 34, no. 4, pp. 320–327, 2020. [3] H. Hantouti, N. Benamar, T. Taleb, and A. Laghrissi, “Traffic steering for service function chaining,” IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 487–507, 2019. [4] S. Sezer, S. Scott-Hayward, P. K. Chouhan, B. Fraser, D. Lake, J. Finnegan, N. Viljoen, M. Miller, and N. Rao, “Are we ready for sdn? implementation challenges for software-defined networks,” IEEE Communications Magazine, vol. 51, no. 7, pp. 36–43, 2013. [5] B. A. A. Nunes, M. Mendonca, X.-N. Nguyen, K. Obraczka, and T. Turletti, “A survey of software-defined networking: Past, present, and future of programmable networks,” IEEE Communications Surveys & Tutorials, vol. 16, no. 3, pp. 1617–1634, 2014. [6] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. The MIT Press, 2015. [7] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015. [8] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” 2019. [Online]. Available: https://arxiv.org/abs/1509.02971 [9] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller., “Deterministic policy gradient algorithms,” International Conference on Machine Learning, vol. 9, 2014. [10] T.-W. Kuo, B.-H. Liou, K. C.-J. Lin, and M.-J. Tsai, “Deploying chains of virtual network functions: On the relation between link and server usage,” IEEE/ACM Transactions on Networking, vol. 26, no. 4, pp. 1562–1576, 2018. [11] Y. Wu, Z. Jia, Q. Wu, and Z. Lu, “Adaptive qoe-aware sfc orchestration in uav networks: A deep reinforcement learning approach,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 6, pp. 6052–6065, 2024. [12] Y.-H. Hsu, T.-R. Tsai, T.-C. Yeh, and Y.-L. Wang, “Deep reinforcement learning based mobility-aware sfc embedding for mec in 5g and beyond,” in 2023 IEEE Wireless Communications and Networking Conference (WCNC), 2023, pp. 1–6. [13] X. Yu, R. Wang, J. Hao, Q. Wu, C. Yi, P. Wang, and D. Niyato, “Priority-aware deployment of autoscaling service function chains based on deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol. 10, no. 3, pp. 1050–1062, 2024. [14] J. Li, R. Wang, and K. Wang, “Service function chaining in industrial internet of things with edge intelligence: A natural actor-critic approach,” IEEE Transactions on Industrial Informatics, vol. 19, no. 1, pp. 491–502, 2023. [15] R. Behravesh, D. Harutyunyan, E. Coronado, and R. Riggio, “Time-sensitive mobile user association and sfc placement in mec-enabled 5g networks,” IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 3006–3020, 2021. [16] T. Jing, Z. Liu, M. Zhu, X. Li, B. Gao, Q. Gao, and Y. Huo, “P-drr: Ppo-based efficient dynamic resource reallocation scheme in industrial internet of things,” in 2023 IEEE 98th Vehicular Technology Conference (VTC2023-Fall), 2023, pp. 1–5. [17] L. Stahlbock, J. Weber, and F. Köster, “An optimization approach of container startup times for time-sensitive embedded systems,” in 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), 2022, pp. 2019–2026. [18] S. He, X. Lyu, W. Ni, H. Tian, R. P. Liu, and E. Hossain, “Virtual service placement for edge computing under finite memory and bandwidth,” IEEE Transactions on Communications, vol. 68, no. 12, pp. 7702–7718, 2020. [19] T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” CoRR, 09 2015. [20] L. Zhang, S. Zhuge, Y. Wang, H. Xu, and E. Sun, “Energy-delay tradeoff for virtual machine placement in virtualized multi-access edge computing: A two-sided matching approach,” TechRxiv, 12 2020.zh_TW