學術產出-Theses
Article View/Open
Publication Export
-
題名 MQTT 分散式主題切片訊息處理架構
MQTT Extensions for Distributed Topic Sharding作者 姚惠馨
Yao, Hui-Hsin貢獻者 廖峻鋒
Liao, Chun-Feng
姚惠馨
Yao, Hui-Hsin關鍵詞 物聯網
MQTT
分散式架構
切片機制
Internet of Things
MQTT
Distribution architecture
Sharding日期 2023 上傳時間 6-Jul-2023 17:04:17 (UTC+8) 摘要 近年來，物聯網（Internet of Things，IoT）技術蓬勃發展，其 IoT 裝置已無所不在。這些裝置利用物聯網傳輸協定將資料傳送至服務端進行分析來提供各種服務。其中，MQTT 是最常被使用的物聯網通訊協定之一，它在設計上非常精簡且具備一定程度的彈性，非常適合在資源受限或網路頻寬有限的物聯網環境使用。然而，隨著物聯網裝置數量的快速增長，這些裝置所產出的大量訊息往往使得單一 MQTT 伺服器不堪負荷。因此，如何分散負荷以減輕單一 MQTT 伺服器的壓力，是目前必須面臨的重要課題。分散負荷的第一步通常是使用多個 MQTT 伺服器來分散交通量，但目前 MQTT 缺乏多伺服器的輔助機制，尤其是欠缺對分散式回執機制的支援。針對這個問題，本研究提出一個 MQTT 的分散式主題（Topic）切片訊息處理架構，此架構可協調將 Topic 的交通量以切片（Sharding）方式分散在多個 MQTT 伺服器中，並且提供 Topic Binding 機制，使得分散式 MQTT Broker 機制得以實現。最後，將對提出的解決方案進行實作，並評估其系統在不同使用情境下的可行性和效能表現，以驗證可在不影響原有功能的情況下提供額外功能，改善系統資源的使用情況。
In recent years, the Internet of Things (IoT) technology has been flourishing, and IoT devices are ubiquitous. These devices utilize IoT communication protocols to transmit data to the server for analysis and provide various services. Among them, MQTT is one of the most commonly used IoT communication protocols. It is designed to be lightweight and flexible, making it ideal for IoT environments with limited resources or network bandwidth. However, with the rapid growth of IoT devices, the large volume of messages generated by these devices often overwhelms a single MQTT server. Therefore, how to distribute the load and alleviate the pressure on a single MQTT server has become an important issue to address.The first step in load distribution is typically to use multiple MQTT servers to distribute the traffic. However, MQTT currently lacks built-in mechanisms for multi-server support, particularly the lack of support for distributed acknowledgment mechanisms. To address this problem, this study proposes a distributed topic sharding message processing architecture for MQTT. This architecture coordinates the distribution of topic traffic among multiple MQTT servers using sharding techniques and provides a topic binding mechanism to enable a distributed MQTT broker mechanism. Finally, the proposed solution will be implemented and evaluated for its feasibility and performance in different usage scenarios to validate its ability to provide additional functionality without affecting the existing features and improve the utilization of system resources.參考文獻 [1] F. Hmissi and S. Ouni, “A review of application protocol enhancements for internet of things,” UBICOMM 2021 : The Fifteenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, pp. 8–13, October 2021.[2] Yassein, M. B., & Shatnawi, M. Q. (2016, September). Application layer protocols for the Internet of Things: A survey. In 2016 International Conference on Engineering & MIS (ICEMIS) (pp. 1-4). IEEE.[3] Vinoski, S. (2006). Advanced message queuing protocol. IEEE Internet Computing, 10(6), 87-89.[4] Colitti, W., Steenhaut, K., De Caro, N., Buta, B., & Dobrota, V. (2011, October). Evaluation of constrained application protocol for wireless sensor networks. In 2011 18th IEEE Workshop on Local & Metropolitan Area Networks (LANMAN) (pp. 1-6). IEEE.[5] Pardo-Castellote, G. (2003, May). Omg data-distribution service: Architectural overview. In 23rd International Conference on Distributed Computing Systems Workshops, 2003. Proceedings. (pp. 200-206). IEEE.[6] D. Thangavel, X. Ma, A. Valera, H.-X. Tan, and C. K.-Y. Tan, “Performance evaluation of mqtt and coap via a common middleware,” pp. 1–6, 2014.[7] Longo, E., Redondi, A. E. C., Cesana, M., & Manzoni, P. (2022). BORDER: A benchmarking framework for distributed MQTT brokers. IEEE Internet of Things Journal, 9(18), 17728-17740.[8] Rausch, T., Nastic, S., & Dustdar, S. (2018, April). Emma: Distributed qos-aware mqtt middleware for edge computing applications. In 2018 IEEE International Conference on Cloud Engineering (IC2E) (pp. 191-197). IEEE.[9] Bellavista, P., Foschini, L., Ghiselli, N., & Reale, A. (2019, June). MQTT-based middleware for container support in fog computing environments. In 2019 IEEE Symposium on Computers and Communications (ISCC) (pp. 1-7). IEEE.[10] Coppen, R. J., Banks, A., Gupta, R., & Cybersecurity, I. (2013). MQTT Version 3.1. 1.[11] Tantitharanukul, N., Osathanunkul, K., Hantrakul, K., Pramokchon, P., & Khoenkaw, P. (2017, March). MQTT-topics management system for sharing of open data. In 2017 International Conference on Digital Arts, Media and Technology (ICDAMT) (pp. 62-65). IEEE.[12] Hmissi, F., & Ouni, S. (2022, May). TD-MQTT: Transparent Distributed MQTT Brokers for Horizontal IoT Applications. In 2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT) (pp. 479-486). IEEE.[13] Longo, E., Redondi, A. E., Cesana, M., Arcia-Moret, A., & Manzoni, P. (2020, June). MQTT-ST: a spanning tree protocol for distributed MQTT brokers. In ICC 2020-2020 IEEE International Conference on Communications (ICC) (pp. 1-6). IEEE.[14] Staglianò, L., Longo, E., & Redondi, A. E. (2021, August). D-MQTT: design and implementation of a pub/sub broker for distributed environments. In 2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS) (pp. 1-6). IEEE.[15] Wang, Y., Zheng, Z., & Shi, C. (2022, December). Routing Distribution Model Based on Node Role for MQTT Broker. In 2022 4th International Academic Exchange Conference on Science and Technology Innovation (IAECST) (pp. 736-739). IEEE.[16] Scott, D., Gamov, V., & Klein, D. (2022). Kafka in Action. Simon and Schuster.[17] Shapira, G., Palino, T., Sivaram, R., & Petty, K. (2021). Kafka: the definitive guide. " O`Reilly Media, Inc.".[18] F. P. Junqueira, B. C. Reed, and M. Serafini, “Zab: Highperformance broadcast for primary-backup systems,” in 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN). IEEE, 2011, pp. 245–256.[19] D. Ongaro and J. Ousterhout, “In search of an understandable consensus algorithm,” in 2014 {USENIX} Annual Technical Conference ({USENIX}{ATC} 14), 2014, pp. 305–319. 描述 碩士
國立政治大學
資訊科學系
110753135資料來源 http://thesis.lib.nccu.edu.tw/record/#G0110753135 資料類型 thesis dc.contributor.advisor 廖峻鋒 zh_TW dc.contributor.advisor Liao, Chun-Feng en_US dc.contributor.author (Authors) 姚惠馨 zh_TW dc.contributor.author (Authors) Yao, Hui-Hsin en_US dc.creator (作者) 姚惠馨 zh_TW dc.creator (作者) Yao, Hui-Hsin en_US dc.date (日期) 2023 en_US dc.date.accessioned 6-Jul-2023 17:04:17 (UTC+8) - dc.date.available 6-Jul-2023 17:04:17 (UTC+8) - dc.date.issued (上傳時間) 6-Jul-2023 17:04:17 (UTC+8) - dc.identifier (Other Identifiers) G0110753135 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/145940 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 110753135 zh_TW dc.description.abstract (摘要) 近年來，物聯網（Internet of Things，IoT）技術蓬勃發展，其 IoT 裝置已無所不在。這些裝置利用物聯網傳輸協定將資料傳送至服務端進行分析來提供各種服務。其中，MQTT 是最常被使用的物聯網通訊協定之一，它在設計上非常精簡且具備一定程度的彈性，非常適合在資源受限或網路頻寬有限的物聯網環境使用。然而，隨著物聯網裝置數量的快速增長，這些裝置所產出的大量訊息往往使得單一 MQTT 伺服器不堪負荷。因此，如何分散負荷以減輕單一 MQTT 伺服器的壓力，是目前必須面臨的重要課題。分散負荷的第一步通常是使用多個 MQTT 伺服器來分散交通量，但目前 MQTT 缺乏多伺服器的輔助機制，尤其是欠缺對分散式回執機制的支援。針對這個問題，本研究提出一個 MQTT 的分散式主題（Topic）切片訊息處理架構，此架構可協調將 Topic 的交通量以切片（Sharding）方式分散在多個 MQTT 伺服器中，並且提供 Topic Binding 機制，使得分散式 MQTT Broker 機制得以實現。最後，將對提出的解決方案進行實作，並評估其系統在不同使用情境下的可行性和效能表現，以驗證可在不影響原有功能的情況下提供額外功能，改善系統資源的使用情況。 zh_TW dc.description.abstract (摘要) In recent years, the Internet of Things (IoT) technology has been flourishing, and IoT devices are ubiquitous. These devices utilize IoT communication protocols to transmit data to the server for analysis and provide various services. Among them, MQTT is one of the most commonly used IoT communication protocols. It is designed to be lightweight and flexible, making it ideal for IoT environments with limited resources or network bandwidth. However, with the rapid growth of IoT devices, the large volume of messages generated by these devices often overwhelms a single MQTT server. Therefore, how to distribute the load and alleviate the pressure on a single MQTT server has become an important issue to address.The first step in load distribution is typically to use multiple MQTT servers to distribute the traffic. However, MQTT currently lacks built-in mechanisms for multi-server support, particularly the lack of support for distributed acknowledgment mechanisms. To address this problem, this study proposes a distributed topic sharding message processing architecture for MQTT. This architecture coordinates the distribution of topic traffic among multiple MQTT servers using sharding techniques and provides a topic binding mechanism to enable a distributed MQTT broker mechanism. Finally, the proposed solution will be implemented and evaluated for its feasibility and performance in different usage scenarios to validate its ability to provide additional functionality without affecting the existing features and improve the utilization of system resources. en_US dc.description.tableofcontents 謝辭 I摘要 IIAbstract III圖目錄 V表目錄 VII第1章 緒論 11.1 研究背景 11.2 研究動機 11.3 研究目標 4第2章 技術背景與相關研究 62.1 技術背景 62.2 相關研究 11第3章 分散式主題切片訊息處理架構 153.1 設計考量 153.2 分散式主題切片訊息處理架構之設計 183.3 回執機制之設計 20第4章 系統實作 244.1 Moquette Broker 解析 244.2 分散式主題切片訊息處理架構實作 284.3 回執機制實作 33第5章 系統評估 355.1 測試環境和客戶端配置參數 355.2 實驗結果 36第6章 結論 44參考文獻 45 zh_TW dc.format.extent 3850236 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0110753135 en_US dc.subject (關鍵詞) 物聯網 zh_TW dc.subject (關鍵詞) MQTT zh_TW dc.subject (關鍵詞) 分散式架構 zh_TW dc.subject (關鍵詞) 切片機制 zh_TW dc.subject (關鍵詞) Internet of Things en_US dc.subject (關鍵詞) MQTT en_US dc.subject (關鍵詞) Distribution architecture en_US dc.subject (關鍵詞) Sharding en_US dc.title (題名) MQTT 分散式主題切片訊息處理架構 zh_TW dc.title (題名) MQTT Extensions for Distributed Topic Sharding en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] F. Hmissi and S. Ouni, “A review of application protocol enhancements for internet of things,” UBICOMM 2021 : The Fifteenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, pp. 8–13, October 2021.[2] Yassein, M. B., & Shatnawi, M. Q. (2016, September). Application layer protocols for the Internet of Things: A survey. In 2016 International Conference on Engineering & MIS (ICEMIS) (pp. 1-4). IEEE.[3] Vinoski, S. (2006). Advanced message queuing protocol. IEEE Internet Computing, 10(6), 87-89.[4] Colitti, W., Steenhaut, K., De Caro, N., Buta, B., & Dobrota, V. (2011, October). Evaluation of constrained application protocol for wireless sensor networks. In 2011 18th IEEE Workshop on Local & Metropolitan Area Networks (LANMAN) (pp. 1-6). IEEE.[5] Pardo-Castellote, G. (2003, May). Omg data-distribution service: Architectural overview. In 23rd International Conference on Distributed Computing Systems Workshops, 2003. Proceedings. (pp. 200-206). IEEE.[6] D. Thangavel, X. Ma, A. Valera, H.-X. Tan, and C. K.-Y. Tan, “Performance evaluation of mqtt and coap via a common middleware,” pp. 1–6, 2014.[7] Longo, E., Redondi, A. E. C., Cesana, M., & Manzoni, P. (2022). BORDER: A benchmarking framework for distributed MQTT brokers. IEEE Internet of Things Journal, 9(18), 17728-17740.[8] Rausch, T., Nastic, S., & Dustdar, S. (2018, April). Emma: Distributed qos-aware mqtt middleware for edge computing applications. In 2018 IEEE International Conference on Cloud Engineering (IC2E) (pp. 191-197). IEEE.[9] Bellavista, P., Foschini, L., Ghiselli, N., & Reale, A. (2019, June). MQTT-based middleware for container support in fog computing environments. In 2019 IEEE Symposium on Computers and Communications (ISCC) (pp. 1-7). IEEE.[10] Coppen, R. J., Banks, A., Gupta, R., & Cybersecurity, I. (2013). MQTT Version 3.1. 1.[11] Tantitharanukul, N., Osathanunkul, K., Hantrakul, K., Pramokchon, P., & Khoenkaw, P. (2017, March). MQTT-topics management system for sharing of open data. In 2017 International Conference on Digital Arts, Media and Technology (ICDAMT) (pp. 62-65). IEEE.[12] Hmissi, F., & Ouni, S. (2022, May). TD-MQTT: Transparent Distributed MQTT Brokers for Horizontal IoT Applications. In 2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT) (pp. 479-486). IEEE.[13] Longo, E., Redondi, A. E., Cesana, M., Arcia-Moret, A., & Manzoni, P. (2020, June). MQTT-ST: a spanning tree protocol for distributed MQTT brokers. In ICC 2020-2020 IEEE International Conference on Communications (ICC) (pp. 1-6). IEEE.[14] Staglianò, L., Longo, E., & Redondi, A. E. (2021, August). D-MQTT: design and implementation of a pub/sub broker for distributed environments. In 2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS) (pp. 1-6). IEEE.[15] Wang, Y., Zheng, Z., & Shi, C. (2022, December). Routing Distribution Model Based on Node Role for MQTT Broker. In 2022 4th International Academic Exchange Conference on Science and Technology Innovation (IAECST) (pp. 736-739). IEEE.[16] Scott, D., Gamov, V., & Klein, D. (2022). Kafka in Action. Simon and Schuster.[17] Shapira, G., Palino, T., Sivaram, R., & Petty, K. (2021). Kafka: the definitive guide. " O`Reilly Media, Inc.".[18] F. P. Junqueira, B. C. Reed, and M. Serafini, “Zab: Highperformance broadcast for primary-backup systems,” in 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN). IEEE, 2011, pp. 245–256.[19] D. Ongaro and J. Ousterhout, “In search of an understandable consensus algorithm,” in 2014 {USENIX} Annual Technical Conference ({USENIX}{ATC} 14), 2014, pp. 305–319. zh_TW
