學術產出-Theses
Article View/Open
Publication Export
-
題名 具有多重隔離層級的MQTT佇列式分散交易處理機制
Middleware Support for Queued and Distributed Transactions with Multiple Isolation Levels based on MQTT作者 莊崴宇
Chuang, Wei-Yu貢獻者 廖峻鋒
Liao, Chun-Feng
莊崴宇
Chuang, Wei-Yu關鍵詞 佇列式交易
分散式交易
隔離層級
同步協調核心
Queued Transaction
Distributed Transaction
Isolation Level
Coordination Kernel日期 2023 上傳時間 6-Jul-2023 17:04:06 (UTC+8) 摘要 隨著物聯網(Internet of Things)設備與技術的發展，一個應用程式層級的使用者指令往往涉及多個資料層級的操作，為確保資料變動的一致性，交易(Transaction)機制的重要程度也隨之提升。然而，與一般企業系統相比，物聯網的交易機制較少被探討著墨。過去，實驗室團隊曾探討物聯網中藉由MQTT實現的佇列分散式交易，但MQTT Broker導致交易成員間狀態與完成時間不明。並且，目前僅以簡單的鎖(Lock)機制避免交易並行時產生Race Condition問題，然而，鎖機制缺少排序功能，導致系統整體訊息量因反覆發送請求增加，且破壞FCFS的順序，產生飢餓(Starving)效應。此外，目前交易系統為確保資料一致性，大多選擇以資料庫隔離層級中的Repeatable Read, Serializable層級控管交易的進行，導致系統效能降低。本研究將藉由同步協調核心，針對上述三個議題提出改善方法，首先，提出結合同步協調核心的二階段提交機制，讓交易成員能透過同步協調核心取得交易資訊，解決過程中交易資訊不明的問題，二是提出結合同步協調核心的具次序性分散式鎖機制，使鎖機制的順序與交易請求順序相同，改善因順序因素而產生的訊息量、飢餓效應問題，第三是基於同步協調核心實現多重隔離層級，增加系統於不同情形應用的彈性。最後，針對上述三個改善方案進行實作，並以交易結果、訊息量與系統效能等指標進行實驗比較，期望能改善MQTT端點間的佇列分散式交易，並促進其發展。
With the development of Internet of Things (IoT) devices and technologies, a user command at the application level often involves operations at multiple data levels. To ensure data consistency, the transaction mechanisms have been more important. However, compared to general enterprise systems, the transaction mechanisms in IoT have been less explored. Previous research has investigated queued and distributed transaction implemented through MQTT, but MQTT broker leads to uncertainty in the states and completion times among transaction members. Currently, the lock mechanism to avoid race condition during concurrent transactions lacks sorting functionality, resulting in an increase in overall message volume due to repeated request transmission and disrupting the First-Come-First-Served (FCFS) order, leading to starvation effects. Additionally, in order to ensure data correctness, current transaction systems mostly rely on isolation levels such as Repeatable Read and Serializable in the database, which will degrade system performance. This paper proposes solutions through the coordination kernel to the three issues mentioned. Firstly, a two-phase commit mechanism combined with coordination kernel is introduced to allow transaction members to obtain transaction information through the coordination kernel. Secondly, a sequentially distributed lock mechanism combined with coordination kernel is proposed to ensure that the order of the lock mechanism aligns with the order of transaction requests. Thirdly, the implementation of multiple isolation levels based of coordination kernel is proposed to increase the flexibility of system application in different scenarios. Finally, this paper implements the proposed solutions and evaluates transaction results, message volume, and system performance of the system.參考文獻 [1] Jergler, M., Zhang, K., and Jacobsen, H.-A. (2018). Multi-client transactions in distributed publish/subscribe systems. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 120-131. IEEE.[2] Michels, J., Hare, K., Kulkarni, K., Zuzarte, C., Liu, Z. H., Hammerschmidt, B., and Zemke, F. (2018). The new and improved sql: 2016 standard. ACM SIGMOD Record, 47(2):51–60.[3] Vargas, L., Pesonen, L. I., Gudes, E., and Bacon, J. (2007). Transactions in content-based publish/subscribe middleware. In 27th International Conference on Distributed Computing Systems Workshops (ICDCSW’07), pages 68–68. IEEE.[4] Herlihy, M. P. and Wing, J. M. (1990). Linearizability: A correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems (TOPLAS), 12(3):463-492.[5] Banks, A. and Gupta, R. (2014). Mqtt version 3.1.1. OASIS standard, 29:89.[6] Banks, A., Briggs, E., Borgendale, K., and Gupta, R. (2019). Mqtt version 5. OASIS standard.[7] Flavio, J., & Benjamin, R. Zookeeper: Distributed Process Coordination.[8] Zare, A. and Iqbal, M. T. (2020). Low-cost esp32, raspberry pi, node-red, and mqtt protocol based scada system. In 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), pages 1–5. IEEE.[9] Jo, H.-C. and Jin, H.-W. (2015). Adaptive periodic communication over mqtt for large-scale cyber-physical systems. In Cyber-Physical Systems, Networks, and Applications (CPSNA), 2015 IEEE 3rd International Conference on, pages 66– 69. IEEE.[10] Hmissi, F. and Ouni, S. (2022). Td-mqtt: Transparent distributed mqtt brokers for horizontal iot applications. In 2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), pages 479–486. IEEE.[11] Dikii, D. I. (2020). Remote access control model for mqtt protocol. In 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), pages 288–291. IEEE.[12] Ganguly, B. and Chatterjee, A. (2020). Mqtt protocol based extensive smart motor control for electric vehicular application. In 2020 IEEE 7th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), pages 1–5. IEEE.[13] Kawaguchi, R. and Bandai, M. (2020). Edge based mqtt broker architecture for geographical iot applications. In 2020 International Conference on Information Networking (ICOIN), pages 232–235. IEEE.[14] Bernstein, P. and Newcomer, E. (2009). Principles of Transaction Processing. The Morgan Kaufmann Series in Data Management Systems. Elsevier Science.[15] Tai, S. and Rouvellou, I. (2000). Strategies for integrating messaging and distributed object transactions. In IFIP/ACM International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 308–330. Springer.[16] Liebig, C. and Tai, S. (2001). Middleware mediated transactions. In Proceedings 3rd International Symposium on Distributed Objects and Applications, pages 340–350. IEEE.[17] Hunt, P., Konar, M., Junqueira, F. P., and Reed, B. (2010). Zookeeper: Wait-free coordination for internet-scale systems. In 2010 USENIX Annual Technical Conference (USENIX ATC 10).[18] Goel, L. B. and Majumdar, R. (2015). Handling mutual exclusion in a distributed application through zookeeper. In 2015 International Conference on Advances in Computer Engineering and Applications, pages 457–460. IEEE.[19] Song, M., Luo, G., and Haihong, E. (2016). A service discovery system based on zookeeper with priority load balance strategy. In 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), pages 117–119. IEEE.[20] Chun-Feng Liao and Peng-Yu Chen, “ROSA: Resource-Oriented Service Management Schemes for Web of Things in a Smart Home,” in Sensors, Vol.17, No.10, 2159, MDPI AG, 2017.[21] Anon et.al. (1985). A Measure of Transaction Processing Power. In Tandem Technical Report 85.2 .[22] Levine, C., Gray, J., Kiss, S., & Kohler, W. (1993). The evolution of TPC-benchmarks: Why TPC-A and TPC-B are obsolete. Digital Equipment Corporation, San Francisco, CA SFSC Technical Report, 93. 描述 碩士
國立政治大學
資訊科學系
110753117資料來源 http://thesis.lib.nccu.edu.tw/record/#G0110753117 資料類型 thesis dc.contributor.advisor 廖峻鋒 zh_TW dc.contributor.advisor Liao, Chun-Feng en_US dc.contributor.author (Authors) 莊崴宇 zh_TW dc.contributor.author (Authors) Chuang, Wei-Yu en_US dc.creator (作者) 莊崴宇 zh_TW dc.creator (作者) Chuang, Wei-Yu en_US dc.date (日期) 2023 en_US dc.date.accessioned 6-Jul-2023 17:04:06 (UTC+8) - dc.date.available 6-Jul-2023 17:04:06 (UTC+8) - dc.date.issued (上傳時間) 6-Jul-2023 17:04:06 (UTC+8) - dc.identifier (Other Identifiers) G0110753117 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/145939 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 110753117 zh_TW dc.description.abstract (摘要) 隨著物聯網(Internet of Things)設備與技術的發展，一個應用程式層級的使用者指令往往涉及多個資料層級的操作，為確保資料變動的一致性，交易(Transaction)機制的重要程度也隨之提升。然而，與一般企業系統相比，物聯網的交易機制較少被探討著墨。過去，實驗室團隊曾探討物聯網中藉由MQTT實現的佇列分散式交易，但MQTT Broker導致交易成員間狀態與完成時間不明。並且，目前僅以簡單的鎖(Lock)機制避免交易並行時產生Race Condition問題，然而，鎖機制缺少排序功能，導致系統整體訊息量因反覆發送請求增加，且破壞FCFS的順序，產生飢餓(Starving)效應。此外，目前交易系統為確保資料一致性，大多選擇以資料庫隔離層級中的Repeatable Read, Serializable層級控管交易的進行，導致系統效能降低。本研究將藉由同步協調核心，針對上述三個議題提出改善方法，首先，提出結合同步協調核心的二階段提交機制，讓交易成員能透過同步協調核心取得交易資訊，解決過程中交易資訊不明的問題，二是提出結合同步協調核心的具次序性分散式鎖機制，使鎖機制的順序與交易請求順序相同，改善因順序因素而產生的訊息量、飢餓效應問題，第三是基於同步協調核心實現多重隔離層級，增加系統於不同情形應用的彈性。最後，針對上述三個改善方案進行實作，並以交易結果、訊息量與系統效能等指標進行實驗比較，期望能改善MQTT端點間的佇列分散式交易，並促進其發展。 zh_TW dc.description.abstract (摘要) With the development of Internet of Things (IoT) devices and technologies, a user command at the application level often involves operations at multiple data levels. To ensure data consistency, the transaction mechanisms have been more important. However, compared to general enterprise systems, the transaction mechanisms in IoT have been less explored. Previous research has investigated queued and distributed transaction implemented through MQTT, but MQTT broker leads to uncertainty in the states and completion times among transaction members. Currently, the lock mechanism to avoid race condition during concurrent transactions lacks sorting functionality, resulting in an increase in overall message volume due to repeated request transmission and disrupting the First-Come-First-Served (FCFS) order, leading to starvation effects. Additionally, in order to ensure data correctness, current transaction systems mostly rely on isolation levels such as Repeatable Read and Serializable in the database, which will degrade system performance. This paper proposes solutions through the coordination kernel to the three issues mentioned. Firstly, a two-phase commit mechanism combined with coordination kernel is introduced to allow transaction members to obtain transaction information through the coordination kernel. Secondly, a sequentially distributed lock mechanism combined with coordination kernel is proposed to ensure that the order of the lock mechanism aligns with the order of transaction requests. Thirdly, the implementation of multiple isolation levels based of coordination kernel is proposed to increase the flexibility of system application in different scenarios. Finally, this paper implements the proposed solutions and evaluates transaction results, message volume, and system performance of the system. en_US dc.description.tableofcontents 謝辭 I摘要 IIAbstract III目錄 IV圖目錄 VI表目錄 VII第一章 緒論 11.1 研究背景 11.2 研究動機 31.3 研究目標 5第二章 技術背景與相關研究 72.1 技術背景 72.2 相關研究 9第三章 系統設計 113.1 系統架構 113.2 交易模型表示式 133.3 具有多重隔離層級的佇列式分散交易機制 143.4 資料庫架設 24第四章 系統實作 26第五章 系統評估 275.1 Isolation Fault實驗 285.2 交易運作延遲實驗 315.3 訊息量差異 34第六章 結論 40參考文獻 41 zh_TW dc.format.extent 1798097 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0110753117 en_US dc.subject (關鍵詞) 佇列式交易 zh_TW dc.subject (關鍵詞) 分散式交易 zh_TW dc.subject (關鍵詞) 隔離層級 zh_TW dc.subject (關鍵詞) 同步協調核心 zh_TW dc.subject (關鍵詞) Queued Transaction en_US dc.subject (關鍵詞) Distributed Transaction en_US dc.subject (關鍵詞) Isolation Level en_US dc.subject (關鍵詞) Coordination Kernel en_US dc.title (題名) 具有多重隔離層級的MQTT佇列式分散交易處理機制 zh_TW dc.title (題名) Middleware Support for Queued and Distributed Transactions with Multiple Isolation Levels based on MQTT en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Jergler, M., Zhang, K., and Jacobsen, H.-A. (2018). Multi-client transactions in distributed publish/subscribe systems. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 120-131. IEEE.[2] Michels, J., Hare, K., Kulkarni, K., Zuzarte, C., Liu, Z. H., Hammerschmidt, B., and Zemke, F. (2018). The new and improved sql: 2016 standard. ACM SIGMOD Record, 47(2):51–60.[3] Vargas, L., Pesonen, L. I., Gudes, E., and Bacon, J. (2007). Transactions in content-based publish/subscribe middleware. In 27th International Conference on Distributed Computing Systems Workshops (ICDCSW’07), pages 68–68. IEEE.[4] Herlihy, M. P. and Wing, J. M. (1990). Linearizability: A correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems (TOPLAS), 12(3):463-492.[5] Banks, A. and Gupta, R. (2014). Mqtt version 3.1.1. OASIS standard, 29:89.[6] Banks, A., Briggs, E., Borgendale, K., and Gupta, R. (2019). Mqtt version 5. OASIS standard.[7] Flavio, J., & Benjamin, R. Zookeeper: Distributed Process Coordination.[8] Zare, A. and Iqbal, M. T. (2020). Low-cost esp32, raspberry pi, node-red, and mqtt protocol based scada system. In 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), pages 1–5. IEEE.[9] Jo, H.-C. and Jin, H.-W. (2015). Adaptive periodic communication over mqtt for large-scale cyber-physical systems. In Cyber-Physical Systems, Networks, and Applications (CPSNA), 2015 IEEE 3rd International Conference on, pages 66– 69. IEEE.[10] Hmissi, F. and Ouni, S. (2022). Td-mqtt: Transparent distributed mqtt brokers for horizontal iot applications. In 2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), pages 479–486. IEEE.[11] Dikii, D. I. (2020). Remote access control model for mqtt protocol. In 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), pages 288–291. IEEE.[12] Ganguly, B. and Chatterjee, A. (2020). Mqtt protocol based extensive smart motor control for electric vehicular application. In 2020 IEEE 7th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), pages 1–5. IEEE.[13] Kawaguchi, R. and Bandai, M. (2020). Edge based mqtt broker architecture for geographical iot applications. In 2020 International Conference on Information Networking (ICOIN), pages 232–235. IEEE.[14] Bernstein, P. and Newcomer, E. (2009). Principles of Transaction Processing. The Morgan Kaufmann Series in Data Management Systems. Elsevier Science.[15] Tai, S. and Rouvellou, I. (2000). Strategies for integrating messaging and distributed object transactions. In IFIP/ACM International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 308–330. Springer.[16] Liebig, C. and Tai, S. (2001). Middleware mediated transactions. In Proceedings 3rd International Symposium on Distributed Objects and Applications, pages 340–350. IEEE.[17] Hunt, P., Konar, M., Junqueira, F. P., and Reed, B. (2010). Zookeeper: Wait-free coordination for internet-scale systems. In 2010 USENIX Annual Technical Conference (USENIX ATC 10).[18] Goel, L. B. and Majumdar, R. (2015). Handling mutual exclusion in a distributed application through zookeeper. In 2015 International Conference on Advances in Computer Engineering and Applications, pages 457–460. IEEE.[19] Song, M., Luo, G., and Haihong, E. (2016). A service discovery system based on zookeeper with priority load balance strategy. In 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), pages 117–119. IEEE.[20] Chun-Feng Liao and Peng-Yu Chen, “ROSA: Resource-Oriented Service Management Schemes for Web of Things in a Smart Home,” in Sensors, Vol.17, No.10, 2159, MDPI AG, 2017.[21] Anon et.al. (1985). A Measure of Transaction Processing Power. In Tandem Technical Report 85.2 .[22] Levine, C., Gray, J., Kiss, S., & Kohler, W. (1993). The evolution of TPC-benchmarks: Why TPC-A and TPC-B are obsolete. Digital Equipment Corporation, San Francisco, CA SFSC Technical Report, 93. zh_TW
