學術產出-Theses
Article View/Open
Publication Export
-
題名 Event Sourcing與CQRS架構下的服務穩定機制
Improving Service Stability for Event Sourcing and CQRS Architecture作者 鄭宇軒
CHENG, YU-HSUAN貢獻者 廖峻鋒
Liao,Chun-Feng
鄭宇軒
CHENG,YU-HSUAN關鍵詞 錯誤回復
錯誤偵測
命令查詢職責分離模式
事件溯源
Failure Recovery
Failure Detection
CQRS
Event Sourcing日期 2021 上傳時間 4-Aug-2021 15:40:38 (UTC+8) 摘要 隨著雲端運算的興起，分散式系統的重要性逐漸受到重視，大量複雜的系統與服務逐漸成為企業的常態。而容器技術的普及，又為新一波軟體部署與管理方式帶來了前所未有的革命，其中以Docker和Kubernetes 等平台的出現，讓微服務的設計概念得以落實。微服務的設計思維不僅能夠降低系統中服務與服務之間的耦合性，也能夠達成跨程式語言的開發模式。傳統資料存取模式(增刪修改)，在面對大量使用者同時寫入的情況下，經常造成單一失效及效能問題。使用命令與查詢責任分離模式 (CQRS, Command Query Responsibility Segregation) 與事件溯源 (ES, Event Sourcing)架構，由軟體層面增加了系統的可擴張性，然而這些機制在基礎設施層面上目前仍欠缺完整的錯誤偵測與回復功能，且面對複數使用者同時對資料庫進行改寫時，亦會遇到一致性的問題。本論文採用CQRS/Event Sourcing架構的系統，設計一個完整且自主的錯誤偵測與回復機制，亦可處理並行讀寫的問題，並透過實作與數據分析驗證此設計機制的可行性。
As the Cloud computing raise rapidly, distributed system gradually attach great importance to a lot of software engineers around the world. Having a great range of complex business’s logic system and services become enterprise’s new normal condition. With the universal of container technology, the new wave of unprecedented revolution has spread into all kinds of software deployments and software managements. The appearance of Kubernetes and Docker platforms makes Microservice pattern have the chance to implement in enterprise’s business system. The design concept of Microservice pattern not only can reduce the coupling between service and service but also can allow user to develop system in multi-platform environment. In the traditional method for data access(CRUD), when a lot of users’ update data simultaneously, this will often cause single failure and low performance. Using Command Query Responsibility Segregation(CQRS) and Event Sourcing can enhance the scalability of system in software level. However, these mechanisms in infrastructure aspect still lack of complete failure detection and failure recovery. Additionally, when facing a huge amount of users update the database at the same time, will also encounter consistency problem. This essay will adopt the pattern of CQRS/Event Sourcing Architecture to design a complete failure detection and automatic failure recovery, this also can solve the problem of data access in parallel. We verify the availability of this system through implementation and data analysis.參考文獻 [1] C. Richardson, Microservices PatternsWith examples in Java. Published 2017 by Manning Publications, October 2018.[2] M. T. Nygard, Release It!: Design and Deploy Production-Ready Software. Pragmatic Bookshelf, 2018.[3] Z. Long, "Improvement and implementation of a high performance CQRS architecture," in 2017 International Conference on Robots & Intelligent System (ICRIS), 2017: IEEE, pp. 170-173.[4] B. Meyer, Object-oriented software construction. 1997.[5] R. Martin, Agile Software Development, Principles, Patterns, and Practices. Prentice Hall, 2002.[6] G. Young, CQRS Documents. 2010.[7] M. Fowler. "Event Sourcing." https://martinfowler.com/eaaDev/EventSourcing.html (accessed.[8] M. T. Nygard, Release It!: Design and Deploy Production-Ready Software. 2007.[9] M. J. Fischer, N. A. Lynch, and M. S. Paterson, "Impossibility of distributed consensus with one faulty process," Journal of the ACM (JACM), vol. 32, no. 2, pp. 374-382, 1985.[10] S. Gilbert and N. Lynch, "Perspectives on the CAP Theorem," Computer, vol. 45, no. 2, pp. 30-36, 2012.[11] D. Ongaro and J. Ousterhout, "In search of an understandable consensus algorithm (extended version)," ed: Tech Report. May, 2014. http://ramcloud. stanford. edu/Raft. pdf, 2013.[12] M. Overeem, M. Spoor, and S. Jansen, "The dark side of event sourcing: Managing data conversion," in 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), 2017: IEEE, pp. 193-204.[13] D. Meißner, B. Erb, and F. Kargl, "Performance Engineering in Distributed Event-sourced Systems," presented at the Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems, Hamilton, New Zealand, 2018. [Online]. Available: https://doi.org/10.1145/3210284.3219770.[14] J. Rybicki, "Application of Event Sourcing in Research Data Management," 2018 : The Fourth International Conference on Big Data, Small Data, Linked Data and Open Data, pp. 22-26, 2018.[15] S. Han and J.-i. Choi, "V2X-Based Event Acquisition and Reproduction Architecture with Event-Sourcing," in Proceedings of 2020 the 6th International Conference on Computing and Data Engineering, 2020, pp. 164-167.[16] Y. Zhong, W. Li, and J. Wang, "Using Event Sourcing and CQRS to Build a High Performance Point Trading System," in Proceedings of the 2019 5th International Conference on E-Business and Applications, 2019, pp. 16-19.[17] A. Bellemare, Building Event-Driven Microservices: Leveraging Organizational Data at Scale. O`Reilly, 2020-07.[18] M. Barnkob and J. Krukow, "Event Sourcing and Command Query Responsibility Segregation Reliability Properties," Computer Science University of Aarhus.—2018.—C, pp. 21-39, 2018. 描述 碩士
國立政治大學
資訊科學系
108753121資料來源 http://thesis.lib.nccu.edu.tw/record/#G0108753121 資料類型 thesis dc.contributor.advisor 廖峻鋒 zh_TW dc.contributor.advisor Liao,Chun-Feng en_US dc.contributor.author (Authors) 鄭宇軒 zh_TW dc.contributor.author (Authors) CHENG,YU-HSUAN en_US dc.creator (作者) 鄭宇軒 zh_TW dc.creator (作者) CHENG, YU-HSUAN en_US dc.date (日期) 2021 en_US dc.date.accessioned 4-Aug-2021 15:40:38 (UTC+8) - dc.date.available 4-Aug-2021 15:40:38 (UTC+8) - dc.date.issued (上傳時間) 4-Aug-2021 15:40:38 (UTC+8) - dc.identifier (Other Identifiers) G0108753121 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/136487 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 108753121 zh_TW dc.description.abstract (摘要) 隨著雲端運算的興起，分散式系統的重要性逐漸受到重視，大量複雜的系統與服務逐漸成為企業的常態。而容器技術的普及，又為新一波軟體部署與管理方式帶來了前所未有的革命，其中以Docker和Kubernetes 等平台的出現，讓微服務的設計概念得以落實。微服務的設計思維不僅能夠降低系統中服務與服務之間的耦合性，也能夠達成跨程式語言的開發模式。傳統資料存取模式(增刪修改)，在面對大量使用者同時寫入的情況下，經常造成單一失效及效能問題。使用命令與查詢責任分離模式 (CQRS, Command Query Responsibility Segregation) 與事件溯源 (ES, Event Sourcing)架構，由軟體層面增加了系統的可擴張性，然而這些機制在基礎設施層面上目前仍欠缺完整的錯誤偵測與回復功能，且面對複數使用者同時對資料庫進行改寫時，亦會遇到一致性的問題。本論文採用CQRS/Event Sourcing架構的系統，設計一個完整且自主的錯誤偵測與回復機制，亦可處理並行讀寫的問題，並透過實作與數據分析驗證此設計機制的可行性。 zh_TW dc.description.abstract (摘要) As the Cloud computing raise rapidly, distributed system gradually attach great importance to a lot of software engineers around the world. Having a great range of complex business’s logic system and services become enterprise’s new normal condition. With the universal of container technology, the new wave of unprecedented revolution has spread into all kinds of software deployments and software managements. The appearance of Kubernetes and Docker platforms makes Microservice pattern have the chance to implement in enterprise’s business system. The design concept of Microservice pattern not only can reduce the coupling between service and service but also can allow user to develop system in multi-platform environment. In the traditional method for data access(CRUD), when a lot of users’ update data simultaneously, this will often cause single failure and low performance. Using Command Query Responsibility Segregation(CQRS) and Event Sourcing can enhance the scalability of system in software level. However, these mechanisms in infrastructure aspect still lack of complete failure detection and failure recovery. Additionally, when facing a huge amount of users update the database at the same time, will also encounter consistency problem. This essay will adopt the pattern of CQRS/Event Sourcing Architecture to design a complete failure detection and automatic failure recovery, this also can solve the problem of data access in parallel. We verify the availability of this system through implementation and data analysis. en_US dc.description.tableofcontents 摘要 I第一章 緒論 - 1 -1.1研究背景 - 1 -1.2研究動機 - 3 -1.3研究貢獻 - 4 -1.4 論文架構 - 4 -第二章 技術背景與相關研究 - 5 -2.1 命令與查詢責任隔離(COMMAND QUERY RESPONSIBILITY SEGREGATION) - 5 -2.2 EVENT SOURCING - 7 -2.3 CIRCUIT BREAKER - 9 -2.4 MESSAGE-ORIENTED MIDDLEWARE(MOM) - 12 -2.5 RAFT演算法 - 14 -2.6相關研究 - 19 -第三章 系統設計 - 21 -3.1系統架構說明 - 21 -3.1.1 系統架構 - 22 -3.1.2 Service端 & Watch Dog - 24 -3.1.3 Event Listener, Event Store, 和Recovery Service機制 - 25 -3.1.4 CQRS運作流程解說 - 26 -3.2 強化集中式監控系統(WATCHDOG) - 28 -3.2.1 WatchDog 機制解說 - 28 -3.3 增強因單一失效而造成崩潰的分散式系統可用性(CIRCUIT BREAKER) - 31 -3.3.1 Circuit Breaker - 32 -3.4 排除EVENT SOURCING系統交易中可能出現的競爭危害 - 35 -3.4.1 交易未寫入情況(Race condition problem) - 35 -3.4.2 交易retry機制 - 36 -3.5 錯誤偵測及錯誤回復機制 - 37 -3.5.1 Service端未回應(Service A 失效) - 37 -3.5.2 Service 重啟機制 (Failure recovery) - 38 -3.6 使用場景 - 40 -第四章 實驗測試與討論 - 42 -4.1 實驗總覽 - 42 -4.2 系統錯誤回復效能 - 42 -4.3 WATCHDOG 共識演算法恢復效能 - 44 -4.4 RACE CONDITION PROBLEM 解決效能 - 46 -4.5 討論 - 48 -第五章 結論 - 50 -參考文獻 - 52 - zh_TW dc.format.extent 2245925 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0108753121 en_US dc.subject (關鍵詞) 錯誤回復 zh_TW dc.subject (關鍵詞) 錯誤偵測 zh_TW dc.subject (關鍵詞) 命令查詢職責分離模式 zh_TW dc.subject (關鍵詞) 事件溯源 zh_TW dc.subject (關鍵詞) Failure Recovery en_US dc.subject (關鍵詞) Failure Detection en_US dc.subject (關鍵詞) CQRS en_US dc.subject (關鍵詞) Event Sourcing en_US dc.title (題名) Event Sourcing與CQRS架構下的服務穩定機制 zh_TW dc.title (題名) Improving Service Stability for Event Sourcing and CQRS Architecture en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] C. Richardson, Microservices PatternsWith examples in Java. Published 2017 by Manning Publications, October 2018.[2] M. T. Nygard, Release It!: Design and Deploy Production-Ready Software. Pragmatic Bookshelf, 2018.[3] Z. Long, "Improvement and implementation of a high performance CQRS architecture," in 2017 International Conference on Robots & Intelligent System (ICRIS), 2017: IEEE, pp. 170-173.[4] B. Meyer, Object-oriented software construction. 1997.[5] R. Martin, Agile Software Development, Principles, Patterns, and Practices. Prentice Hall, 2002.[6] G. Young, CQRS Documents. 2010.[7] M. Fowler. "Event Sourcing." https://martinfowler.com/eaaDev/EventSourcing.html (accessed.[8] M. T. Nygard, Release It!: Design and Deploy Production-Ready Software. 2007.[9] M. J. Fischer, N. A. Lynch, and M. S. Paterson, "Impossibility of distributed consensus with one faulty process," Journal of the ACM (JACM), vol. 32, no. 2, pp. 374-382, 1985.[10] S. Gilbert and N. Lynch, "Perspectives on the CAP Theorem," Computer, vol. 45, no. 2, pp. 30-36, 2012.[11] D. Ongaro and J. Ousterhout, "In search of an understandable consensus algorithm (extended version)," ed: Tech Report. May, 2014. http://ramcloud. stanford. edu/Raft. pdf, 2013.[12] M. Overeem, M. Spoor, and S. Jansen, "The dark side of event sourcing: Managing data conversion," in 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), 2017: IEEE, pp. 193-204.[13] D. Meißner, B. Erb, and F. Kargl, "Performance Engineering in Distributed Event-sourced Systems," presented at the Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems, Hamilton, New Zealand, 2018. [Online]. Available: https://doi.org/10.1145/3210284.3219770.[14] J. Rybicki, "Application of Event Sourcing in Research Data Management," 2018 : The Fourth International Conference on Big Data, Small Data, Linked Data and Open Data, pp. 22-26, 2018.[15] S. Han and J.-i. Choi, "V2X-Based Event Acquisition and Reproduction Architecture with Event-Sourcing," in Proceedings of 2020 the 6th International Conference on Computing and Data Engineering, 2020, pp. 164-167.[16] Y. Zhong, W. Li, and J. Wang, "Using Event Sourcing and CQRS to Build a High Performance Point Trading System," in Proceedings of the 2019 5th International Conference on E-Business and Applications, 2019, pp. 16-19.[17] A. Bellemare, Building Event-Driven Microservices: Leveraging Organizational Data at Scale. O`Reilly, 2020-07.[18] M. Barnkob and J. Krukow, "Event Sourcing and Command Query Responsibility Segregation Reliability Properties," Computer Science University of Aarhus.—2018.—C, pp. 21-39, 2018. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202101070 en_US
