Publications-Theses
Article View/Open
Publication Export
Google ScholarTM
NCCU Library
Citation Infomation
Related Publications in TAIR
| Title | 解耦交易：基於Kafka 的分散式支付系統 Decoupling Payments: A Distributed Payment System Based on Kafka |
| Creator | 陳亮酉 Chen, Liang-You |
| Contributor | 郭桐惟 Kuo, Tung-Wei 陳亮酉 Chen, Liang-You |
| Key Words | 分散式系統 Apache Kafka 中央銀行數位貨幣 最終一致性 Distributed System Apache Kafka Central Bank Digital Currency (CBDC) Eventual Consistency |
| Date | 2024 |
| Date Issued | 1-Jul-2024 12:30:38 (UTC+8) |
| Summary | 本論文利用Apache Kafka，提出了和以往數位貨幣不同的兩種分散
式支付系統的開發模型。我們會著重於通過分散式系統架構和一種新
的用戶資金管理模型(用於提高了平行處理的程度) 來優化性能和可擴
展性。在開發的兩種模型中，並行模型因其卓越的能力而脫穎而出，
支持每秒超過200,000 筆交易（TPS），且對硬體要求低，並展現出對
網絡挑戰和系統故障的顯著韌性。此研究通過提供一種新方法：通過
將交易記錄分段處理，能夠增加交易處理的效率和韌性，同時還能保
證銀行系統中的最終一致性。 This thesis presents the development of two models for a distributed payment system, utilizing Apache Kafka to depart from conventional central bank digital currency systems. The focus is on optimizing performance and scalability through distributed system architecture and a new model for managing user funds which improves the level of parallelism. Among the two models developed, the Concurrent model stands out for its exceptional capability, supporting over 200,000 transactions per second (TPS) with minimal hardware requirements and demonstrating significant resilience to network challenges and system faults. This research introduces a new method: by segmenting transaction records, it can enhance the efficiency and resilience of transaction processing, while also ensuring eventual consistency within the banking system. |
| 參考文獻 | [1] J. Lovejoy, C. Fields, M. Virza, T. Frederick, D. Urness, K. Karwaski, A. Brownworth, and N. Narula, “A high performance payment processing system designed for central bank digital currencies,” Cryptology ePrint Archive, 2022. [2] S. Allen, S. Čapkun, I. Eyal, G. Fanti, B. A. Ford, J. Grimmelmann, A. Juels, K. Kostiainen, S. Meiklejohn, A. Miller, et al., “Design choices for central bank digital currency: Policy and technical considerations,” tech. rep., National Bureau of Economic Research, 2020. [3] R. Auer and R. Böhme, “The technology of retail central bank digital currency,” BIS Quarterly Review, March, 2020. [4] R. Auer, G. Cornelli, and J. Frost, “Rise of the central bank digital currencies: drivers, approaches and technologies,” 2020. [5] J. Zahnentferner, “Chimeric ledgers: Translating and unifying utxo-based and account-based cryptocurrencies,” Cryptology ePrint Archive, 2018. [6] N. Garg, Apache kafka. Packt Publishing Birmingham, UK, 2013. [7] B. Bejeck, Kafka Streams in Action: Real-time apps and microservices with the Kafka Streams API. Simon and Schuster, 2018. [8] M. Seymour, Mastering Kafka Streams and ksqlDB. O’Reilly Media, 2021. [9] P. Hunt, M. Konar, F. P. Junqueira, and B. Reed, “Zookeeper: wait-free coordination for internet-scale systems.,” in USENIX annual technical conference, vol. 8, 2010. [10] G. Danezis and S. Meiklejohn, “Centrally banked cryptocurrencies,” arXiv preprint arXiv:1505.06895, 2015. [11] L. Luu, V. Narayanan, C. Zheng, K. Baweja, S. Gilbert, and P. Saxena, “A secure sharding protocol for open blockchains,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 17–30, 2016. [12] G. Wood et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum project yellow paper, vol. 151, no. 2014, pp. 1–32, 2014. [13] G. Pîrlea and I. Sergey, “Mechanising blockchain consensus,” in Proceedings of the 7th ACM SIGPLAN International Conference on Certified Programs and Proofs, pp. 78–90, 2018. [14] “CBDCs: an opportunity for the monetary system,” BIS Annual Report Economic Report, p. 65–91, 2021. [15] B. für Internationalen Zahlungsausgleich Committee on Payments and M. Infrastructures, Fast payments: Enhancing the speed and availability of retail payments. Bank for International Settlements, 2016. [16] TARGET Instant Payments Settlement user requirements. European Central Bank, 2017. [17] T2-T2S consolidation user requirements document for T2-RTGS component. European Central Bank, 2018. [18] J. Jiang and K. Lucero, “Background and implications of china’s e-cny,” U. Fla. JL & Pub. Pol’y, vol. 33, p. 237, 2022. [19] Y. Qian, “Technical aspects of cbdc in a two-tiered system,” in presentation materials presented at ITU Workshop on Standardizing Digital Fiat Currency (DFC) and its Applications, New York City, United States, pp. 18–19, 2018. [20] D. W. Patrick and T. Lyle, “Central bank digital currency: Caribbean pathways,” how the challenges of today prepare the ground for tomorrow, p. 340, 2022. [21] Work stream 3: A new solution–blockchain & eID. Eesti Pank, 2021. [22] G. Wood et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum project yellow paper, vol. 151, no. 2014, pp. 1–32, 2014. [23] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized business review, 2008. [24] P.-H. Chiang, “An empirical study on the scalability of kafka in a scenario of online payments,” Master’s thesis, National Chengchi University, 2023. [25] A. Adya, D. Myers, J. Howell, J. Elson, C. Meek, V. Khemani, S. Fulger, P. Gu, L. Bhuvanagiri, J. Hunter, et al., “Slicer:{Auto-Sharding} for datacenter applications,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 739–753, 2016. [26] J. Wang and H. Wang, “Monoxide: Scale out blockchains with asynchronous consensus zones,” in 16th USENIX symposium on networked systems design and implementation (NSDI 19), pp. 95–112, 2019. [27] J. N. Gray, “Notes on data base operating systems,” Operating systems: An advanced course, pp. 393–481, 2005. [28] L. Yang, X. Yan, and B. Wong, “Natto: Providing distributed transaction prioritization for high-contention workloads,” in Proceedings of the 2022 International Conference on Management of Data, pp. 715–729, 2022. [29] X. Yan, L. Yang, H. Zhang, X. C. Lin, B. Wong, K. Salem, and T. Brecht, “Carousel: Low-latency transaction processing for globally-distributed data,” in Proceedings of the 2018 International Conference on Management of Data, pp. 231–243, 2018. [30] J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, et al., “Spanner: Google’s globally distributed database,” ACM Transactions on Computer Systems (TOCS), vol. 31, no. 3, pp. 1–22, 2013. [31] D. Ongaro and J. Ousterhout, “In search of an understandable consensus algorithm,” in 2014 USENIX annual technical conference (USENIX ATC 14), pp. 305–319, 2014. |
| Description | 碩士 國立政治大學 資訊科學系 110753209 |
| 資料來源 | http://thesis.lib.nccu.edu.tw/record/#G0110753209 |
| Type | thesis |
| dc.contributor.advisor | 郭桐惟 | zh_TW |
| dc.contributor.advisor | Kuo, Tung-Wei | en_US |
| dc.contributor.author (Authors) | 陳亮酉 | zh_TW |
| dc.contributor.author (Authors) | Chen, Liang-You | en_US |
| dc.creator (作者) | 陳亮酉 | zh_TW |
| dc.creator (作者) | Chen, Liang-You | en_US |
| dc.date (日期) | 2024 | en_US |
| dc.date.accessioned | 1-Jul-2024 12:30:38 (UTC+8) | - |
| dc.date.available | 1-Jul-2024 12:30:38 (UTC+8) | - |
| dc.date.issued (上傳時間) | 1-Jul-2024 12:30:38 (UTC+8) | - |
| dc.identifier (Other Identifiers) | G0110753209 | en_US |
| dc.identifier.uri (URI) | https://nccur.lib.nccu.edu.tw/handle/140.119/152037 | - |
| dc.description (描述) | 碩士 | zh_TW |
| dc.description (描述) | 國立政治大學 | zh_TW |
| dc.description (描述) | 資訊科學系 | zh_TW |
| dc.description (描述) | 110753209 | zh_TW |
| dc.description.abstract (摘要) | 本論文利用Apache Kafka，提出了和以往數位貨幣不同的兩種分散 式支付系統的開發模型。我們會著重於通過分散式系統架構和一種新 的用戶資金管理模型(用於提高了平行處理的程度) 來優化性能和可擴 展性。在開發的兩種模型中，並行模型因其卓越的能力而脫穎而出， 支持每秒超過200,000 筆交易（TPS），且對硬體要求低，並展現出對 網絡挑戰和系統故障的顯著韌性。此研究通過提供一種新方法：通過 將交易記錄分段處理，能夠增加交易處理的效率和韌性，同時還能保 證銀行系統中的最終一致性。 | zh_TW |
| dc.description.abstract (摘要) | This thesis presents the development of two models for a distributed payment system, utilizing Apache Kafka to depart from conventional central bank digital currency systems. The focus is on optimizing performance and scalability through distributed system architecture and a new model for managing user funds which improves the level of parallelism. Among the two models developed, the Concurrent model stands out for its exceptional capability, supporting over 200,000 transactions per second (TPS) with minimal hardware requirements and demonstrating significant resilience to network challenges and system faults. This research introduces a new method: by segmenting transaction records, it can enhance the efficiency and resilience of transaction processing, while also ensuring eventual consistency within the banking system. | en_US |
| dc.description.tableofcontents | 誌謝 i 摘要 ii Abstract iii Contents iv List of Figures vii List of Tables ix 1 Introduction 1 2 Background 5 2.1 Apache Kafka 5 2.2 Common Fund Representation Models: UTXO & account based 7 2.3 Real-Time Distributed Banking System for Inter-bank Transfers 11 3 System Design 12 3.1 Our Fund Representation: Credit Model 13 3.2 Basic Idea 14 3.2.1 Topics and Partitions 14 3.2.2 Processors 16 3.2.3 Lifecycle of a Trade 17 3.2.4 Logic behind Partition Control 19 3.3 Limitations of the Simplified Design 21 4 Details of the System 23 4.1 Trade Design and Partitioning 23 4.1.1 Partitioning Details 24 4.1.2 Orders Topic and Serializability 26 4.2 Sequential Model 27 4.3 Concurrent Model 29 4.4 Eventual Consistency 31 5 Implementation 33 5.1 Shared Hash map 34 5.2 Choosing Avro Over JSON 37 6 Experimental Results 38 6.1 The Impact of Block Size 39 6.2 The Impact of Kafka parameters 41 6.3 The Impact of Requests Per Second (RPS) 43 6.4 The Impact of Additional Network Latency 48 6.4.1 the Impact of Block Size with Additional Network Latency 48 6.4.2 The Impact of RPS with Additional Network Latency 51 6.4.3 The Impact of Increasing Network Latency 53 6.5 Scalability 55 6.5.1 Scenario 1: Two High-End Machines 56 6.5.2 Scenario 2: Two Lower-Spec Machines 56 6.5.3 Scenario 3: Pairing a High-End with a Lower-Spec Machines 57 6.5.4 Scenario 4: Integrating Two High-End and Two Lower-Spec Machines 59 6.5.5 Analysis of Four Scenarios 60 7 Related Work 62 7.1 Central Bank Digital Currency 62 7.2 Fund Representation 64 7.3 Two-phase Commit 66 8 Conclusion 67 Reference 68 | zh_TW |
| dc.format.extent | 1006634 bytes | - |
| dc.format.mimetype | application/pdf | - |
| dc.source.uri (資料來源) | http://thesis.lib.nccu.edu.tw/record/#G0110753209 | en_US |
| dc.subject (關鍵詞) | 分散式系統 | zh_TW |
| dc.subject (關鍵詞) | Apache Kafka | zh_TW |
| dc.subject (關鍵詞) | 中央銀行數位貨幣 | zh_TW |
| dc.subject (關鍵詞) | 最終一致性 | zh_TW |
| dc.subject (關鍵詞) | Distributed System | en_US |
| dc.subject (關鍵詞) | Apache Kafka | en_US |
| dc.subject (關鍵詞) | Central Bank Digital Currency (CBDC) | en_US |
| dc.subject (關鍵詞) | Eventual Consistency | en_US |
| dc.title (題名) | 解耦交易：基於Kafka 的分散式支付系統 | zh_TW |
| dc.title (題名) | Decoupling Payments: A Distributed Payment System Based on Kafka | en_US |
| dc.type (資料類型) | thesis | en_US |
| dc.relation.reference (參考文獻) | [1] J. Lovejoy, C. Fields, M. Virza, T. Frederick, D. Urness, K. Karwaski, A. Brownworth, and N. Narula, “A high performance payment processing system designed for central bank digital currencies,” Cryptology ePrint Archive, 2022. [2] S. Allen, S. Čapkun, I. Eyal, G. Fanti, B. A. Ford, J. Grimmelmann, A. Juels, K. Kostiainen, S. Meiklejohn, A. Miller, et al., “Design choices for central bank digital currency: Policy and technical considerations,” tech. rep., National Bureau of Economic Research, 2020. [3] R. Auer and R. Böhme, “The technology of retail central bank digital currency,” BIS Quarterly Review, March, 2020. [4] R. Auer, G. Cornelli, and J. Frost, “Rise of the central bank digital currencies: drivers, approaches and technologies,” 2020. [5] J. Zahnentferner, “Chimeric ledgers: Translating and unifying utxo-based and account-based cryptocurrencies,” Cryptology ePrint Archive, 2018. [6] N. Garg, Apache kafka. Packt Publishing Birmingham, UK, 2013. [7] B. Bejeck, Kafka Streams in Action: Real-time apps and microservices with the Kafka Streams API. Simon and Schuster, 2018. [8] M. Seymour, Mastering Kafka Streams and ksqlDB. O’Reilly Media, 2021. [9] P. Hunt, M. Konar, F. P. Junqueira, and B. Reed, “Zookeeper: wait-free coordination for internet-scale systems.,” in USENIX annual technical conference, vol. 8, 2010. [10] G. Danezis and S. Meiklejohn, “Centrally banked cryptocurrencies,” arXiv preprint arXiv:1505.06895, 2015. [11] L. Luu, V. Narayanan, C. Zheng, K. Baweja, S. Gilbert, and P. Saxena, “A secure sharding protocol for open blockchains,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 17–30, 2016. [12] G. Wood et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum project yellow paper, vol. 151, no. 2014, pp. 1–32, 2014. [13] G. Pîrlea and I. Sergey, “Mechanising blockchain consensus,” in Proceedings of the 7th ACM SIGPLAN International Conference on Certified Programs and Proofs, pp. 78–90, 2018. [14] “CBDCs: an opportunity for the monetary system,” BIS Annual Report Economic Report, p. 65–91, 2021. [15] B. für Internationalen Zahlungsausgleich Committee on Payments and M. Infrastructures, Fast payments: Enhancing the speed and availability of retail payments. Bank for International Settlements, 2016. [16] TARGET Instant Payments Settlement user requirements. European Central Bank, 2017. [17] T2-T2S consolidation user requirements document for T2-RTGS component. European Central Bank, 2018. [18] J. Jiang and K. Lucero, “Background and implications of china’s e-cny,” U. Fla. JL & Pub. Pol’y, vol. 33, p. 237, 2022. [19] Y. Qian, “Technical aspects of cbdc in a two-tiered system,” in presentation materials presented at ITU Workshop on Standardizing Digital Fiat Currency (DFC) and its Applications, New York City, United States, pp. 18–19, 2018. [20] D. W. Patrick and T. Lyle, “Central bank digital currency: Caribbean pathways,” how the challenges of today prepare the ground for tomorrow, p. 340, 2022. [21] Work stream 3: A new solution–blockchain & eID. Eesti Pank, 2021. [22] G. Wood et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum project yellow paper, vol. 151, no. 2014, pp. 1–32, 2014. [23] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized business review, 2008. [24] P.-H. Chiang, “An empirical study on the scalability of kafka in a scenario of online payments,” Master’s thesis, National Chengchi University, 2023. [25] A. Adya, D. Myers, J. Howell, J. Elson, C. Meek, V. Khemani, S. Fulger, P. Gu, L. Bhuvanagiri, J. Hunter, et al., “Slicer:{Auto-Sharding} for datacenter applications,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 739–753, 2016. [26] J. Wang and H. Wang, “Monoxide: Scale out blockchains with asynchronous consensus zones,” in 16th USENIX symposium on networked systems design and implementation (NSDI 19), pp. 95–112, 2019. [27] J. N. Gray, “Notes on data base operating systems,” Operating systems: An advanced course, pp. 393–481, 2005. [28] L. Yang, X. Yan, and B. Wong, “Natto: Providing distributed transaction prioritization for high-contention workloads,” in Proceedings of the 2022 International Conference on Management of Data, pp. 715–729, 2022. [29] X. Yan, L. Yang, H. Zhang, X. C. Lin, B. Wong, K. Salem, and T. Brecht, “Carousel: Low-latency transaction processing for globally-distributed data,” in Proceedings of the 2018 International Conference on Management of Data, pp. 231–243, 2018. [30] J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, et al., “Spanner: Google’s globally distributed database,” ACM Transactions on Computer Systems (TOCS), vol. 31, no. 3, pp. 1–22, 2013. [31] D. Ongaro and J. Ousterhout, “In search of an understandable consensus algorithm,” in 2014 USENIX annual technical conference (USENIX ATC 14), pp. 305–319, 2014. | zh_TW |