Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 電力監控資料傳輸之一致性與可靠性問題研究—以A公司電力調度平台架構為例
Consistency and Availability Problems of Electricity Monitoring Data Transmission – A Case Study of an Electricity Dispatch Platform of A Company
作者 黃俊鈞
Huang, Chun-Chun
貢獻者 陳恭<br>廖峻鋒
Chen, Kung<br>Liao, Chun-Feng
黃俊鈞
Huang, Chun-Chun
關鍵詞 電力監控
分散式系統
一致性
可靠性
Electricity monitoring
Distributed systems
Consistency
Availability
日期 2022
上傳時間 5-Oct-2022 09:09:19 (UTC+8)
摘要 在電力交易市場裡,台電公司會使用各個交易群組回傳之電表監控資料作為即時調度、月度結算的依據。但電表的監控資料卻有著幾個根本的問題。首先,傳輸距離長,但Edge端的網路可靠性不足,造成了明顯的單一脆弱點問題;第二,資料本身缺少完整性、不可否認性的驗證措施,然而這是智慧電表普遍的情形,目前已有研究進行改善;第三,資料模型缺乏一致性的保證,這使得不同端點間對資料筆數及時序關聯認定會有出入。因此本研究便希望探究並解決這幾個問題,並藉由這個研究過程,來深入了解資料一致性的本質,以及如何設計跨地域性的資料傳輸架構。
     對此,本研究提出三個機制來解決第一項與第三項問題:第一是藉由建立高可用的Edge端網路以增加傳輸路線,減少Edge端斷線的機率;第二是藉由建立資料間鏈結式關聯,如:時序間的因果關聯,使得Cloud端得以掌握與Edge端資料的數量與次序差異,確保一致性;第三是使用Piggyback的機制進行補值,以利當缺值發生時,可透過自動補值的方式將Cloud端資料補足。最後透過兩項網路面的實驗情境,驗證在這三項機制相互配合下,資料能自動地補足,以達成最終一致性的目標,並使得傳輸中的Edge端DataSender服務失效時,備援的另一組服務就能在短時間內接手並開始回傳資料,以達成高可用性的目標。
Taiwan Power Company (TPC) uses the Electricity Monitoring Data for dispatching ancillary services and the calculating fee of Transaction Groups monthly. However, these data read from AMI on the Edge side have several critical issues. First, the network of the Edge side is not reliable for long-range transmission, because there is a single point of failure problem. Second, the data don’t have self-verification and non-deniable mechanisms for ensuring integrity. Third, there is no guarantee of consistency of the data model, it makes data content and counts will be different between Cloud and Edge sides. This research aims to understand the essentials of consistency and know how to design a good cross-region data transmission architecture by solving these problems.
     This thesis proposes three methods to deal with these problems: High-Available Network of Edge, Chained Data Relationship, and Piggyback Fixing. Lastly, the author verifies these suggestions by two Network failure experiments to show how data are automatically fixed and how the standing by DataSender will take control and send to Cloud when another is failed. And how the data in Cloud will become consistent eventually.
參考文獻 ArentzMarcel. (2021年11月23日). The piggyback mechanism. 擷取自 checkmk documentation: https://docs.checkmk.com/latest/en/piggyback.html
     Brewer, E. A. (2000). Towards Robust Distributed Systems. Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing (p. 7). Portland, Oregon, USA: Association for Computing Machinery. doi:10.1145/343477.343502
     Cambridge Dictionary: piggyback. (無日期). 擷取自 Cambridge Dictionary: https://dictionary.cambridge.org/us/dictionary/english/piggyback
     Herlihy, M. P. (1990). Linearizability: A Correctness Condition for Concurrent Objects. New York, NY, USA: Association for Computing Machinery. doi:10.1145/78969.78972
     KleppmannMartin. (2015年5月11日). Please stop calling databases CP or AP. 擷取自 Martin Kleppmann: https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
     KleppmannMartin. (2021). 一致性與共識. 於 KleppmannMartin, 資料密集型應用系統設計 (李健榮, 譯者). 台北市: 碁峰/O`REILLY.
     liuyongfei. (2020年7月20日). 阿里canal是怎么通过zookeeper实现HA机制的?. 擷取自 segmentfault: 阿里canal是怎么通过zookeeper实现HA机制的?
     Ongaroand Ousterhout, JohnDiego. (2014). In Search of an Understandable Consensus Algorithm. Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (頁 305–320). Philadelphia, PA: USENIX Association. doi:10.5555/2643634.2643666
     Terry, D. (2011). Replicated Data Consistency Explained Through Baseball. Microsoft.
     V. Deshpande, L. G. (2020). Blockchain Based Decentralized Framework for Energy Demand Response Marketplace. NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium, (pp. 1-9). doi:10.1109/NOMS47738.2020.9110378
     ZooKeeper. (2022, 6 5). ZooKeeper: Because Coordinating Distributed Systems is a Zoo. Retrieved from Zookeeper: https://zookeeper.apache.org/doc/r3.8.0/index.html
     ZooKeeper Programmer`s Guide. (2022年6月5日). 擷取自 Apache Zookeeper: https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html
     台灣電力公司. (2022年8月9日). 輔助服務IEC 61850 及 DNP 3.0 傳輸格式說明. 擷取自 電力交易平台: https://etp.taipower.com.tw/web/download/3-1%EF%BC%9A%E9%99%84%E4%BB%B6%E4%BA%8C%20%E8%BC%94%E5%8A%A9%E6%9C%8D%E5%8B%99IEC%2061850%20%E5%8F%8A%20DNP%203.0%20%E5%82%B3%E8%BC%B8%E6%A0%BC%E5%BC%8F%E8%AA%AA%E6%98%8E%20TPC-MT-C0301-v02-9.2.pdf
     台灣電力公司. (2022年5月6日). 電力交易平台參考資料 電力系統運轉與調度. 擷取自 電力交易平台: https://etp.taipower.com.tw/web/download/2.電力系統運轉與調度.pdf
     梁敏雄. (2021年2月23日). 臺灣電力市場-電力輔助服務市場初步介紹. 擷取自 FIND: https://www.find.org.tw/index/knowledge/browse/26c27cabcebae579b988cb356ab7a023/
描述 碩士
國立政治大學
資訊科學系碩士在職專班
108971005
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0108971005
資料類型 thesis
dc.contributor.advisor 陳恭<br>廖峻鋒zh_TW
dc.contributor.advisor Chen, Kung<br>Liao, Chun-Fengen_US
dc.contributor.author (Authors) 黃俊鈞zh_TW
dc.contributor.author (Authors) Huang, Chun-Chunen_US
dc.creator (作者) 黃俊鈞zh_TW
dc.creator (作者) Huang, Chun-Chunen_US
dc.date (日期) 2022en_US
dc.date.accessioned 5-Oct-2022 09:09:19 (UTC+8)-
dc.date.available 5-Oct-2022 09:09:19 (UTC+8)-
dc.date.issued (上傳時間) 5-Oct-2022 09:09:19 (UTC+8)-
dc.identifier (Other Identifiers) G0108971005en_US
dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/142101-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學系碩士在職專班zh_TW
dc.description (描述) 108971005zh_TW
dc.description.abstract (摘要) 在電力交易市場裡,台電公司會使用各個交易群組回傳之電表監控資料作為即時調度、月度結算的依據。但電表的監控資料卻有著幾個根本的問題。首先,傳輸距離長,但Edge端的網路可靠性不足,造成了明顯的單一脆弱點問題;第二,資料本身缺少完整性、不可否認性的驗證措施,然而這是智慧電表普遍的情形,目前已有研究進行改善;第三,資料模型缺乏一致性的保證,這使得不同端點間對資料筆數及時序關聯認定會有出入。因此本研究便希望探究並解決這幾個問題,並藉由這個研究過程,來深入了解資料一致性的本質,以及如何設計跨地域性的資料傳輸架構。
     對此,本研究提出三個機制來解決第一項與第三項問題:第一是藉由建立高可用的Edge端網路以增加傳輸路線,減少Edge端斷線的機率;第二是藉由建立資料間鏈結式關聯,如:時序間的因果關聯,使得Cloud端得以掌握與Edge端資料的數量與次序差異,確保一致性;第三是使用Piggyback的機制進行補值,以利當缺值發生時,可透過自動補值的方式將Cloud端資料補足。最後透過兩項網路面的實驗情境,驗證在這三項機制相互配合下,資料能自動地補足,以達成最終一致性的目標,並使得傳輸中的Edge端DataSender服務失效時,備援的另一組服務就能在短時間內接手並開始回傳資料,以達成高可用性的目標。
zh_TW
dc.description.abstract (摘要) Taiwan Power Company (TPC) uses the Electricity Monitoring Data for dispatching ancillary services and the calculating fee of Transaction Groups monthly. However, these data read from AMI on the Edge side have several critical issues. First, the network of the Edge side is not reliable for long-range transmission, because there is a single point of failure problem. Second, the data don’t have self-verification and non-deniable mechanisms for ensuring integrity. Third, there is no guarantee of consistency of the data model, it makes data content and counts will be different between Cloud and Edge sides. This research aims to understand the essentials of consistency and know how to design a good cross-region data transmission architecture by solving these problems.
     This thesis proposes three methods to deal with these problems: High-Available Network of Edge, Chained Data Relationship, and Piggyback Fixing. Lastly, the author verifies these suggestions by two Network failure experiments to show how data are automatically fixed and how the standing by DataSender will take control and send to Cloud when another is failed. And how the data in Cloud will become consistent eventually.
en_US
dc.description.tableofcontents 第一章 緒論 1
     第一節 研究背景與動機 1
     第二節 研究目的 2
     第二章 技術背景與文獻回顧 4
     第一節 電力交易平台 4
     第二節 電力輔助服務 6
     第三節 電力調度平台 8
     第四節 Zookeeper 10
     第五節 資料一致性 12
     第六節 Piggyback 14
     第三章 技術架構設計 15
     第一節 高可用Edge網路(Multi-Links) 15
     第二節 鏈式資料關聯(Chained Data) 18
     第三節 Piggyback補值機制(Piggy Patch) 19
     第四章 系統設計與實證 20
     第四節 系統設計 20
     第五節 實證結果 26
     第五章 結論與展望 32
     第一節 研究結論 32
     第二節 未來展望 32
     第六章 參考文獻 34
     第七章 附件一-測試資料 36
     第一節 實驗一 36
     第二節 實驗二 46
zh_TW
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0108971005en_US
dc.subject (關鍵詞) 電力監控zh_TW
dc.subject (關鍵詞) 分散式系統zh_TW
dc.subject (關鍵詞) 一致性zh_TW
dc.subject (關鍵詞) 可靠性zh_TW
dc.subject (關鍵詞) Electricity monitoringen_US
dc.subject (關鍵詞) Distributed systemsen_US
dc.subject (關鍵詞) Consistencyen_US
dc.subject (關鍵詞) Availabilityen_US
dc.title (題名) 電力監控資料傳輸之一致性與可靠性問題研究—以A公司電力調度平台架構為例zh_TW
dc.title (題名) Consistency and Availability Problems of Electricity Monitoring Data Transmission – A Case Study of an Electricity Dispatch Platform of A Companyen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) ArentzMarcel. (2021年11月23日). The piggyback mechanism. 擷取自 checkmk documentation: https://docs.checkmk.com/latest/en/piggyback.html
     Brewer, E. A. (2000). Towards Robust Distributed Systems. Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing (p. 7). Portland, Oregon, USA: Association for Computing Machinery. doi:10.1145/343477.343502
     Cambridge Dictionary: piggyback. (無日期). 擷取自 Cambridge Dictionary: https://dictionary.cambridge.org/us/dictionary/english/piggyback
     Herlihy, M. P. (1990). Linearizability: A Correctness Condition for Concurrent Objects. New York, NY, USA: Association for Computing Machinery. doi:10.1145/78969.78972
     KleppmannMartin. (2015年5月11日). Please stop calling databases CP or AP. 擷取自 Martin Kleppmann: https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
     KleppmannMartin. (2021). 一致性與共識. 於 KleppmannMartin, 資料密集型應用系統設計 (李健榮, 譯者). 台北市: 碁峰/O`REILLY.
     liuyongfei. (2020年7月20日). 阿里canal是怎么通过zookeeper实现HA机制的?. 擷取自 segmentfault: 阿里canal是怎么通过zookeeper实现HA机制的?
     Ongaroand Ousterhout, JohnDiego. (2014). In Search of an Understandable Consensus Algorithm. Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (頁 305–320). Philadelphia, PA: USENIX Association. doi:10.5555/2643634.2643666
     Terry, D. (2011). Replicated Data Consistency Explained Through Baseball. Microsoft.
     V. Deshpande, L. G. (2020). Blockchain Based Decentralized Framework for Energy Demand Response Marketplace. NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium, (pp. 1-9). doi:10.1109/NOMS47738.2020.9110378
     ZooKeeper. (2022, 6 5). ZooKeeper: Because Coordinating Distributed Systems is a Zoo. Retrieved from Zookeeper: https://zookeeper.apache.org/doc/r3.8.0/index.html
     ZooKeeper Programmer`s Guide. (2022年6月5日). 擷取自 Apache Zookeeper: https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html
     台灣電力公司. (2022年8月9日). 輔助服務IEC 61850 及 DNP 3.0 傳輸格式說明. 擷取自 電力交易平台: https://etp.taipower.com.tw/web/download/3-1%EF%BC%9A%E9%99%84%E4%BB%B6%E4%BA%8C%20%E8%BC%94%E5%8A%A9%E6%9C%8D%E5%8B%99IEC%2061850%20%E5%8F%8A%20DNP%203.0%20%E5%82%B3%E8%BC%B8%E6%A0%BC%E5%BC%8F%E8%AA%AA%E6%98%8E%20TPC-MT-C0301-v02-9.2.pdf
     台灣電力公司. (2022年5月6日). 電力交易平台參考資料 電力系統運轉與調度. 擷取自 電力交易平台: https://etp.taipower.com.tw/web/download/2.電力系統運轉與調度.pdf
     梁敏雄. (2021年2月23日). 臺灣電力市場-電力輔助服務市場初步介紹. 擷取自 FIND: https://www.find.org.tw/index/knowledge/browse/26c27cabcebae579b988cb356ab7a023/
zh_TW
dc.identifier.doi (DOI) 10.6814/NCCU202201555en_US