學術產出-Theses
Article View/Open
Publication Export
-
題名 以GeoJSON壓縮技術增進網路資料傳輸效能之研究
Efficiency improvement of spatial data transmission by GeoJSON compression techniques作者 陳欣瑜
Chen,Hsin Yu貢獻者 何瑁鎧
Hor,Maw Kae
陳欣瑜
Chen,Hsin Yu關鍵詞 GeoJSON
差分編碼
單向雜湊函數
GeoJSON
delta encoding
one-way hash function日期 2010 上傳時間 4-Sep-2013 17:05:25 (UTC+8) 摘要 標準化的地理資料交換格式是開放式地理資訊系統不可或缺的一環，可提供不同的平台經由統一的交換格式而順利進行資料交換。GeoJSON除了具備基本的資料互通性外，其結構簡單且容易讀取的特性為地理資訊服務軟體帶來許多效益。然而座標資料量之多寡直接影響資料傳輸效能，為提昇傳輸效率，必須配合有效的資料壓縮技術以降低傳輸之資料量。此外，GeoJSON壓縮方法的設計與結構訴求，也須融合GeoJSON的概念，以簡單、方便運用與容易了解為目標。本研究提出一套GeoJSON壓縮技術進行空間資料的壓縮，以降低資料量而增進Web GIS地理資料傳輸的效能。除評估其壓縮效率，並透過不同規模以及不同型態的地理資料，分析影響壓縮效率的原因。最後藉由HTTP壓縮技術輔助，並從資料傳遞的時間與壓縮率，評估本研究提出的方法所帶來的成效。我們同時以單向雜湊函數，建立資料傳遞時的檢查機制，以確保資料傳遞時的正確性與一致性。實作中，我們採用GeoJSON壓縮技術，進行座標資料的大小減量實驗，結果顯示本研究方法可以得到不錯的壓縮成果與傳輸效能，並且可避免資料傳輸發生問題。
Standardized GIS data exchanging format is an essential part of Open GIS. This enables GIS data providers, software developers, and system integrators to exchange GIS data from different platform. The simple structure of GeoJSON not only has the data interoperability potential but also has the characteristics of easy processed and readability. These properties directly benefit the GIS service software. However, the amount of spatial information encoded in GIS documents usually has direct impact to the efficiency of GIS data transmission. In order to improve the data transmission efficiency, one has to reduce the amount of spatial data transmitted through data compression techniques.In this thesis, we proposed a data compression mechanism for spatial data. Our mechanism, co-operated with the concept of GeoJSON, aim at simple, easy to understand, and easy to use, can reduce the amount of spatial data transmitted and improve the transmission efficiency. We analyzed the compression ratio of various data types and data amounts through different base parameters. We also measured the system response time reduced using this method and compared with the combination of using our method as well as the HTTP compression modules. A one-way hashing technique is used to ensure data accuracy and consistency during the transmission processes. The experimental results show that our GeoJSON-based compression mechanism can significantly reduce the file size of spatial data and improve the efficiency of spatial data transmission. In addition, the data communication errors can be avoided.參考文獻 [1] 徐百輝，”地理資訊標準格式之簡介(電子版) ”，國土資訊系統通訊，第71期，pp. 14-34，2009。[2] 林昂賢，”一個高效率的XML資料壓縮演算法”，國立臺灣大學資訊工程學研究所碩士論文，1999。[3] 廖泫銘、林農堯、廖宜真，”地理資訊開放服務的規範與應用軟體架構”，國土資訊系統通訊，第71期，pp. 53-64，2008。[4] 鍾國亮，”資料壓縮的原理與應用”，台北：全華科技圖書，2004。[5] 陳仁德，”一個針對GPRS之資料壓縮演算法”，國立交通大學資訊管理研究所碩士論文，2005。[6] 賴溪松、韓亮、張真誠，”近代密碼學及其應用”，台北：松崗，1998。[7] 楊佑寧，”有限信任讀卡機下安全服務機制”，國立暨南國際大學資訊管理學系碩士論文，2006。[8] 孫志堅，”地理空間資料與座標系統間關係之研析”，2010年6月20日，取自http://163.29.126.136/share/地理空間資料與座標系統間關係之研析，2010[9] 微軟技術及技術論壇(2002)，”效能比較：安全性設計選擇”，MSDN Library，2010年6月20日，取自http://www.microsoft.com/taiwan/msdn/library/2002/Nov-2002/bdadotnetarch15.htm。[10] PCNET網路研究所，”TCP與UDP”， 2010年9月20日，取自http://www.pcnet.idv.tw/pcnet/network/network_ip_tcp.htm。[11] Bassiouni, M.A.，” Data compression in Scientific and Statistical Databases”，IEEE Transactions on Software Engineering., vol.11, pp.1047-1058, 1985.[12] Bakhtiari, S. and Safavi-Naini, R. and Pieprzyk J.,”Cryptographic Hash Functions: A Survey.” Technical Report, Department of Computer Science, University of Wollongong, pp. 95-09, 1995.[13] Do-Hyun, “K., and K. Min-Soo,”Web GIS service component based on open environment”, In Geoscience and Remote Sensing Symposium,. IEEE International, pp. 3346-3348, 2002.[14] D.A. Huffman, “A method for the construction of minimum-redundancy codes”, Proceedings of the I.R.E., pp.1098-1102, 1952.[15] James J. Hunt and Kiem-Phong Vo and Walter F. Tichy, “Delta Algorithms: An Empirical Analysis”, ACM Transactions on Software Engineering and Methodology, vol.7, pp. 192-214, 1998.[16] Jens Müller, “Data Compression-LZ77”, Universität Stuttgart, 2008.[17] Jihong Guan, Shuigeng Zhou, “GPress: Towards Effective GML Documents Compresssion”, IEEE 23rd International Conference on Data Engineering, pp. 1473-1474, 2007. [18] J. Rissanen and G. G. Langdon, “An Introduction to Arithmetic Coding”, IBM Journal of Research and Development, vol. 28, pp. 135, 1979.[19] Jeffery N. Ladino, “Data Compression Algorithms”, College of Computer Science at Northeastern University, Honors Project, 1996.[20] J. Ziv and A. Lemple, “A universal algorithm for data compression”, IEEE Transactions n Information Theory, vol. 23, pp. 337-343, 1997.[21] Torsten Suel and Nasir Memon, “Algorithms for Delta Compression and Remote File Synchronization”, In Khalid Sayood, editor, Lossless Compression Handbook, 2002.[22] Timothy J. McLaughlin,” The Benefits and Drawbacks of HTTP Compression” , Lehigh CSC 2002 Technical Reports,vol. 2, pp. 104, 2002.[23] OGC(2007), OpenGIS® Web Processing Service Implementation Specification Version: 1.0.0.[24] Peng, Z. R. A., and C. A. Zhang, “The roles of geography markup language (GML), scalable vector graphics (SVG), and Web feature service (WFS) specifications in the development of Internet geographic information systems (GIS)”，Journal of Geographical Systems, vol.6, pp.95-116, 2006.[25] R. Rivest(1992),”The MD5 Message-Digest Algorithm”, MIT Laboratory for Computer Science and RSA Data Security, Inc.,1992.[26] Yuzhen Li, Takashi Imaizumi, Jihong Guan, “Spatial Data Compression Techniques for GML”, Japan-China Joint Workshop on Frontier of Computer Science and Technology, pp.79-84, 2008.[27] 3 Top Data Formats for Map Mashups: KML, GeoRSS and GeoJSON, Retrieved Aug 11, 2009, from http://blog.programmableweb.com/2008/08/27/3-top-data-formats-for-map-mashups-kml-georss-and-geojson/.[28] Behram Mistree and Dmitry Kashlev , “, GZIP Encoding”, Retrieved Jun 20, 2009, from http://csg.csail.mit.edu/6.375/6_375_2007_www/projects/group6_final_report.pdf .[29] ArcDeveloper Project, Retrieved Aug 11, 2009, from http://groups.google.com/group/arcdeveloper-dev. [30] Automation for the people: Deployment-automation patterns, Retrieved Aug 11, 2009, from http://www.ibm.com/developerworks/java/library/j-ap02109/index.html.[31] Douglas Crockford, Retrieved Aug 11, 2009, from http://www.crockford.com/.[32] Data compression tutorial: Part 2, Retrieved Aug 11, 2009, from http://www.eetimes.com/design/automotive-design/4017499/Data-compression-tutorial-Part-2.[33] Ekrem seren’s weblog, Retrieved May 20, 2009, from http://www.ekremseren.com/2009/05/compression-tools-lzma-bzip2-gzip/.[34] GeoJSON WIKI, Retrieved May 20, 2009, from http://wiki.geojson.org/Main_Page.[35] GeoJSON Python Library, Retrieved May 20, 2009, from http://pypi.python.org/pypi/geojson/1.0.[36] Google maps, Retrieved May 20, 2009, from http://maps.google.com/support/bin/static.py?hl=b5&page=guide.cs&guide=21670, [37] Introducing JSON, Retrieved May 20, 2009, from http://json.org. [38] J. Stolfi. Hash function , Retrieved Aug 11, 2009, from http://en.wikipedia.org/wiki/Hash_function.[39] LZ77 and LZ78, Retrieved Aug 11, 2009, from http://en.wikipedia.org/wiki/LZ77_and_LZ78.[40] MapFish, Retrieved May 20, 2009, from http://mapfish.org/.[41] MD5-Wiki, Retrieved May 20, 2009, from http://en.wikipedia.org/wiki/MD5. [42] Open Source Geospatial Foundation, Retrieved May 20, 2009, from http://www.osgeo.org/.[43] OpenLayers, Retrieved May 20, 2009, from http://openlayers.org/.[44] OSGEO, Retrieved May 20, 2009, from http://www.osgeo.org/.[45] OpenLayers Vector Formats Example, Retrieved Aug 11, 2009, from http://openlayers.org/dev/examples/vector-formats.html.[46] pgRouting On Ubuntu Netbook Remix 9.10, Retrieved May 20, 2009, from http://www.mkgeomatics.com/wordpress/?p=312.[47] Speed Web delivery with HTTP compression, Retrieved Apr 20, 2009, from http://www.ibm.com/developerworks/web/library/wa-httpcomp/.[48] The GeoJSON Format Specification, Retrieved July 25, 2009, from http://geojson.org/geojson-spec.html.[49] Unzipping the GZIP compression protocol, Retrieved Apr 20, 2009, from http://www.chipestimate.com/techtalk.php?d=2010-03-23.)[50] Web Services, Output Formats and GZIP Compression, Retrieved Feb 11, 2010, from http://www.sendung.de/archives/2007/04/09/web-services-output-formats-and-gzip-compression/.[51] Yahoo Developer Network, Retrieved Apr 20, 2010, from http://developer.yahoo.com/common/json.html#xml. 描述 碩士
國立政治大學
資訊科學學系
96971012
99資料來源 http://thesis.lib.nccu.edu.tw/record/#G0096971012 資料類型 thesis dc.contributor.advisor 何瑁鎧 zh_TW dc.contributor.advisor Hor,Maw Kae en_US dc.contributor.author (Authors) 陳欣瑜 zh_TW dc.contributor.author (Authors) Chen,Hsin Yu en_US dc.creator (作者) 陳欣瑜 zh_TW dc.creator (作者) Chen,Hsin Yu en_US dc.date (日期) 2010 en_US dc.date.accessioned 4-Sep-2013 17:05:25 (UTC+8) - dc.date.available 4-Sep-2013 17:05:25 (UTC+8) - dc.date.issued (上傳時間) 4-Sep-2013 17:05:25 (UTC+8) - dc.identifier (Other Identifiers) G0096971012 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/60236 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學學系 zh_TW dc.description (描述) 96971012 zh_TW dc.description (描述) 99 zh_TW dc.description.abstract (摘要) 標準化的地理資料交換格式是開放式地理資訊系統不可或缺的一環，可提供不同的平台經由統一的交換格式而順利進行資料交換。GeoJSON除了具備基本的資料互通性外，其結構簡單且容易讀取的特性為地理資訊服務軟體帶來許多效益。然而座標資料量之多寡直接影響資料傳輸效能，為提昇傳輸效率，必須配合有效的資料壓縮技術以降低傳輸之資料量。此外，GeoJSON壓縮方法的設計與結構訴求，也須融合GeoJSON的概念，以簡單、方便運用與容易了解為目標。本研究提出一套GeoJSON壓縮技術進行空間資料的壓縮，以降低資料量而增進Web GIS地理資料傳輸的效能。除評估其壓縮效率，並透過不同規模以及不同型態的地理資料，分析影響壓縮效率的原因。最後藉由HTTP壓縮技術輔助，並從資料傳遞的時間與壓縮率，評估本研究提出的方法所帶來的成效。我們同時以單向雜湊函數，建立資料傳遞時的檢查機制，以確保資料傳遞時的正確性與一致性。實作中，我們採用GeoJSON壓縮技術，進行座標資料的大小減量實驗，結果顯示本研究方法可以得到不錯的壓縮成果與傳輸效能，並且可避免資料傳輸發生問題。 zh_TW dc.description.abstract (摘要) Standardized GIS data exchanging format is an essential part of Open GIS. This enables GIS data providers, software developers, and system integrators to exchange GIS data from different platform. The simple structure of GeoJSON not only has the data interoperability potential but also has the characteristics of easy processed and readability. These properties directly benefit the GIS service software. However, the amount of spatial information encoded in GIS documents usually has direct impact to the efficiency of GIS data transmission. In order to improve the data transmission efficiency, one has to reduce the amount of spatial data transmitted through data compression techniques.In this thesis, we proposed a data compression mechanism for spatial data. Our mechanism, co-operated with the concept of GeoJSON, aim at simple, easy to understand, and easy to use, can reduce the amount of spatial data transmitted and improve the transmission efficiency. We analyzed the compression ratio of various data types and data amounts through different base parameters. We also measured the system response time reduced using this method and compared with the combination of using our method as well as the HTTP compression modules. A one-way hashing technique is used to ensure data accuracy and consistency during the transmission processes. The experimental results show that our GeoJSON-based compression mechanism can significantly reduce the file size of spatial data and improve the efficiency of spatial data transmission. In addition, the data communication errors can be avoided. en_US dc.description.tableofcontents 第一章 緒論 11.1研究動機與背景 11.2研究目的 21.3問題描述 21.4本論文貢獻 31.5章節架構 4第二章 文獻探討 62.1地理資料交換格式簡介 62.2GeoJSON介紹 82.3差分編碼簡介 112.4HTTP與GZIP壓縮 122.5資料傳輸驗證 142.6開放式原始碼Web GIS平台簡介 17第三章 GeoJSON資料壓縮 193.1系統架構 193.2資料前處理 213.3資料壓縮技術 233.4效能與壓縮率分析 303.5資料傳輸檢查機制 31第四章 實驗結果與分析 374.1資料處理與GeoJSON輸出 374.2GeoJSON資料壓縮技術實驗結果 414.3傳輸效能與壓縮率分析 474.4資料傳輸檢查與效能評估實驗結果 53第五章 結論與未來展望 565.1結論 565.2未來研究方向 57參考文獻 58附錄 63附錄1.浮點式編碼函式的編碼 63附錄2.浮點式編碼函式的解碼 64附錄3.差分編碼函式 65 zh_TW dc.format.extent 1385393 bytes - dc.format.mimetype application/pdf - dc.language.iso en_US - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0096971012 en_US dc.subject (關鍵詞) GeoJSON zh_TW dc.subject (關鍵詞) 差分編碼 zh_TW dc.subject (關鍵詞) 單向雜湊函數 zh_TW dc.subject (關鍵詞) GeoJSON en_US dc.subject (關鍵詞) delta encoding en_US dc.subject (關鍵詞) one-way hash function en_US dc.title (題名) 以GeoJSON壓縮技術增進網路資料傳輸效能之研究 zh_TW dc.title (題名) Efficiency improvement of spatial data transmission by GeoJSON compression techniques en_US dc.type (資料類型) thesis en dc.relation.reference (參考文獻) [1] 徐百輝，”地理資訊標準格式之簡介(電子版) ”，國土資訊系統通訊，第71期，pp. 14-34，2009。[2] 林昂賢，”一個高效率的XML資料壓縮演算法”，國立臺灣大學資訊工程學研究所碩士論文，1999。[3] 廖泫銘、林農堯、廖宜真，”地理資訊開放服務的規範與應用軟體架構”，國土資訊系統通訊，第71期，pp. 53-64，2008。[4] 鍾國亮，”資料壓縮的原理與應用”，台北：全華科技圖書，2004。[5] 陳仁德，”一個針對GPRS之資料壓縮演算法”，國立交通大學資訊管理研究所碩士論文，2005。[6] 賴溪松、韓亮、張真誠，”近代密碼學及其應用”，台北：松崗，1998。[7] 楊佑寧，”有限信任讀卡機下安全服務機制”，國立暨南國際大學資訊管理學系碩士論文，2006。[8] 孫志堅，”地理空間資料與座標系統間關係之研析”，2010年6月20日，取自http://163.29.126.136/share/地理空間資料與座標系統間關係之研析，2010[9] 微軟技術及技術論壇(2002)，”效能比較：安全性設計選擇”，MSDN Library，2010年6月20日，取自http://www.microsoft.com/taiwan/msdn/library/2002/Nov-2002/bdadotnetarch15.htm。[10] PCNET網路研究所，”TCP與UDP”， 2010年9月20日，取自http://www.pcnet.idv.tw/pcnet/network/network_ip_tcp.htm。[11] Bassiouni, M.A.，” Data compression in Scientific and Statistical Databases”，IEEE Transactions on Software Engineering., vol.11, pp.1047-1058, 1985.[12] Bakhtiari, S. and Safavi-Naini, R. and Pieprzyk J.,”Cryptographic Hash Functions: A Survey.” Technical Report, Department of Computer Science, University of Wollongong, pp. 95-09, 1995.[13] Do-Hyun, “K., and K. Min-Soo,”Web GIS service component based on open environment”, In Geoscience and Remote Sensing Symposium,. IEEE International, pp. 3346-3348, 2002.[14] D.A. Huffman, “A method for the construction of minimum-redundancy codes”, Proceedings of the I.R.E., pp.1098-1102, 1952.[15] James J. Hunt and Kiem-Phong Vo and Walter F. Tichy, “Delta Algorithms: An Empirical Analysis”, ACM Transactions on Software Engineering and Methodology, vol.7, pp. 192-214, 1998.[16] Jens Müller, “Data Compression-LZ77”, Universität Stuttgart, 2008.[17] Jihong Guan, Shuigeng Zhou, “GPress: Towards Effective GML Documents Compresssion”, IEEE 23rd International Conference on Data Engineering, pp. 1473-1474, 2007. [18] J. Rissanen and G. G. Langdon, “An Introduction to Arithmetic Coding”, IBM Journal of Research and Development, vol. 28, pp. 135, 1979.[19] Jeffery N. Ladino, “Data Compression Algorithms”, College of Computer Science at Northeastern University, Honors Project, 1996.[20] J. Ziv and A. Lemple, “A universal algorithm for data compression”, IEEE Transactions n Information Theory, vol. 23, pp. 337-343, 1997.[21] Torsten Suel and Nasir Memon, “Algorithms for Delta Compression and Remote File Synchronization”, In Khalid Sayood, editor, Lossless Compression Handbook, 2002.[22] Timothy J. McLaughlin,” The Benefits and Drawbacks of HTTP Compression” , Lehigh CSC 2002 Technical Reports,vol. 2, pp. 104, 2002.[23] OGC(2007), OpenGIS® Web Processing Service Implementation Specification Version: 1.0.0.[24] Peng, Z. R. A., and C. A. Zhang, “The roles of geography markup language (GML), scalable vector graphics (SVG), and Web feature service (WFS) specifications in the development of Internet geographic information systems (GIS)”，Journal of Geographical Systems, vol.6, pp.95-116, 2006.[25] R. Rivest(1992),”The MD5 Message-Digest Algorithm”, MIT Laboratory for Computer Science and RSA Data Security, Inc.,1992.[26] Yuzhen Li, Takashi Imaizumi, Jihong Guan, “Spatial Data Compression Techniques for GML”, Japan-China Joint Workshop on Frontier of Computer Science and Technology, pp.79-84, 2008.[27] 3 Top Data Formats for Map Mashups: KML, GeoRSS and GeoJSON, Retrieved Aug 11, 2009, from http://blog.programmableweb.com/2008/08/27/3-top-data-formats-for-map-mashups-kml-georss-and-geojson/.[28] Behram Mistree and Dmitry Kashlev , “, GZIP Encoding”, Retrieved Jun 20, 2009, from http://csg.csail.mit.edu/6.375/6_375_2007_www/projects/group6_final_report.pdf .[29] ArcDeveloper Project, Retrieved Aug 11, 2009, from http://groups.google.com/group/arcdeveloper-dev. [30] Automation for the people: Deployment-automation patterns, Retrieved Aug 11, 2009, from http://www.ibm.com/developerworks/java/library/j-ap02109/index.html.[31] Douglas Crockford, Retrieved Aug 11, 2009, from http://www.crockford.com/.[32] Data compression tutorial: Part 2, Retrieved Aug 11, 2009, from http://www.eetimes.com/design/automotive-design/4017499/Data-compression-tutorial-Part-2.[33] Ekrem seren’s weblog, Retrieved May 20, 2009, from http://www.ekremseren.com/2009/05/compression-tools-lzma-bzip2-gzip/.[34] GeoJSON WIKI, Retrieved May 20, 2009, from http://wiki.geojson.org/Main_Page.[35] GeoJSON Python Library, Retrieved May 20, 2009, from http://pypi.python.org/pypi/geojson/1.0.[36] Google maps, Retrieved May 20, 2009, from http://maps.google.com/support/bin/static.py?hl=b5&page=guide.cs&guide=21670, [37] Introducing JSON, Retrieved May 20, 2009, from http://json.org. [38] J. Stolfi. Hash function , Retrieved Aug 11, 2009, from http://en.wikipedia.org/wiki/Hash_function.[39] LZ77 and LZ78, Retrieved Aug 11, 2009, from http://en.wikipedia.org/wiki/LZ77_and_LZ78.[40] MapFish, Retrieved May 20, 2009, from http://mapfish.org/.[41] MD5-Wiki, Retrieved May 20, 2009, from http://en.wikipedia.org/wiki/MD5. [42] Open Source Geospatial Foundation, Retrieved May 20, 2009, from http://www.osgeo.org/.[43] OpenLayers, Retrieved May 20, 2009, from http://openlayers.org/.[44] OSGEO, Retrieved May 20, 2009, from http://www.osgeo.org/.[45] OpenLayers Vector Formats Example, Retrieved Aug 11, 2009, from http://openlayers.org/dev/examples/vector-formats.html.[46] pgRouting On Ubuntu Netbook Remix 9.10, Retrieved May 20, 2009, from http://www.mkgeomatics.com/wordpress/?p=312.[47] Speed Web delivery with HTTP compression, Retrieved Apr 20, 2009, from http://www.ibm.com/developerworks/web/library/wa-httpcomp/.[48] The GeoJSON Format Specification, Retrieved July 25, 2009, from http://geojson.org/geojson-spec.html.[49] Unzipping the GZIP compression protocol, Retrieved Apr 20, 2009, from http://www.chipestimate.com/techtalk.php?d=2010-03-23.)[50] Web Services, Output Formats and GZIP Compression, Retrieved Feb 11, 2010, from http://www.sendung.de/archives/2007/04/09/web-services-output-formats-and-gzip-compression/.[51] Yahoo Developer Network, Retrieved Apr 20, 2010, from http://developer.yahoo.com/common/json.html#xml. zh_TW
