Publications-Theses
Article View/Open
Publication Export
-
題名 動態側錄Python指令級行為應用於類神經網路
Profiling Instruction-Level Python Execution on Neural Networks作者 林郁豪
Lin, Yu-Hao貢獻者 郁方<br>蕭舜文
Yu, Fang<br>Hsiao, Shun-Wen
林郁豪
Lin, Yu-Hao關鍵詞 Python
神經網路
動態側錄
原始碼
函式
Python
Profiler
Function Call
Neural network
Machine Learning日期 2020 上傳時間 2-Sep-2020 11:45:46 (UTC+8) 摘要 Python語言已被廣泛用於開發現代應用程序，例如Web應用程序，數據分析，機器學習和機器人技術。 由於其高層次的互動性質和多元的套件。 作為一種通用語言，它不僅在學術環境中而且在工業中都越來越多地使用。儘管它是算法開發和探索性數據分析的理想選擇，但係統地分析Python程序行為的方法對於軟件效率和安全性至關重要。為了在python腳本上進行運行時驗證，我們實現了自定義的側錄器，該側錄器可以在執行過程中記錄所需的功能及其運行狀態，而無需更改源代碼。 我們通過構造自己的編譯器來實現此目標，使其可以採用相同的源代碼（或字節碼），但會產生帶有附加代碼的相應機器代碼，以記錄所需的資訊。具體來說，我們實現了一個新的Python編譯器，該編譯器將生成檢測到的機器代碼，以供執行者分析實際的執行跟踪。 每個執行跟踪都記錄為基於層的函數序列，其中每個函數都有其基本的執行資訊，例如參數類型，值，返回值以及開始時間，結束時間和效能成本。 然後可以將分層序列明確表示為跳表，其中較高層的序列表示外部函數調用，其內部函數調用包含在較低層的序列中。
Python language has been widely adopted to develop modern applications such as web applications, data analytic, machine learning, and robotics. Due to its high-level interactive nature and its maturing ecosystem of scientific libraries. As a general-purpose language, it is increasingly used not only in academic settings but also in industry.While it is an appealing choice for algorithmic development and exploratory data analysis, a systematic approach to analyze behaviors of Python programs is of the essence for software efficiency and security.To facilitate runtime verification on python scripts, we implement a customized profiler that can record desired functions and their running status along execution without changing the source code. We achieve this goal by constructing our own interpreter such that it can take the same source code (or bytecode) but yield the corresponding machine code with additional codes to record desired information.Specifically, we implement a new Python compiler that generates instrumented machine codes for executor to profile real execution traces. Each execution trace is recorded as a layer-based function sequence, where each function has its essential runtime information such as parameter types, values, return values, as well as starting time, ending time, and performance cost. The layered sequences can then be explicitly represented as a skip list, where a higher layer sequence represents outer function calls whose inner function calls are included in lower layer sequences.參考文獻 [1] M. F. Sanner et al., “Python: a programming language for software integration and development,” J Mol Graph Model, vol. 17, no. 1, pp. 57–61, 1999.[2] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg et al., “Scikit-learn: Machine learning inpython,” Journal of machine learning research, vol. 12, no. Oct, pp. 2825–2830, 2011.[3] G. Tanganelli, C. Vallati, and E. Mingozzi, “Coapthon: Easy development of coapbased iot applications with python,” in 2015 IEEE 2nd World Forum on Internet ofThings (WF-IoT). IEEE, 2015, pp. 63–68.[4] S. Bird, E. Klein, and E. Loper, Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.", 2009.[5] J. Demšar, T. Curk, A. Erjavec, Č. Gorup, T. Hočevar, M. Milutinovič, M. Možina, M. Polajnar, M. Toplak, A. Starič et al., “Orange: data mining toolbox in python,”The Journal of Machine Learning Research, vol. 14, no. 1, pp. 2349–2353, 2013.[6] K. J. Millman and M. Aivazis, “Python for scientists and engineers,” Computing in Science & Engineering, vol. 13, no. 2, pp. 9–12, 2011.[7] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: A convolutional neural-network approach,” IEEE transactions on neural networks, vol. 8,no. 1, pp. 98–113, 1997.[8] M. Dahm, “Byte code engineering,” in JIT’99. Springer, 1999, pp. 267–277.[9] Wikipedia contributors, “Machine code — Wikipedia, the free encyclopedia,” 2020,[Online; accessed 28-February-2020]. [Online]. Available: https://en.wikipedia.org/w/index.php?title=Machine_code&oldid=939857422[10] “Pyobject,” https://docs.python.org/3/c-api/structures.html, accessed: 2020-02-29.[11] “The python profiler,” https://docs.python.org/2/library/profile.html, accessed:2020-02-29.[12] “Pypi, memory–profiler 0.55.0.” https://pypi.org/project/memory-profiler/, accessed: 2020-02-29.[13] “Skip list,” https://en.wikipedia.org/wiki/Skip_list, accessed: 2020-02-29.[14] “Python’s website,” https://www.python.org/downloads/source/, accessed: 2020-02-29.[15] “psutil,” https://pypi.org/project/psutil/, accessed: 2020-02-29.[16] “Ntquerysysteminformation(),” https://docs.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntquerysysteminformation, accessed: 2020-02-29.[17] “tracemalloc(),” https://docs.python.org/3/library/tracemalloc.html, accessed:2020-02-29.[18] “Cnn example,” https://www.tensorflow.org/tutorials/keras/classification?hl=zh_tw&fbclid=IwAR2H5BZtaLcU96Yntod5rr4KV4jrN5GK055EaHlRow0jNyyPs6uDyxbSP78#make_predictions, accessed: 2020-02-29.[19] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.[20] “Rnn example,” https://keras.io/examples/babi_rnn/, accessed: 2020-02-29. 描述 碩士
國立政治大學
資訊管理學系
107356012資料來源 http://thesis.lib.nccu.edu.tw/record/#G0107356012 資料類型 thesis dc.contributor.advisor 郁方<br>蕭舜文 zh_TW dc.contributor.advisor Yu, Fang<br>Hsiao, Shun-Wen en_US dc.contributor.author (Authors) 林郁豪 zh_TW dc.contributor.author (Authors) Lin, Yu-Hao en_US dc.creator (作者) 林郁豪 zh_TW dc.creator (作者) Lin, Yu-Hao en_US dc.date (日期) 2020 en_US dc.date.accessioned 2-Sep-2020 11:45:46 (UTC+8) - dc.date.available 2-Sep-2020 11:45:46 (UTC+8) - dc.date.issued (上傳時間) 2-Sep-2020 11:45:46 (UTC+8) - dc.identifier (Other Identifiers) G0107356012 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/131491 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊管理學系 zh_TW dc.description (描述) 107356012 zh_TW dc.description.abstract (摘要) Python語言已被廣泛用於開發現代應用程序，例如Web應用程序，數據分析，機器學習和機器人技術。 由於其高層次的互動性質和多元的套件。 作為一種通用語言，它不僅在學術環境中而且在工業中都越來越多地使用。儘管它是算法開發和探索性數據分析的理想選擇，但係統地分析Python程序行為的方法對於軟件效率和安全性至關重要。為了在python腳本上進行運行時驗證，我們實現了自定義的側錄器，該側錄器可以在執行過程中記錄所需的功能及其運行狀態，而無需更改源代碼。 我們通過構造自己的編譯器來實現此目標，使其可以採用相同的源代碼（或字節碼），但會產生帶有附加代碼的相應機器代碼，以記錄所需的資訊。具體來說，我們實現了一個新的Python編譯器，該編譯器將生成檢測到的機器代碼，以供執行者分析實際的執行跟踪。 每個執行跟踪都記錄為基於層的函數序列，其中每個函數都有其基本的執行資訊，例如參數類型，值，返回值以及開始時間，結束時間和效能成本。 然後可以將分層序列明確表示為跳表，其中較高層的序列表示外部函數調用，其內部函數調用包含在較低層的序列中。 zh_TW dc.description.abstract (摘要) Python language has been widely adopted to develop modern applications such as web applications, data analytic, machine learning, and robotics. Due to its high-level interactive nature and its maturing ecosystem of scientific libraries. As a general-purpose language, it is increasingly used not only in academic settings but also in industry.While it is an appealing choice for algorithmic development and exploratory data analysis, a systematic approach to analyze behaviors of Python programs is of the essence for software efficiency and security.To facilitate runtime verification on python scripts, we implement a customized profiler that can record desired functions and their running status along execution without changing the source code. We achieve this goal by constructing our own interpreter such that it can take the same source code (or bytecode) but yield the corresponding machine code with additional codes to record desired information.Specifically, we implement a new Python compiler that generates instrumented machine codes for executor to profile real execution traces. Each execution trace is recorded as a layer-based function sequence, where each function has its essential runtime information such as parameter types, values, return values, as well as starting time, ending time, and performance cost. The layered sequences can then be explicitly represented as a skip list, where a higher layer sequence represents outer function calls whose inner function calls are included in lower layer sequences. en_US dc.description.tableofcontents 1 Introduction 12 Related Work 92.1 cProfile 92.2 memory-profiler 102.3 line_profiler 102.4 yappi 103 Methodology 113.1 Python Opcode Profiling 113.1.1 Python Opcode 113.1.2 Opcode Profiling 123.2 Python Interpreter Instrumentation 143.3 Nested Call Sequence 184 Opcode Performance Analysis On Neural Networks 214.1 Profiling Result of CNN 214.2 Profiling Result of RNN 264.3 Profiling Result of GAN 274.4 Profiling Result of CNN using GPU 284.5 Profiling Result of Keras Applications 295 Conclusion 326 Reference 49 zh_TW dc.format.extent 5198265 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0107356012 en_US dc.subject (關鍵詞) Python zh_TW dc.subject (關鍵詞) 神經網路 zh_TW dc.subject (關鍵詞) 動態側錄 zh_TW dc.subject (關鍵詞) 原始碼 zh_TW dc.subject (關鍵詞) 函式 zh_TW dc.subject (關鍵詞) Python en_US dc.subject (關鍵詞) Profiler en_US dc.subject (關鍵詞) Function Call en_US dc.subject (關鍵詞) Neural network en_US dc.subject (關鍵詞) Machine Learning en_US dc.title (題名) 動態側錄Python指令級行為應用於類神經網路 zh_TW dc.title (題名) Profiling Instruction-Level Python Execution on Neural Networks en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] M. F. Sanner et al., “Python: a programming language for software integration and development,” J Mol Graph Model, vol. 17, no. 1, pp. 57–61, 1999.[2] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg et al., “Scikit-learn: Machine learning inpython,” Journal of machine learning research, vol. 12, no. Oct, pp. 2825–2830, 2011.[3] G. Tanganelli, C. Vallati, and E. Mingozzi, “Coapthon: Easy development of coapbased iot applications with python,” in 2015 IEEE 2nd World Forum on Internet ofThings (WF-IoT). IEEE, 2015, pp. 63–68.[4] S. Bird, E. Klein, and E. Loper, Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.", 2009.[5] J. Demšar, T. Curk, A. Erjavec, Č. Gorup, T. Hočevar, M. Milutinovič, M. Možina, M. Polajnar, M. Toplak, A. Starič et al., “Orange: data mining toolbox in python,”The Journal of Machine Learning Research, vol. 14, no. 1, pp. 2349–2353, 2013.[6] K. J. Millman and M. Aivazis, “Python for scientists and engineers,” Computing in Science & Engineering, vol. 13, no. 2, pp. 9–12, 2011.[7] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: A convolutional neural-network approach,” IEEE transactions on neural networks, vol. 8,no. 1, pp. 98–113, 1997.[8] M. Dahm, “Byte code engineering,” in JIT’99. Springer, 1999, pp. 267–277.[9] Wikipedia contributors, “Machine code — Wikipedia, the free encyclopedia,” 2020,[Online; accessed 28-February-2020]. [Online]. Available: https://en.wikipedia.org/w/index.php?title=Machine_code&oldid=939857422[10] “Pyobject,” https://docs.python.org/3/c-api/structures.html, accessed: 2020-02-29.[11] “The python profiler,” https://docs.python.org/2/library/profile.html, accessed:2020-02-29.[12] “Pypi, memory–profiler 0.55.0.” https://pypi.org/project/memory-profiler/, accessed: 2020-02-29.[13] “Skip list,” https://en.wikipedia.org/wiki/Skip_list, accessed: 2020-02-29.[14] “Python’s website,” https://www.python.org/downloads/source/, accessed: 2020-02-29.[15] “psutil,” https://pypi.org/project/psutil/, accessed: 2020-02-29.[16] “Ntquerysysteminformation(),” https://docs.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntquerysysteminformation, accessed: 2020-02-29.[17] “tracemalloc(),” https://docs.python.org/3/library/tracemalloc.html, accessed:2020-02-29.[18] “Cnn example,” https://www.tensorflow.org/tutorials/keras/classification?hl=zh_tw&fbclid=IwAR2H5BZtaLcU96Yntod5rr4KV4jrN5GK055EaHlRow0jNyyPs6uDyxbSP78#make_predictions, accessed: 2020-02-29.[19] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.[20] “Rnn example,” https://keras.io/examples/babi_rnn/, accessed: 2020-02-29. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202001453 en_US
