學術產出-Theses
Article View/Open
Publication Export
-
題名 強記軟化及整合演算法：以ReLU激發函數與實數輸入/輸出為例
The Cramming, Softening and Integrating Learning Algorithm with ReLU Activation Function for Real Number Input / Output Problems作者 梁惟婷
Liang, Wei-Ting貢獻者 蔡瑞煌
Tsaih, Rua-Huan
梁惟婷
Liang, Wei-Ting關鍵詞 強記軟化及整合學習
強記機制
軟化及整合機制
最小裁減平方
正規化
線性整流函數
CSI learning
Cramming mechanism
Softening and integrating mechanism
Least trimmed squares
Regularization
Rectified linear units日期 2019 上傳時間 7-Aug-2019 16:08:17 (UTC+8) 摘要 本研究提出了一種基於序列的學習演算法─強記軟化及整合演算法。本研究所提出之演算法具有以下特色：（1） 實現自適應單隱藏層前饋神經網路（adaptive single-hidden layer feed-forward neural networks） （2） 利用最小裁減平方（least trimmed squares）原則決定訓練樣本的輸入序列 （3） 使用線性整流函數（rectified linear units）作為隱藏節點之激發函數 （4） 演算法能在精確學習所有訓練資料之下，同時緩解過度擬合的傾向。本研究對中華航空公司的實際數據進行實驗，以探索（1） 本研究所提出之強記軟化及整合演算法是否能模仿人類的學習方式（2） 本研究所提出之強記軟化及整合演算法是否能不僅精確學習所有的訓練資料，還能透過正規化項、軟化及整合機制來緩解過度擬合的傾向。
This study proposes the cramming, softening and integrating (CSI) algorithm, a sequentially-learning-based algorithm. The proposed CSI learning algorithm has the following features: (1) the implementation through the adaptive single-hidden layer feed-forward neural networks, (2) the usage of least trimmed squares principle for determining the sequence of learning samples, (3) the usage of rectified linear units activation function, (4) the practice of precisely learning all training data, while alleviating the overfitting pain. An experiment with real data from China Airlines has been conducted to explore (1) whether the proposed CSI algorithm can imitate the way of learning in human beings, as it claims, and (2) whether the proposed CSI algorithm can not only precisely learn all of the training data, but also alleviate the overfitting pain through the regularization term, softening and integrating mechanism.參考文獻 [1] B. Freisleben, and G. Gleichmann, “Controlling airline seat allocations with neural networks,” in System Sciences, Proceeding of the Twenty-Sixth Hawaii International Conference on. IEEE, vol. 4, pp. 635-642, January 1993.[2] C. Lemke, S. Riedel, and B. Gabrys, “Dynamic combination of forecasts generated by diversification procedures applied to forecasting of airline cancellations,” IEEE Symposium on Computational Intelligence for Financial Engineering Proceedings, pp. 85-91, 2009.[3] C. Meng, M. Sun, J. Yang, M. Qiu, and Y. Gu, “Training Deeper Models by GPU Memory Optimization on TensorFlow,” in Proc. of ML Systems Workshop in NIPS, December, 2017.[4] G. B. Huang, Q. Y. Zhu, and C. K. Siew, “Extreme learning machine: a new learning scheme of feedforward neural networks,” in Neural Networks, Proceedings, IEEE International Joint Conference on. IEEE, vol. 2, pp. 985-990, 2004.[5] H. S. Jenatabadi, and N. A. Ismail, “The determination of load factors in the airline industry,” International Review of Business Research Papers, vol. 3, no. 4, pp. 125-133, 2007.[6] K. Littlewood, “Special issue papers: Forecasting and control of passenger bookings,” Journal of Revenue and Pricing Management, vol. 4, no.2, pp. 111-123, 2005.[7] K. T. Wu, and F. C. Lin, “Forecasting airline seat show rates with neural networks,” in Neural Networks, IJCNN`99, International Joint Conference on. IEEE, vol. 6, pp. 3974-3977, July 1999.[8] L. Devriendt, G. Burghouwt, B. Derudder, J. D. Wit, and F. Witlox, “Calculating load factors for the transatlantic airline market using supply and demand data – A note on the identification of gaps in the available airline statistics,” Journal of Air Transport Management, vol. 15, no. 6, pp. 337-343, 2009.[9] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” in 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp. 165-283, 2016.[10] M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, and J. Kautz, “GA3C: GPU-based A3C for deep reinforcement learning,” CoRR abs/1611.06256, 2016.[11] M. O. Aomo, D. O. Oima, and M. N. Oginda, “An Empirical Investigation into the Effect of Enhancing Airline Capacity on Load Factor: A Case of Kenya’s Low-Cost Carriers,” American Journal of Industrial and Business Management, vol. 6, no. 6, pp. 717-731, 2016.[12] R. H. Tsaih, and T. C. Cheng, “A resistant learning procedure for coping with outliers,” Annals of Mathematics and Artificial Intelligence, vol. 57, no.2, pp. 161-180, 2009.[13] R. H. Tsaih, B. S. Kuo, T. H. Lin, and C. C. Hsu, “The use of big data analytics to predict the foreign exchange rate based on public media: A machine-learning experiment,” IT Professional, vol. 20, no. 2, pp. 34-41, 2018.[14] R. R. Tsaih, “An explanation of reasoning neural networks,” Mathematical and computer modelling, vol. 28, no. 2, pp. 37-44, 1998.[15] R. R. Tsaih, “The softening learning procedure,” Mathematical and computer modelling, vol. 18, no. 8, pp. 61-64, 1993.[16] S. Belciug, and F. Gorunescu, “Learning a single-hidden layer feedforward neural network using a rank correlation-based strategy with application to high dimensional gene expression and proteomic spectra datasets in cancer detection,” Journal of biomedical informatics, vol. 83, pp. 159-166, July 2018.[17] S. Y. Huang, J. W. Lin, and R. H. Tsaih, “Outlier detection in the concept drifting environment,” in Neural Networks, International Joint Conference on. IEEE, vol. 6, pp. 31-37, November 2016.[18] S. Y. Huang, R. H. Tsaih, and F. Yu, “Topological pattern discovery and feature extraction for fraudulent financial reporting,” Expert Systems with Applications, vol. 41, no. 9, pp. 4360-4372, 2014.[19] W. Peetawan, “Determination of passenger load factor: the case of Thai Airlines,” unpublished, April 2018.[20] X. S. Sun, E. Brauner, and S. Hormby, “A large-scale neural network for airline forecasting in revenue management,” in Operations Research in the Airline Industry, vol. 9, Springer: Boston, MA, 1998, pp. 46-67.[21] Y. Y. Tesfay, “Modified panel data regression model and its applications to the airline industry: Modeling the load factor of Europe North and Europe Mid Atlantic flights,” Journal of Traffic and Transportation Engineering, English ed., vol. 3, no. 4, pp. 283-295, 2016.[22] B. Xu, N. Wang, T. Chen, and M. Li, “Empirical Evaluation of Rectified Activations in Convolution Network,” arXiv preprint, arXiv: 1505.00853, 2005.[23] V. Nair, and G. E. Hinton, “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceeding of the 27th international conference on machine learning, pp. 807-814, 2010. 描述 碩士
國立政治大學
資訊管理學系
106356035資料來源 http://thesis.lib.nccu.edu.tw/record/#G0106356035 資料類型 thesis dc.contributor.advisor 蔡瑞煌 zh_TW dc.contributor.advisor Tsaih, Rua-Huan en_US dc.contributor.author (Authors) 梁惟婷 zh_TW dc.contributor.author (Authors) Liang, Wei-Ting en_US dc.creator (作者) 梁惟婷 zh_TW dc.creator (作者) Liang, Wei-Ting en_US dc.date (日期) 2019 en_US dc.date.accessioned 7-Aug-2019 16:08:17 (UTC+8) - dc.date.available 7-Aug-2019 16:08:17 (UTC+8) - dc.date.issued (上傳時間) 7-Aug-2019 16:08:17 (UTC+8) - dc.identifier (Other Identifiers) G0106356035 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/124717 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊管理學系 zh_TW dc.description (描述) 106356035 zh_TW dc.description.abstract (摘要) 本研究提出了一種基於序列的學習演算法─強記軟化及整合演算法。本研究所提出之演算法具有以下特色：（1） 實現自適應單隱藏層前饋神經網路（adaptive single-hidden layer feed-forward neural networks） （2） 利用最小裁減平方（least trimmed squares）原則決定訓練樣本的輸入序列 （3） 使用線性整流函數（rectified linear units）作為隱藏節點之激發函數 （4） 演算法能在精確學習所有訓練資料之下，同時緩解過度擬合的傾向。本研究對中華航空公司的實際數據進行實驗，以探索（1） 本研究所提出之強記軟化及整合演算法是否能模仿人類的學習方式（2） 本研究所提出之強記軟化及整合演算法是否能不僅精確學習所有的訓練資料，還能透過正規化項、軟化及整合機制來緩解過度擬合的傾向。 zh_TW dc.description.abstract (摘要) This study proposes the cramming, softening and integrating (CSI) algorithm, a sequentially-learning-based algorithm. The proposed CSI learning algorithm has the following features: (1) the implementation through the adaptive single-hidden layer feed-forward neural networks, (2) the usage of least trimmed squares principle for determining the sequence of learning samples, (3) the usage of rectified linear units activation function, (4) the practice of precisely learning all training data, while alleviating the overfitting pain. An experiment with real data from China Airlines has been conducted to explore (1) whether the proposed CSI algorithm can imitate the way of learning in human beings, as it claims, and (2) whether the proposed CSI algorithm can not only precisely learn all of the training data, but also alleviate the overfitting pain through the regularization term, softening and integrating mechanism. en_US dc.description.tableofcontents 謝辭 1摘要 2Abstract 3Content 4Table Index 6Figure Index 81. Introduction 92. Literature Review 122.1. The rectified linear units (ReLU) 122.2. The single-hidden layer feed-forward neural networks (SLFN) with one output node 122.3. The adaptive single-hidden layer feed-forward neural networks (ASLFN) with one output node 142.4. The back-propagation learning algorithm associated with SLFN with one output node 162.5. The Least Trimmed Squares 172.6. TensorFlow and GPU 182.7. The flight load factor in aviation industry 213. The Proposed CSI Learning Concept and Its Corresponding Learning Algorithm 274. Experimental Design 364.1. Experiment design 374.2. Variable description 384.3. Data sampling 415. Experimental Result 446. Summary and Future works 51Reference 53 zh_TW dc.format.extent 1546575 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0106356035 en_US dc.subject (關鍵詞) 強記軟化及整合學習 zh_TW dc.subject (關鍵詞) 強記機制 zh_TW dc.subject (關鍵詞) 軟化及整合機制 zh_TW dc.subject (關鍵詞) 最小裁減平方 zh_TW dc.subject (關鍵詞) 正規化 zh_TW dc.subject (關鍵詞) 線性整流函數 zh_TW dc.subject (關鍵詞) CSI learning en_US dc.subject (關鍵詞) Cramming mechanism en_US dc.subject (關鍵詞) Softening and integrating mechanism en_US dc.subject (關鍵詞) Least trimmed squares en_US dc.subject (關鍵詞) Regularization en_US dc.subject (關鍵詞) Rectified linear units en_US dc.title (題名) 強記軟化及整合演算法：以ReLU激發函數與實數輸入/輸出為例 zh_TW dc.title (題名) The Cramming, Softening and Integrating Learning Algorithm with ReLU Activation Function for Real Number Input / Output Problems en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] B. Freisleben, and G. Gleichmann, “Controlling airline seat allocations with neural networks,” in System Sciences, Proceeding of the Twenty-Sixth Hawaii International Conference on. IEEE, vol. 4, pp. 635-642, January 1993.[2] C. Lemke, S. Riedel, and B. Gabrys, “Dynamic combination of forecasts generated by diversification procedures applied to forecasting of airline cancellations,” IEEE Symposium on Computational Intelligence for Financial Engineering Proceedings, pp. 85-91, 2009.[3] C. Meng, M. Sun, J. Yang, M. Qiu, and Y. Gu, “Training Deeper Models by GPU Memory Optimization on TensorFlow,” in Proc. of ML Systems Workshop in NIPS, December, 2017.[4] G. B. Huang, Q. Y. Zhu, and C. K. Siew, “Extreme learning machine: a new learning scheme of feedforward neural networks,” in Neural Networks, Proceedings, IEEE International Joint Conference on. IEEE, vol. 2, pp. 985-990, 2004.[5] H. S. Jenatabadi, and N. A. Ismail, “The determination of load factors in the airline industry,” International Review of Business Research Papers, vol. 3, no. 4, pp. 125-133, 2007.[6] K. Littlewood, “Special issue papers: Forecasting and control of passenger bookings,” Journal of Revenue and Pricing Management, vol. 4, no.2, pp. 111-123, 2005.[7] K. T. Wu, and F. C. Lin, “Forecasting airline seat show rates with neural networks,” in Neural Networks, IJCNN`99, International Joint Conference on. IEEE, vol. 6, pp. 3974-3977, July 1999.[8] L. Devriendt, G. Burghouwt, B. Derudder, J. D. Wit, and F. Witlox, “Calculating load factors for the transatlantic airline market using supply and demand data – A note on the identification of gaps in the available airline statistics,” Journal of Air Transport Management, vol. 15, no. 6, pp. 337-343, 2009.[9] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” in 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp. 165-283, 2016.[10] M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, and J. Kautz, “GA3C: GPU-based A3C for deep reinforcement learning,” CoRR abs/1611.06256, 2016.[11] M. O. Aomo, D. O. Oima, and M. N. Oginda, “An Empirical Investigation into the Effect of Enhancing Airline Capacity on Load Factor: A Case of Kenya’s Low-Cost Carriers,” American Journal of Industrial and Business Management, vol. 6, no. 6, pp. 717-731, 2016.[12] R. H. Tsaih, and T. C. Cheng, “A resistant learning procedure for coping with outliers,” Annals of Mathematics and Artificial Intelligence, vol. 57, no.2, pp. 161-180, 2009.[13] R. H. Tsaih, B. S. Kuo, T. H. Lin, and C. C. Hsu, “The use of big data analytics to predict the foreign exchange rate based on public media: A machine-learning experiment,” IT Professional, vol. 20, no. 2, pp. 34-41, 2018.[14] R. R. Tsaih, “An explanation of reasoning neural networks,” Mathematical and computer modelling, vol. 28, no. 2, pp. 37-44, 1998.[15] R. R. Tsaih, “The softening learning procedure,” Mathematical and computer modelling, vol. 18, no. 8, pp. 61-64, 1993.[16] S. Belciug, and F. Gorunescu, “Learning a single-hidden layer feedforward neural network using a rank correlation-based strategy with application to high dimensional gene expression and proteomic spectra datasets in cancer detection,” Journal of biomedical informatics, vol. 83, pp. 159-166, July 2018.[17] S. Y. Huang, J. W. Lin, and R. H. Tsaih, “Outlier detection in the concept drifting environment,” in Neural Networks, International Joint Conference on. IEEE, vol. 6, pp. 31-37, November 2016.[18] S. Y. Huang, R. H. Tsaih, and F. Yu, “Topological pattern discovery and feature extraction for fraudulent financial reporting,” Expert Systems with Applications, vol. 41, no. 9, pp. 4360-4372, 2014.[19] W. Peetawan, “Determination of passenger load factor: the case of Thai Airlines,” unpublished, April 2018.[20] X. S. Sun, E. Brauner, and S. Hormby, “A large-scale neural network for airline forecasting in revenue management,” in Operations Research in the Airline Industry, vol. 9, Springer: Boston, MA, 1998, pp. 46-67.[21] Y. Y. Tesfay, “Modified panel data regression model and its applications to the airline industry: Modeling the load factor of Europe North and Europe Mid Atlantic flights,” Journal of Traffic and Transportation Engineering, English ed., vol. 3, no. 4, pp. 283-295, 2016.[22] B. Xu, N. Wang, T. Chen, and M. Li, “Empirical Evaluation of Rectified Activations in Convolution Network,” arXiv preprint, arXiv: 1505.00853, 2005.[23] V. Nair, and G. E. Hinton, “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceeding of the 27th international conference on machine learning, pp. 807-814, 2010. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU201900576 en_US
