學術產出-學位論文
文章檢視/開啟
書目匯出
-
題名 應用於長期時間序列預測的新穎學習機制
An Advanced Learning Mechanism for Long-Term Time-Series Forecasting作者 李鴻禧
Lee, Hung-Hsi貢獻者 蔡瑞煌<br>郭炳伸
Tsaih, Rua-Huan<br>Kuo, Biing-Shen
李鴻禧
Lee, Hung-Hsi關鍵詞 單層線性模型
長期時間序列預測
多變量預測任務
黃金價格
概念漂移
移動窗口機制
單隱藏層前饋神經網絡
自適應單隱藏層前饋神經網絡
Single-layer linear model
Long-term time series forecasting
Multivariate forecasting tasks
Gold prices
Concept drift
Moving window mechanism
Single-hidden layer feedforward neural network
Adaptive SLFN model日期 2024 上傳時間 4-九月-2024 14:03:32 (UTC+8) 摘要 本研究受到Zeng, Chen, Zhang, & Xu (2023)發現單層線性模型在長期時間序列預測（LTSF）中出乎意料的有效性啟發，該模型在多變量預測任務中的表現超越了現有的基於Transformer的模型。考慮到黃金的獨特性及其作為一個獨立資產類別的地位，本研究選擇黃金價格作為研究樣本。我們關注黃金價格預測中面臨的非穩態學習挑戰——概念漂移，並探索使用移動窗口機制搭配單隱藏層前饋神經網絡（SLFN）作為一種類似單層線性模型的結構較簡單的神經網絡模型來解決此問題。為了克服模型訓練過程中遇到的梯度消失和過擬合問題，我們提出了IOSFCR機制來調整SLFN模型裡面的隱藏節點數量以增強模型的適應性和預測能力，並將此SLFN模型命名為自適應單隱藏層前饋神經網路（Adaptive SLFN）模型。本研究旨在評估IOSFCR機制對於訓練Adaptive SLFN模型的有效性，並比較其預測結果與當前在預測時間序列的領域上最先進的Transformer模型，FEDformer的性能。
This study is inspired by the findings of Zeng, Chen, Zhang, & Xu (2023), which highlighted the unexpected efficacy of single-layer linear models in long-term time series forecasting (LTSF), outperforming existing Transformer-based models in multivariate forecasting tasks. Given gold's unique properties and its status as a distinct asset class, this research selects gold prices as the sample. We address the non-stationary learning challenge of concept drift in forecasting gold prices and explore the use of a moving window mechanism combined with a single-hidden layer feedforward neural network (SLFN) as a simpler neural network model, akin to a single-layer linear model, to solve this issue. To overcome the challenges of vanishing gradient and overfitting encountered during model training, we introduce the IOSFCR mechanism to adjust the number of hidden nodes within the SLFN model to enhance the model's adaptability and forecasting capability, and we name this enhanced SLFN model as the adaptive single-hidden layer feedforward neural network (Adaptive SLFN) model. The aim of this study is to assess the effectiveness of the IOSFCR mechanism in training the Adaptive SLFN model and to compare its forecasting performance against the current state-of-the-art Transformer model in the realm of time series forecasting, FEDformer.參考文獻 Baur, D. G., & McDermott, T. K. (2010). Is gold a safe haven? International evidence. Journal of Banking & Finance, 34(8), 1886-1898. Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons. Brockwell, P. J., & Davis, R. A. (Eds.). (2002). Introduction to time series and forecasting. New York, NY: Springer New York. Cai, J., Cheung, Y. L., & Wong, M. C. (2001). What moves the gold market?. Journal of Futures Markets: Futures, Options, and Other Derivative Products, 21(3), 257-278. Cheng, S., Wu, Y., Li, Y., Yao, F., & Min, F. (2021). TWD-SFNN: Three-way decisions with a single hidden layer feedforward neural network. Information Sciences, 579, 15-32. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Erb, C. B., & Harvey, C. R. (2013). The golden dilemma. Financial Analysts Journal, 69(4), 10-42. Ghosh, D., Levin, E. J., Macmillan, P., & Wright, R. E. (2004). Gold as an inflation hedge?. Studies in Economics and Finance, 22(1), 1-25. Granger, C. W., & Teräsvirta, T. (1993). Modelling nonlinear economic relationships. Oxford University Press. Guha, B., & Bandyopadhyay, G. (2016). Gold price forecasting using ARIMA model. Journal of Advanced Management Science, 4(2). He, Z., Zhou, J., Dai, H. N., & Wang, H. (2019, August). Gold price forecast based on LSTM-CNN model. In 2019 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) (pp. 1046-1053). IEEE. Koychev, I. (2000). Gradual forgetting for adaptation to concept drift. Proceedings of ECAI 2000 Workshop on Current Issues in Spatio-Temporal Reasoning. Kumar, M., & Anand, M. (2014). An application of time series ARIMA forecasting model for predicting sugarcane production in India. Studies in Business and Economics, 9(1), 81-94. Liu, A., Zhang, G., & Lu, J. (2017, July). Fuzzy time windowing for gradual concept drift adaptation. In 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 1-6). IEEE. Liu, D., & Li, Z. (2017). Gold price forecasting and related influence factors analysis based on random forest. In Proceedings of the Tenth International Conference on Management Science and Engineering Management (pp. 711-723). Springer Singapore. Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under concept drift: A review. IEEE transactions on knowledge and data engineering, 31(12), 2346-2363. Mahdavi, S., & Zhou, S. (1997). Gold and commodity prices as leading indicators of inflation: Tests of long-run relationship and predictive performance. Journal of Economics and Business, 49(5), 475-489. O'Connor, F. A., Lucey, B. M., Batten, J. A., & Baur, D. G. (2015). The financial economics of gold—A survey. International Review of Financial Analysis, 41, 186-205. Pankratz, A. (2009). Forecasting with univariate Box-Jenkins models: Concepts and cases. John Wiley & Sons. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9. Reboredo, J. C., & Rivera-Castro, M. A. (2014). Can gold hedge and preserve value when the US dollar depreciates?. Economic Modelling, 39, 168-173. Shafiee, S., & Topal, E. (2010). An overview of global gold market and gold price forecasting. Resources policy, 35(3), 178-189. Tsaih, R. R. (1998). An explanation of reasoning neural networks. Mathematical and computer modelling, 28(2), 37-44. Tully, E., & Lucey, B. M. (2007). A power GARCH examination of the gold market. Research in International Business and Finance, 21(2), 316-325. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. Wang, Y. S., & Chueh, Y. L. (2013). Dynamic transmission effects between the interest rate, the US dollar, and gold and crude oil prices. Economic Modelling, 30, 792-798. Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50, 159-175. Zeng, A., Chen, M., Zhang, L., & Xu, Q. (2023, June). Are transformers effective for time series forecasting?. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, No. 9, pp. 11121-11128). Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022, June). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning (pp. 27268-27286). PMLR. Žliobaitė, I. (2010). Learning under concept drift: an overview. arXiv preprint arXiv:1010.4784. 描述 碩士
國立政治大學
資訊管理學系
111356013資料來源 http://thesis.lib.nccu.edu.tw/record/#G0111356013 資料類型 thesis dc.contributor.advisor 蔡瑞煌<br>郭炳伸 zh_TW dc.contributor.advisor Tsaih, Rua-Huan<br>Kuo, Biing-Shen en_US dc.contributor.author (作者) 李鴻禧 zh_TW dc.contributor.author (作者) Lee, Hung-Hsi en_US dc.creator (作者) 李鴻禧 zh_TW dc.creator (作者) Lee, Hung-Hsi en_US dc.date (日期) 2024 en_US dc.date.accessioned 4-九月-2024 14:03:32 (UTC+8) - dc.date.available 4-九月-2024 14:03:32 (UTC+8) - dc.date.issued (上傳時間) 4-九月-2024 14:03:32 (UTC+8) - dc.identifier (其他 識別碼) G0111356013 en_US dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/153149 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊管理學系 zh_TW dc.description (描述) 111356013 zh_TW dc.description.abstract (摘要) 本研究受到Zeng, Chen, Zhang, & Xu (2023)發現單層線性模型在長期時間序列預測（LTSF）中出乎意料的有效性啟發，該模型在多變量預測任務中的表現超越了現有的基於Transformer的模型。考慮到黃金的獨特性及其作為一個獨立資產類別的地位，本研究選擇黃金價格作為研究樣本。我們關注黃金價格預測中面臨的非穩態學習挑戰——概念漂移，並探索使用移動窗口機制搭配單隱藏層前饋神經網絡（SLFN）作為一種類似單層線性模型的結構較簡單的神經網絡模型來解決此問題。為了克服模型訓練過程中遇到的梯度消失和過擬合問題，我們提出了IOSFCR機制來調整SLFN模型裡面的隱藏節點數量以增強模型的適應性和預測能力，並將此SLFN模型命名為自適應單隱藏層前饋神經網路（Adaptive SLFN）模型。本研究旨在評估IOSFCR機制對於訓練Adaptive SLFN模型的有效性，並比較其預測結果與當前在預測時間序列的領域上最先進的Transformer模型，FEDformer的性能。 zh_TW dc.description.abstract (摘要) This study is inspired by the findings of Zeng, Chen, Zhang, & Xu (2023), which highlighted the unexpected efficacy of single-layer linear models in long-term time series forecasting (LTSF), outperforming existing Transformer-based models in multivariate forecasting tasks. Given gold's unique properties and its status as a distinct asset class, this research selects gold prices as the sample. We address the non-stationary learning challenge of concept drift in forecasting gold prices and explore the use of a moving window mechanism combined with a single-hidden layer feedforward neural network (SLFN) as a simpler neural network model, akin to a single-layer linear model, to solve this issue. To overcome the challenges of vanishing gradient and overfitting encountered during model training, we introduce the IOSFCR mechanism to adjust the number of hidden nodes within the SLFN model to enhance the model's adaptability and forecasting capability, and we name this enhanced SLFN model as the adaptive single-hidden layer feedforward neural network (Adaptive SLFN) model. The aim of this study is to assess the effectiveness of the IOSFCR mechanism in training the Adaptive SLFN model and to compare its forecasting performance against the current state-of-the-art Transformer model in the realm of time series forecasting, FEDformer. en_US dc.description.tableofcontents 第一章 緒論 1 第二章 文獻探討 4 第一節 預測變數 4 第二節 長期時間序列預測研究 6 2.2.1 ARIMA模型 6 2.2.2 單隱藏層前饋神經網絡 7 2.2.3 基於Transformer的模型 7 第三節 楊氏(2020)自適應學習預測模型 8 2.3.1 概念漂移與移動窗口 8 2.3.2 SS機制 9 第三章 提出進階學習演算法 11 第一節 移動窗口機制 12 第二節 IOSFCR機制 13 第三節 測試模型 22 第四章 實驗方法與驗證 23 第一節 數據描述 23 第二節 數據預處理 25 第三節 驗證與評估 25 第五章 實驗結果 28 第一節 IOSFCR機制驗證 28 5.1.1 第一個窗口的評估 28 5.1.2 增加的隱藏節點數量 31 5.1.3 修剪的隱藏節點數量 34 5.1.4 採用的隱藏節點數量 35 5.1.5 訓練時間 37 5.1.6 表現的驗證 39 第二節 模型性能評估 41 第六章 結論與未來工作 44 第一節 摘要 44 第二節 限制與未來工作 45 參考文獻 47 zh_TW dc.format.extent 1673044 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0111356013 en_US dc.subject (關鍵詞) 單層線性模型 zh_TW dc.subject (關鍵詞) 長期時間序列預測 zh_TW dc.subject (關鍵詞) 多變量預測任務 zh_TW dc.subject (關鍵詞) 黃金價格 zh_TW dc.subject (關鍵詞) 概念漂移 zh_TW dc.subject (關鍵詞) 移動窗口機制 zh_TW dc.subject (關鍵詞) 單隱藏層前饋神經網絡 zh_TW dc.subject (關鍵詞) 自適應單隱藏層前饋神經網絡 zh_TW dc.subject (關鍵詞) Single-layer linear model en_US dc.subject (關鍵詞) Long-term time series forecasting en_US dc.subject (關鍵詞) Multivariate forecasting tasks en_US dc.subject (關鍵詞) Gold prices en_US dc.subject (關鍵詞) Concept drift en_US dc.subject (關鍵詞) Moving window mechanism en_US dc.subject (關鍵詞) Single-hidden layer feedforward neural network en_US dc.subject (關鍵詞) Adaptive SLFN model en_US dc.title (題名) 應用於長期時間序列預測的新穎學習機制 zh_TW dc.title (題名) An Advanced Learning Mechanism for Long-Term Time-Series Forecasting en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) Baur, D. G., & McDermott, T. K. (2010). Is gold a safe haven? International evidence. Journal of Banking & Finance, 34(8), 1886-1898. Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons. Brockwell, P. J., & Davis, R. A. (Eds.). (2002). Introduction to time series and forecasting. New York, NY: Springer New York. Cai, J., Cheung, Y. L., & Wong, M. C. (2001). What moves the gold market?. Journal of Futures Markets: Futures, Options, and Other Derivative Products, 21(3), 257-278. Cheng, S., Wu, Y., Li, Y., Yao, F., & Min, F. (2021). TWD-SFNN: Three-way decisions with a single hidden layer feedforward neural network. Information Sciences, 579, 15-32. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Erb, C. B., & Harvey, C. R. (2013). The golden dilemma. Financial Analysts Journal, 69(4), 10-42. Ghosh, D., Levin, E. J., Macmillan, P., & Wright, R. E. (2004). Gold as an inflation hedge?. Studies in Economics and Finance, 22(1), 1-25. Granger, C. W., & Teräsvirta, T. (1993). Modelling nonlinear economic relationships. Oxford University Press. Guha, B., & Bandyopadhyay, G. (2016). Gold price forecasting using ARIMA model. Journal of Advanced Management Science, 4(2). He, Z., Zhou, J., Dai, H. N., & Wang, H. (2019, August). Gold price forecast based on LSTM-CNN model. In 2019 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) (pp. 1046-1053). IEEE. Koychev, I. (2000). Gradual forgetting for adaptation to concept drift. Proceedings of ECAI 2000 Workshop on Current Issues in Spatio-Temporal Reasoning. Kumar, M., & Anand, M. (2014). An application of time series ARIMA forecasting model for predicting sugarcane production in India. Studies in Business and Economics, 9(1), 81-94. Liu, A., Zhang, G., & Lu, J. (2017, July). Fuzzy time windowing for gradual concept drift adaptation. In 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 1-6). IEEE. Liu, D., & Li, Z. (2017). Gold price forecasting and related influence factors analysis based on random forest. In Proceedings of the Tenth International Conference on Management Science and Engineering Management (pp. 711-723). Springer Singapore. Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under concept drift: A review. IEEE transactions on knowledge and data engineering, 31(12), 2346-2363. Mahdavi, S., & Zhou, S. (1997). Gold and commodity prices as leading indicators of inflation: Tests of long-run relationship and predictive performance. Journal of Economics and Business, 49(5), 475-489. O'Connor, F. A., Lucey, B. M., Batten, J. A., & Baur, D. G. (2015). The financial economics of gold—A survey. International Review of Financial Analysis, 41, 186-205. Pankratz, A. (2009). Forecasting with univariate Box-Jenkins models: Concepts and cases. John Wiley & Sons. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9. Reboredo, J. C., & Rivera-Castro, M. A. (2014). Can gold hedge and preserve value when the US dollar depreciates?. Economic Modelling, 39, 168-173. Shafiee, S., & Topal, E. (2010). An overview of global gold market and gold price forecasting. Resources policy, 35(3), 178-189. Tsaih, R. R. (1998). An explanation of reasoning neural networks. Mathematical and computer modelling, 28(2), 37-44. Tully, E., & Lucey, B. M. (2007). A power GARCH examination of the gold market. Research in International Business and Finance, 21(2), 316-325. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. Wang, Y. S., & Chueh, Y. L. (2013). Dynamic transmission effects between the interest rate, the US dollar, and gold and crude oil prices. Economic Modelling, 30, 792-798. Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50, 159-175. Zeng, A., Chen, M., Zhang, L., & Xu, Q. (2023, June). Are transformers effective for time series forecasting?. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, No. 9, pp. 11121-11128). Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022, June). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning (pp. 27268-27286). PMLR. Žliobaitė, I. (2010). Learning under concept drift: an overview. arXiv preprint arXiv:1010.4784. zh_TW
