Publications-Theses
Article View/Open
Publication Export
-
Google ScholarTM
NCCU Library
Citation Infomation
Related Publications in TAIR
題名 基於擴散式資料增強與SimSiam架構之時間序列自監督表示學習研究
Diffusion-Augmented Contrastive Representation Learning for Time-Series Forecasting作者 賴皓千
Lai, Hao-Chien貢獻者 蔡炎龍
Tsai, Yen-Lung
賴皓千
Lai, Hao-Chien關鍵詞 擴散模型
對比學習
時間序列
資料增強
股票市場
正樣本生成
模式一致性
結構保留
無監督學習
回報預測
異常檢測
Diffusion Models
Contrastive Learning
Time-Series Data
Data Augmentation
Stock Market
Positive Sample Generation
Pattern Consistency
Structural Preservation
Unsupervised Learning
Return Prediction
Anomaly Detection日期 2025 上傳時間 4-Aug-2025 13:10:41 (UTC+8) 摘要 在對比學習（Contrastive Learning）中，資料增強是生成正樣本的關鍵手段，對模型效果有著重要影響。在圖像數據中，常見的增強方法如裁剪、翻轉等可以生成有效的正樣本，但在時間序列數據中，這些方法可能破壞數據的時序結構及內部關係，導致模型學習效果下降。儘管擴散模型（Diffusion Models）已成為時間序列數據分析與預測的有效工具，但其在對比學習資料增強中的應用尚未被廣泛討論，部分原因在於傳統的擴散模型生成過程多依賴隨機採樣，難以生成與特定數據對應的正樣本。 為解決這一挑戰，本研究設計了一種針對時間序列數據的擴散模型應用手法，摒棄傳統隨機採樣策略，通過重新編輯數據生成具有模式一致性和結構保留的正樣本，並將其應用於對比學習框架。實驗結果表明，該方法在台灣股票市場數據上的應用顯著提升了模型的特徵表徵能力，在回報預測和異常檢測等下游任務中展現出優越性能，尤其是在資料稀缺或不平衡的情境下效果尤為顯著。本研究不僅填補了擴散模型在對比學習中的研究空白，還為時間序列數據的資料增強提供了一種新穎的解決方案。
Data augmentation is a critical component in contrastive learning (CL) for generating positive samples, significantly impacting the model’s performance. While common augmentation methods such as cropping and flipping are effective for image data, these approaches often disrupt the temporal structure and relationships in time-series data, leading to suboptimal learning outcomes. Although diffusion models have become powerful tools for analyzing and forecasting time-series data, their application in data augmentation for contrastive learning remains underexplored. One reason is that conventional diffusion model approaches rely on random sampling, which generates points from the data distribution rather than specific positive samples corresponding to existing data. To address this limitation, this study proposes a novel approach to applying diffusion models for time-series data. By discarding the traditional random sampling strategy, we utilize a tailored editing process to generate positive samples that preserve pattern consistency and structural integrity. These samples are then integrated into a contrastive learning framework. Experimental results demonstrate that the proposed method significantly enhances feature representation on Taiwan stock market data, achieving superior performance in downstream tasks such as return prediction and anomaly detection, particularly in data-scarce or imbalanced scenarios. This study not only bridges the gap in utilizing diffusion models for contrastive learning but also provides an innovative solution for time-series data augmentation.參考文獻 [1] Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In *International conference on machine learning* (pp. 1597-1607). PMLR. [2] Chen, X., & He, K. (2021). Exploring simple siamese representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 15750-15758). [3] Demirel, B. U., & Holz, C. (2023). Finding order in chaos: A novel data augmentation method for time series in contrastive learning. *Advances in Neural Information Processing Systems*, 36, 30750-30783. [4] Dhariwal, P., & Nichol, A. (2021). Diffusion models beat gans on image synthesis. *Advances in neural information processing systems*, 34, 8780-8794. [5] Esteban, C., Hyland, S. L., & Rätsch, G. (2017). Real-valued (medical) time series generation with recurrent conditional gans. [6] Guo, Z., Wang, H., Yang, J., & Miller, D. J. (2015). A stock market forecasting model combining two-directional two-dimensional principal component analysis and radial basis function neural network. *PloS one*, 10(4), e0122385. [7] He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 9729-9738). [8] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. *Advances in neural information processing systems*, 33, 6840-6851. [9] Iwana, B. K., & Uchida, S. (2021). An empirical survey of data augmentation for time series classification with neural networks. *PLOS ONE*, 16(7). [10] Jing, B., Wang, Y., Sui, G., Hong, J., He, J., Yang, Y., Li, D., & Ren, K. (2024, October). Automated contrastive learning strategy search for time series. In *Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM '24* (pp. 4612-4620). ACM. [11] Kalbande, D., Prabhu, P., Gharat, A., & Rajabally, T. (2021). A fraud detection system using machine learning. In *2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT)* (pp. 1-7). IEEE. [12] Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2021). Diffwave: A versatile diffusion model for audio synthesis. [13] Lee, S., Lee, G., Kim, H., Kim, J., & Uh, Y. (2023). Sequential data generation with groupwise diffusion process. [14] Lin, L., Li, Z., Li, R., Li, X., & Gao, J. (2024). Diffusion models for time-series applications: a survey. *Frontiers of Information Technology & Electronic Engineering*, 25(1), 19-41. [15] Luo, D., Cheng, W., Wang, Y., Xu, D., Ni, J., Yu, W., Zhang, X., Liu, Y., Chen, Y., Chen, H., et al. (2023). Time series contrastive learning with information-aware augmentations. In *Proceedings of the AAAI Conference on Artificial Intelligence* (Vol. 37, pp. 4534-4542). [16] Ma, C., & Yan, S. (2022). Deep learning in the chinese stock market: the role of technical indicators. *Finance Research Letters*, 49, 103025. [17] Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J. Y., & Ermon, S. (2022). Sdedit: Guided image synthesis and editing with stochastic differential equations. [18] Rasul, K., Seward, C., Schuster, I., & Vollgraf, R. (2021). Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In *International conference on machine learning* (pp. 8857-8868). PMLR. [19] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In *Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18* (pp. 234-241). Springer. [20] Shobayo, O., Adeyemi-Longe, S., Popoola, O., & Ogunleye, B. (2024, October). Innovative sentiment analysis and prediction of stock price using finbert, gpt-4 and logistic regression: A data-driven approach. *Big Data and Cognitive Computing*, 8(11), 143. [21] Solis-Martin, D., Galan-Paez, J., & Borrego-Diaz, J. (2023). D3a-ts: Denoising-driven data augmentation in time series. [22] Song, J., Meng, C., & Ermon, S. (2022). Denoising diffusion implicit models. [23] Wang, T., & Isola, P. (2020). Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In *International conference on machine learning* (pp. 9929-9939). PMLR. [24] Wang, W., Song, H., Si, S., Lu, W., & Cai, Z. (2024). Data augmentation based on diffusion probabilistic model for remaining useful life estimation of aero-engines. *Reliability Engineering & System Safety*, 252, 110394. [25] Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021, August). Time series data augmentation for deep learning: A survey. In *Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-2021* (pp. 4653-4660). International Joint Conferences on Artificial Intelligence Organization. [26] Yoon, J., Jarrett, D., & van der Schaar, M. (2019). Time-series generative adversarial networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, & R. Garnett (Eds.), *Advances in Neural Information Processing Systems* (Vol. 32). Curran Associates, Inc. [27] Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021). A transformer-based framework for multivariate time series representation learning. In *Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining* (pp. 2114-2124). 描述 碩士
國立政治大學
應用數學系
111751016資料來源 http://thesis.lib.nccu.edu.tw/record/#G0111751016 資料類型 thesis dc.contributor.advisor 蔡炎龍 zh_TW dc.contributor.advisor Tsai, Yen-Lung en_US dc.contributor.author (Authors) 賴皓千 zh_TW dc.contributor.author (Authors) Lai, Hao-Chien en_US dc.creator (作者) 賴皓千 zh_TW dc.creator (作者) Lai, Hao-Chien en_US dc.date (日期) 2025 en_US dc.date.accessioned 4-Aug-2025 13:10:41 (UTC+8) - dc.date.available 4-Aug-2025 13:10:41 (UTC+8) - dc.date.issued (上傳時間) 4-Aug-2025 13:10:41 (UTC+8) - dc.identifier (Other Identifiers) G0111751016 en_US dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/158370 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 應用數學系 zh_TW dc.description (描述) 111751016 zh_TW dc.description.abstract (摘要) 在對比學習（Contrastive Learning）中，資料增強是生成正樣本的關鍵手段，對模型效果有著重要影響。在圖像數據中，常見的增強方法如裁剪、翻轉等可以生成有效的正樣本，但在時間序列數據中，這些方法可能破壞數據的時序結構及內部關係，導致模型學習效果下降。儘管擴散模型（Diffusion Models）已成為時間序列數據分析與預測的有效工具，但其在對比學習資料增強中的應用尚未被廣泛討論，部分原因在於傳統的擴散模型生成過程多依賴隨機採樣，難以生成與特定數據對應的正樣本。 為解決這一挑戰，本研究設計了一種針對時間序列數據的擴散模型應用手法，摒棄傳統隨機採樣策略，通過重新編輯數據生成具有模式一致性和結構保留的正樣本，並將其應用於對比學習框架。實驗結果表明，該方法在台灣股票市場數據上的應用顯著提升了模型的特徵表徵能力，在回報預測和異常檢測等下游任務中展現出優越性能，尤其是在資料稀缺或不平衡的情境下效果尤為顯著。本研究不僅填補了擴散模型在對比學習中的研究空白，還為時間序列數據的資料增強提供了一種新穎的解決方案。 zh_TW dc.description.abstract (摘要) Data augmentation is a critical component in contrastive learning (CL) for generating positive samples, significantly impacting the model’s performance. While common augmentation methods such as cropping and flipping are effective for image data, these approaches often disrupt the temporal structure and relationships in time-series data, leading to suboptimal learning outcomes. Although diffusion models have become powerful tools for analyzing and forecasting time-series data, their application in data augmentation for contrastive learning remains underexplored. One reason is that conventional diffusion model approaches rely on random sampling, which generates points from the data distribution rather than specific positive samples corresponding to existing data. To address this limitation, this study proposes a novel approach to applying diffusion models for time-series data. By discarding the traditional random sampling strategy, we utilize a tailored editing process to generate positive samples that preserve pattern consistency and structural integrity. These samples are then integrated into a contrastive learning framework. Experimental results demonstrate that the proposed method significantly enhances feature representation on Taiwan stock market data, achieving superior performance in downstream tasks such as return prediction and anomaly detection, particularly in data-scarce or imbalanced scenarios. This study not only bridges the gap in utilizing diffusion models for contrastive learning but also provides an innovative solution for time-series data augmentation. en_US dc.description.tableofcontents 致謝 ii 中文摘要 iii Abstract iv Contents vi List of Tables viii List of Figures ix 1 Introduction 1 1.1 Research Background 1 1.2 Research Problem 2 1.3 Research Status 3 1.4 Existing Problems and Limitations 4 1.5 Research Objectives and Scope 6 1.5.1 Research Objectives 6 1.5.2 Scope of the Study 7 2 Methodology 9 2.1 Overview of the Proposed Method 9 2.2 Diffusion-Based Augmentation for Time Series 10 2.2.1 Fundamentals and Mathematical Formulation of Diffusion Models 11 2.2.2 Application to Data Augmentation 12 2.2.3 Architecture of the Denoising Network 13 2.3 SimSiam Architecture 15 2.3.1 Data and Augmentation 15 2.3.2 Siamese Network Structure 16 2.3.3 Loss Function 17 2.3.4 Training Flow 18 2.3.5 Advantages of Diffusion-Based Augmentation 18 2.4 Advantages of the Proposed Method 19 3 Experimental Design and Training Process 20 3.1 Dataset Description 20 3.2 Data Processing and Preprocessing 21 3.3 Diffusion Model Training 22 3.4 SimSiam Training Procedure 23 3.5 Downstream Task and Evaluation Metrics 25 4 Experimental Results and Analysis 27 4.1 Impact of Data Augmentation Methods and Encoder Architectures on Performance 27 4.2 Effect of Diffusion Steps on Model Performance 28 4.3 Comparison and Summary of Model Variations 29 4.4 Evaluation Metrics and Their Relevance in Financial Classification 32 5 Conclusion 35 Appendix A Implementation Details 38 A.1 Diffusion-Based Augmentation Function 38 A.2 Transformer Encoder for Time-Series 40 A.3 SimSiam Model and Loss for Time-Series 41 A.4 Training Loop for SimSiam with Diffusion-Based Augmentation 42 Bibliography 45 zh_TW dc.format.extent 1636391 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0111751016 en_US dc.subject (關鍵詞) 擴散模型 zh_TW dc.subject (關鍵詞) 對比學習 zh_TW dc.subject (關鍵詞) 時間序列 zh_TW dc.subject (關鍵詞) 資料增強 zh_TW dc.subject (關鍵詞) 股票市場 zh_TW dc.subject (關鍵詞) 正樣本生成 zh_TW dc.subject (關鍵詞) 模式一致性 zh_TW dc.subject (關鍵詞) 結構保留 zh_TW dc.subject (關鍵詞) 無監督學習 zh_TW dc.subject (關鍵詞) 回報預測 zh_TW dc.subject (關鍵詞) 異常檢測 zh_TW dc.subject (關鍵詞) Diffusion Models en_US dc.subject (關鍵詞) Contrastive Learning en_US dc.subject (關鍵詞) Time-Series Data en_US dc.subject (關鍵詞) Data Augmentation en_US dc.subject (關鍵詞) Stock Market en_US dc.subject (關鍵詞) Positive Sample Generation en_US dc.subject (關鍵詞) Pattern Consistency en_US dc.subject (關鍵詞) Structural Preservation en_US dc.subject (關鍵詞) Unsupervised Learning en_US dc.subject (關鍵詞) Return Prediction en_US dc.subject (關鍵詞) Anomaly Detection en_US dc.title (題名) 基於擴散式資料增強與SimSiam架構之時間序列自監督表示學習研究 zh_TW dc.title (題名) Diffusion-Augmented Contrastive Representation Learning for Time-Series Forecasting en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [1] Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In *International conference on machine learning* (pp. 1597-1607). PMLR. [2] Chen, X., & He, K. (2021). Exploring simple siamese representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 15750-15758). [3] Demirel, B. U., & Holz, C. (2023). Finding order in chaos: A novel data augmentation method for time series in contrastive learning. *Advances in Neural Information Processing Systems*, 36, 30750-30783. [4] Dhariwal, P., & Nichol, A. (2021). Diffusion models beat gans on image synthesis. *Advances in neural information processing systems*, 34, 8780-8794. [5] Esteban, C., Hyland, S. L., & Rätsch, G. (2017). Real-valued (medical) time series generation with recurrent conditional gans. [6] Guo, Z., Wang, H., Yang, J., & Miller, D. J. (2015). A stock market forecasting model combining two-directional two-dimensional principal component analysis and radial basis function neural network. *PloS one*, 10(4), e0122385. [7] He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition* (pp. 9729-9738). [8] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. *Advances in neural information processing systems*, 33, 6840-6851. [9] Iwana, B. K., & Uchida, S. (2021). An empirical survey of data augmentation for time series classification with neural networks. *PLOS ONE*, 16(7). [10] Jing, B., Wang, Y., Sui, G., Hong, J., He, J., Yang, Y., Li, D., & Ren, K. (2024, October). Automated contrastive learning strategy search for time series. In *Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM '24* (pp. 4612-4620). ACM. [11] Kalbande, D., Prabhu, P., Gharat, A., & Rajabally, T. (2021). A fraud detection system using machine learning. In *2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT)* (pp. 1-7). IEEE. [12] Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2021). Diffwave: A versatile diffusion model for audio synthesis. [13] Lee, S., Lee, G., Kim, H., Kim, J., & Uh, Y. (2023). Sequential data generation with groupwise diffusion process. [14] Lin, L., Li, Z., Li, R., Li, X., & Gao, J. (2024). Diffusion models for time-series applications: a survey. *Frontiers of Information Technology & Electronic Engineering*, 25(1), 19-41. [15] Luo, D., Cheng, W., Wang, Y., Xu, D., Ni, J., Yu, W., Zhang, X., Liu, Y., Chen, Y., Chen, H., et al. (2023). Time series contrastive learning with information-aware augmentations. In *Proceedings of the AAAI Conference on Artificial Intelligence* (Vol. 37, pp. 4534-4542). [16] Ma, C., & Yan, S. (2022). Deep learning in the chinese stock market: the role of technical indicators. *Finance Research Letters*, 49, 103025. [17] Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J. Y., & Ermon, S. (2022). Sdedit: Guided image synthesis and editing with stochastic differential equations. [18] Rasul, K., Seward, C., Schuster, I., & Vollgraf, R. (2021). Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In *International conference on machine learning* (pp. 8857-8868). PMLR. [19] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In *Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18* (pp. 234-241). Springer. [20] Shobayo, O., Adeyemi-Longe, S., Popoola, O., & Ogunleye, B. (2024, October). Innovative sentiment analysis and prediction of stock price using finbert, gpt-4 and logistic regression: A data-driven approach. *Big Data and Cognitive Computing*, 8(11), 143. [21] Solis-Martin, D., Galan-Paez, J., & Borrego-Diaz, J. (2023). D3a-ts: Denoising-driven data augmentation in time series. [22] Song, J., Meng, C., & Ermon, S. (2022). Denoising diffusion implicit models. [23] Wang, T., & Isola, P. (2020). Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In *International conference on machine learning* (pp. 9929-9939). PMLR. [24] Wang, W., Song, H., Si, S., Lu, W., & Cai, Z. (2024). Data augmentation based on diffusion probabilistic model for remaining useful life estimation of aero-engines. *Reliability Engineering & System Safety*, 252, 110394. [25] Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021, August). Time series data augmentation for deep learning: A survey. In *Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-2021* (pp. 4653-4660). International Joint Conferences on Artificial Intelligence Organization. [26] Yoon, J., Jarrett, D., & van der Schaar, M. (2019). Time-series generative adversarial networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, & R. Garnett (Eds.), *Advances in Neural Information Processing Systems* (Vol. 32). Curran Associates, Inc. [27] Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021). A transformer-based framework for multivariate time series representation learning. In *Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining* (pp. 2114-2124). zh_TW
