1. NCCU Academic Hub
  2. Publications
  3. College of Informatics
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 結合彈性壓縮技術之機器遺忘機制
Machine Unlearning Mechanisms with Flexible Compression Schemes
作者 羅鈺涵
Lo, Yu-Han
貢獻者 廖文宏
Liao, Wen-Hung
羅鈺涵
Lo, Yu-Han
關鍵詞 深度學習
神經網路
模型壓縮
機器遺忘
可解釋性人工智慧
Deep Learning
Neural Network
Model Compression
Machine Unlearning
XAI
日期 2025
上傳時間 3-Mar-2025 14:04:12 (UTC+8)
摘要 隨著機器學習的蓬勃發展，特別是生成式人工智慧模型的興起，AI 技術已從專業領域延伸至大眾生活。然而，機器學習模型仰賴大量訓練資料的特性，引發智慧財產權與個人隱私的重要議題。歐盟的《一般資料保護規則》(GDPR) 和美國《加州消費者隱私保護法》(CCPA) 明確規範資料遺忘權，但在深度學習時代，單純刪除原始資料已不足以保護隱私，因為這些資料在訓練過程中已深植於模型參數中。 為了有效解決這個問題，本論文提出一套結合彈性壓縮技術的機器遺忘機制。這種方法不僅能快速且有效地從模型中移除指定資訊，更能透過模型壓縮與稀疏化技術，大幅降低計算成本並提升效率。我們的方法透過零值填充和重新稀疏化訓練的方式，實現靈活的漸進式遺忘，使模型能夠在多輪遺忘操作後仍維持良好性能。 與傳統方法相比，本研究提出的彈性壓縮遺忘機制在運算效率與隱私保護之間取得更好的平衡。透過對模型參數的動態稀疏化和彈性壓縮，不僅能有效移除遺忘資料集的相關資訊，還能防止潛在的成員推斷攻擊和模型反轉攻擊。這種方法特別適合計算資源受限且需要高度隱私保護的應用場景，為實際部署提供一個兼具效率與安全性的解決方案。
With the rapid advancement of machine learning, particularly the emergence of generative AI models, AI technology has expanded from professional domains to everyday life. However, machine learning models' reliance on extensive training data raises significant concerns regarding intellectual property rights and personal privacy. While the European Union's General Data Protection Regulation (GDPR) and California's Consumer Privacy Protection Act (CCPA) explicitly regulate the right to be forgotten, merely deleting original data is insufficient for privacy protection in the deep learning era, as this data becomes deeply embedded within model parameters during training. To address this challenge, this thesis proposes a novel machine unlearning mechanism integrated with flexible compression techniques. This approach not only enables swift and effective removal of specified information from models but also significantly reduces computational costs and improves efficiency through model compression and sparsification techniques. Our method achieves flexible progressive unlearning through zero-value filling and re-sparsification training, enabling models to maintain high performance even after multiple rounds of unlearning operations. Compared to traditional approaches, our proposed flexible compression unlearning mechanism achieves a better balance between computational efficiency and privacy protection. Through dynamic sparsification and flexible compression of model parameters, we effectively remove information related to the forgotten dataset while preventing potential membership inference attacks and model inversion attacks. This method is particularly suitable for applications with limited computational resources requiring high privacy protection, providing a practical deployment solution that combines both efficiency and security.
參考文獻 1. S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” Advances in Neural Information Processing Systems, vol. 28, 2015. 2. S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149, 2015. 3. G. Fang, X. Ma, M. Song, M. B. Mi, and X. Wang, “Depgraph: Towards any structural pruning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16091–16101, 2023. 4. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016. 5. P. P. Ray, “Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,” Internet of Things and Cyber-Physical Systems, vol. 3, pp. 121–154, 2023. 6. P. Regulation, “Regulation (EU) 2016/679.” Official Journal of the European Union, 2016. General Data Protection Regulation (GDPR). 7. D. U. CCPA, “California Consumer Privacy Act (CCPA) Website Policy,” 2020. 8. Government of Japan, “Amended act on the protection of personal information,” 2016. Accessed: 2024-06-01. 9. National People’s Congress of the People’s Republic of China, “Personal information protection law of the people’s republic of China,” 2021. Accessed: 2024-06-01. 10. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159, IEEE, 2021. 11. R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18, IEEE, 2017. 12. “NeurIPS 2023 Machine Unlearning Challenge.” https://unlearning-challenge.github.io/, 2023. Accessed: 2024/4/5. 13. E. Ullah, T. Mai, A. Rao, R. A. Rossi, and R. Arora, “Machine unlearning via algorithmic stability,” in Conference on Learning Theory, pp. 4126–4142, PMLR, 2021. 14. A. Ginart, M. Guan, G. Valiant, and J. Zou, “Making AI forget you: Data deletion in machine learning,” Advances in Neural Information Processing Systems (NeurIPS), 2019. 15. L. Graves, V. Nagisetty, and V. Ganesh, “Amnesiac machine learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 13, pp. 11516–11524, 2021. 16. Y. Cao, J. Yang, and C. Yang, “Towards making systems forget with machine unlearning,” in IEEE Symposium on Security and Privacy (SP), 2015. 17. S. Schelter, D. Kossmann, M. Zeller, and A. Halevy, “The case for data versioning in machine learning,” in Proceedings of the 2021 International Conference on Management of Data (SIGMOD), 2021. 18. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Proceedings of the 36th International Conference on Machine Learning (ICML), 2019. 19. J. Wang, S. Guo, X. Xie, and H. Qi, “Federated unlearning via class-discriminative pruning,” in Proceedings of the ACM Web Conference 2022, pp. 622–632, 2022. 20. A. Golatkar, A. Achille, and S. Soatto, “Eternal sunshine of the spotless net: Selective forgetting in deep networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304–9312, 2020. 21. A. Thudi, H. Jia, M. Goldblum, T. Goldstein, and A. Shrivastava, “Model agnostic unlearning,” arXiv preprint arXiv:2108.11577, 2021. 22. A. K. Tarun, V. S. Chundawat, M. Mandal, and M. Kankanhalli, “Fast yet effective machine unlearning,” IEEE Transactions on Neural Networks and Learning Systems, 2023. 23. H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” arXiv preprint arXiv:1608.08710, 2016. 24. N.-W. Chen, “A compression mechanism of neural networks based on convolution kernel redundancy,” Journal of Machine Learning Research, vol. 22, no. 1, pp. 123–135, 2021. 25. A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, 05 2012. 26. V. S. Chundawat, A. K. Tarun, M. Mandal, and M. Kankanhalli, “Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 6, pp. 7210–7217, 2023. 27. J. Lin, “Divergence measures based on the Shannon entropy,” IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 145–151, 1991. 28. P. Xia, L. Zhang, and F. Li, “Learning similarity with cosine similarity ensemble,” Information Sciences, vol. 307, pp. 39–52, 2015.
描述 碩士
國立政治大學
資訊科學系
112753208
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0112753208
資料類型 thesis
dc.contributor.advisor 廖文宏zh_TW
dc.contributor.advisor Liao, Wen-Hungen_US
dc.contributor.author (Authors) 羅鈺涵zh_TW
dc.contributor.author (Authors) Lo, Yu-Hanen_US
dc.creator (作者) 羅鈺涵zh_TW
dc.creator (作者) Lo, Yu-Hanen_US
dc.date (日期) 2025en_US
dc.date.accessioned 3-Mar-2025 14:04:12 (UTC+8)-
dc.date.available 3-Mar-2025 14:04:12 (UTC+8)-
dc.date.issued (上傳時間) 3-Mar-2025 14:04:12 (UTC+8)-
dc.identifier (Other Identifiers) G0112753208en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/155974-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學系zh_TW
dc.description (描述) 112753208zh_TW
dc.description.abstract (摘要) 隨著機器學習的蓬勃發展，特別是生成式人工智慧模型的興起，AI 技術已從專業領域延伸至大眾生活。然而，機器學習模型仰賴大量訓練資料的特性，引發智慧財產權與個人隱私的重要議題。歐盟的《一般資料保護規則》(GDPR) 和美國《加州消費者隱私保護法》(CCPA) 明確規範資料遺忘權，但在深度學習時代，單純刪除原始資料已不足以保護隱私，因為這些資料在訓練過程中已深植於模型參數中。 為了有效解決這個問題，本論文提出一套結合彈性壓縮技術的機器遺忘機制。這種方法不僅能快速且有效地從模型中移除指定資訊，更能透過模型壓縮與稀疏化技術，大幅降低計算成本並提升效率。我們的方法透過零值填充和重新稀疏化訓練的方式，實現靈活的漸進式遺忘，使模型能夠在多輪遺忘操作後仍維持良好性能。 與傳統方法相比，本研究提出的彈性壓縮遺忘機制在運算效率與隱私保護之間取得更好的平衡。透過對模型參數的動態稀疏化和彈性壓縮，不僅能有效移除遺忘資料集的相關資訊，還能防止潛在的成員推斷攻擊和模型反轉攻擊。這種方法特別適合計算資源受限且需要高度隱私保護的應用場景，為實際部署提供一個兼具效率與安全性的解決方案。zh_TW
dc.description.abstract (摘要) With the rapid advancement of machine learning, particularly the emergence of generative AI models, AI technology has expanded from professional domains to everyday life. However, machine learning models' reliance on extensive training data raises significant concerns regarding intellectual property rights and personal privacy. While the European Union's General Data Protection Regulation (GDPR) and California's Consumer Privacy Protection Act (CCPA) explicitly regulate the right to be forgotten, merely deleting original data is insufficient for privacy protection in the deep learning era, as this data becomes deeply embedded within model parameters during training. To address this challenge, this thesis proposes a novel machine unlearning mechanism integrated with flexible compression techniques. This approach not only enables swift and effective removal of specified information from models but also significantly reduces computational costs and improves efficiency through model compression and sparsification techniques. Our method achieves flexible progressive unlearning through zero-value filling and re-sparsification training, enabling models to maintain high performance even after multiple rounds of unlearning operations. Compared to traditional approaches, our proposed flexible compression unlearning mechanism achieves a better balance between computational efficiency and privacy protection. Through dynamic sparsification and flexible compression of model parameters, we effectively remove information related to the forgotten dataset while preventing potential membership inference attacks and model inversion attacks. This method is particularly suitable for applications with limited computational resources requiring high privacy protection, providing a practical deployment solution that combines both efficiency and security.en_US
dc.description.tableofcontents 致謝詞 i 摘要 ii Abstract iii 目錄 v 表次 viii 圖次 x 第一章 緒論 1 1.1 研究動機 1 1.2 機器遺忘 2 1.3 研究貢獻 5 1.4 論文架構 5 第二章 技術背景與相關研究 6 2.1 機器遺忘方法 6 2.1.1 精確遺忘 6 2.1.2 近似遺忘 6 2.2 模型壓縮 9 2.2.1 Deep Compression 10 2.2.2 Pruning Filters for Efficient ConvNets 13 2.2.3 基於卷積核冗餘的神經網路壓縮機制 13 2.2.4 DepGraph: Towards Any Structural Pruning 13 第三章 研究方法 18 3.1 遺忘和模型壓縮實驗流程 18 3.1.1 遺忘目標與任務選擇 19 3.1.2 CIFAR 資料集 20 3.1.3 ResNet 模型 21 3.1.4 實驗模型與參數設定 23 3.1.5 剪枝方法介紹 24 3.2 漸進式遺忘 24 3.2.1 解壓縮模型和漸進式遺忘技術 25 3.2.2 解壓縮模型與漸進式遺忘步驟 25 3.2.3 漸進式遺忘的優點 26 3.3 機器遺忘評估指標 27 第四章 實驗結果與分析 29 4.1 遺忘 10class₁ 和 10class₂ 類之實驗結果 30 4.1.1 稀疏模型對於遺忘資料集與保留資料集的準確率 30 4.1.2 剪枝比例和加速比對於準確率的影響 32 4.1.3 模型對於遺忘資料集之各項指標 38 4.2 遺忘 20class₁ 和 20class₂ 類之實驗結果 44 4.2.1 稀疏模型對於遺忘資料集與保留資料集的準確率 44 4.2.2 剪枝比例和加速比對於準確率的影響 45 4.2.3 模型對於遺忘資料集之各項指標 51 4.3 解壓縮模型之準確率 59 4.4 遺忘 10 + 10class₁ 和 10 + 10class₂ 類之實驗結果 61 4.4.1 稀疏模型對於遺忘資料集與保留資料集的準確率 61 4.4.2 剪枝比例和加速比對於準確率的影響 63 4.4.3 模型對於遺忘資料集之各項指標 64 4.5 遺忘 10 + 10zero₁ + 10 和 10 + 10zero₂ + 10 類之實驗結果 70 4.5.1 稀疏模型對於遺忘資料集與保留資料集的準確率 70 4.5.2 剪枝比例和加速比對於準確率的影響 73 4.5.3 模型對於遺忘資料集之各項指標 74 4.6 剪枝模型在遺忘不同類別之評估指標結果 76 第五章 結論與未來工作 79 附錄 A 30 及 50 類別稀疏模型的準確率變化 81 參考文獻 85zh_TW
dc.format.extent 37369420 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0112753208en_US
dc.subject (關鍵詞) 深度學習zh_TW
dc.subject (關鍵詞) 神經網路zh_TW
dc.subject (關鍵詞) 模型壓縮zh_TW
dc.subject (關鍵詞) 機器遺忘zh_TW
dc.subject (關鍵詞) 可解釋性人工智慧zh_TW
dc.subject (關鍵詞) Deep Learningen_US
dc.subject (關鍵詞) Neural Networken_US
dc.subject (關鍵詞) Model Compressionen_US
dc.subject (關鍵詞) Machine Unlearningen_US
dc.subject (關鍵詞) XAIen_US
dc.title (題名) 結合彈性壓縮技術之機器遺忘機制zh_TW
dc.title (題名) Machine Unlearning Mechanisms with Flexible Compression Schemesen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) 1. S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” Advances in Neural Information Processing Systems, vol. 28, 2015. 2. S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149, 2015. 3. G. Fang, X. Ma, M. Song, M. B. Mi, and X. Wang, “Depgraph: Towards any structural pruning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16091–16101, 2023. 4. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016. 5. P. P. Ray, “Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,” Internet of Things and Cyber-Physical Systems, vol. 3, pp. 121–154, 2023. 6. P. Regulation, “Regulation (EU) 2016/679.” Official Journal of the European Union, 2016. General Data Protection Regulation (GDPR). 7. D. U. CCPA, “California Consumer Privacy Act (CCPA) Website Policy,” 2020. 8. Government of Japan, “Amended act on the protection of personal information,” 2016. Accessed: 2024-06-01. 9. National People’s Congress of the People’s Republic of China, “Personal information protection law of the people’s republic of China,” 2021. Accessed: 2024-06-01. 10. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159, IEEE, 2021. 11. R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18, IEEE, 2017. 12. “NeurIPS 2023 Machine Unlearning Challenge.” https://unlearning-challenge.github.io/, 2023. Accessed: 2024/4/5. 13. E. Ullah, T. Mai, A. Rao, R. A. Rossi, and R. Arora, “Machine unlearning via algorithmic stability,” in Conference on Learning Theory, pp. 4126–4142, PMLR, 2021. 14. A. Ginart, M. Guan, G. Valiant, and J. Zou, “Making AI forget you: Data deletion in machine learning,” Advances in Neural Information Processing Systems (NeurIPS), 2019. 15. L. Graves, V. Nagisetty, and V. Ganesh, “Amnesiac machine learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 13, pp. 11516–11524, 2021. 16. Y. Cao, J. Yang, and C. Yang, “Towards making systems forget with machine unlearning,” in IEEE Symposium on Security and Privacy (SP), 2015. 17. S. Schelter, D. Kossmann, M. Zeller, and A. Halevy, “The case for data versioning in machine learning,” in Proceedings of the 2021 International Conference on Management of Data (SIGMOD), 2021. 18. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Proceedings of the 36th International Conference on Machine Learning (ICML), 2019. 19. J. Wang, S. Guo, X. Xie, and H. Qi, “Federated unlearning via class-discriminative pruning,” in Proceedings of the ACM Web Conference 2022, pp. 622–632, 2022. 20. A. Golatkar, A. Achille, and S. Soatto, “Eternal sunshine of the spotless net: Selective forgetting in deep networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304–9312, 2020. 21. A. Thudi, H. Jia, M. Goldblum, T. Goldstein, and A. Shrivastava, “Model agnostic unlearning,” arXiv preprint arXiv:2108.11577, 2021. 22. A. K. Tarun, V. S. Chundawat, M. Mandal, and M. Kankanhalli, “Fast yet effective machine unlearning,” IEEE Transactions on Neural Networks and Learning Systems, 2023. 23. H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” arXiv preprint arXiv:1608.08710, 2016. 24. N.-W. Chen, “A compression mechanism of neural networks based on convolution kernel redundancy,” Journal of Machine Learning Research, vol. 22, no. 1, pp. 123–135, 2021. 25. A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, 05 2012. 26. V. S. Chundawat, A. K. Tarun, M. Mandal, and M. Kankanhalli, “Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 6, pp. 7210–7217, 2023. 27. J. Lin, “Divergence measures based on the Shannon entropy,” IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 145–151, 1991. 28. P. Xia, L. Zhang, and F. Li, “Learning similarity with cosine similarity ensemble,” Information Sciences, vol. 307, pp. 39–52, 2015.zh_TW