DSpace Collection:
https://ah.lib.nccu.edu.tw/handle/140.119/2258
2024-03-29T12:04:10Z基於自監督學習之生成語言模型序列文本知識更新
https://ah.lib.nccu.edu.tw/handle/140.119/147747
題名: 基於自監督學習之生成語言模型序列文本知識更新; Sequential Text-based Knowledge Update with Self-Supervised Learning for Generative Language Models
Authors: 宋浩茹; Sung, Hao-Ru
摘要: 本研究提出新的自然語言處理(NLP)任務,以解決多輪、序列式的文本知識更新問題。該研究引入了一種混合學習架構和新穎的自監督訓練策略,旨在使生成語言模型能夠像人類一樣有效地鞏固和更新知識。這種方式對於改善語言模型的學習和理解能力具有重大意義。為了驗證這種策略的有效性,我們還創建了一個新的數據集以進行評估。從實驗結果來看,我們的方法在效能上超越了現有的模型和GPT-3.5-Turbo。本研究所提出的任務和模型架構能夠提升知識組織的自動化程度,使得基於文本知識的大型語言模型(LLM),成為協助人類執行各種任務的重要資源。; This work proposes a new natural language processing (NLP) task to tackle the issue of multi-round, sequential text-based knowledge update. The study introduces a hybrid learning architecture and a novel self-supervised training strategy to enable generative language models to consolidate knowledge in the same way as humans. A dataset was also created for evaluation and results showed the effectiveness of our methodology. Experimental results confirm the superiority of the proposed approach over existing models and GPT-3.5-Turbo. The proposed task and model framework have the potential to significantly improve the automation of knowledge organization, making text-based knowledge an increasingly crucial resource for powerful large language models (LLM) to perform various tasks for humans.
描述: 碩士; 國立政治大學; 資訊科學系; 1107531242023-10-03T02:49:40Z基於Associated Learning架構優化MEC環境訓練模型之效能
https://ah.lib.nccu.edu.tw/handle/140.119/147745
題名: 基於Associated Learning架構優化MEC環境訓練模型之效能; Optimize the Performance of the Training Model in the MEC Environment based on the Associated Learning Architecture
Authors: 張皓博; Chang, Hao-Po
摘要: 近年來,隨著行動通訊網路的進步,邊緣設備的數量及運算能力提升,再加上人工智慧的蓬勃發展,以及資料隱私意識的抬頭,催生出運用邊緣設備訓練模型的分散式機器學習,其中包括聯邦學習以及拆分學習,然而這兩種方法在架構上存在明顯的優缺點。本研究旨在提出一個訓練架構,與聯邦學習相比,不僅能達到相似的模型準確度,同時在訓練過程中也能減少邊緣設備的運算量以及降低邊緣伺服器的流量,並且改善使用模型時的延遲,進一步提升使用者體驗。為了實現這一目標,在系統架構中採用兩層式設計,提出一個啟發式的分群演算法,群組內各邊緣設備只訓練部分模型,邊緣設備間使用設備到設備通訊技術,利用Associated Learning架構來解決拆分模型後反向傳播的流量問題,此外群組內僅透過主設備與邊緣伺服器通訊,進一步降低了邊緣伺服器的流量負擔。為了驗證本研究是否有達成預期指標,模擬實驗中採用PyTorch及ns3進行模擬,從實驗結果可以驗證本研究相較於聯邦學習在實驗中有更佳的準確度,且透過Associated Learning特色能降低使用時的延遲,提升使用者體驗,針對特定情況下也能夠降低邊緣設備運算量及邊緣伺服器流量,最後提出本研究可優化之部分,並歸納出未來學者可持續往安全性、更通用的架構、更合乎現實情況的模擬等方向研究。; In recent years, with the advancement of cellular networks, the number and computing power of edge devices have increased. The vigorous development of artificial intelligence and the rise of data privacy awareness have spawned distributed machine learning that uses edge device training models, including federated learning and split learning. However, both have obvious advantages and disadvantages in terms of architecture. The purpose of this study is to propose a training framework. Compared with federated learning, it can not only achieve similar model accuracy but also reduce the computation of edge devices and the traffic of edge server during the training process, improve the latency when using the model, and further enhances the user experience. Therefore, a heuristic grouping algorithm is proposed, and a two-layer design is adopted in the system architecture. Each edge device in the group only trains parts of the model and communications through Device-to-Device. The Associated Learning architecture is used to decouple the dependency relationship of backpropagation when updating the model parameters, and it is expected to reduce the amount of computational required to train the model. After grouping, the multi-objective function is used to select the master edge device, and the group only communicates with the edge server through the master edge device, which is expected to reduce the traffic of the edge server. To verify whether this study has achieved the expected indicators, PyTorch and ns3 are used to simulate the experiment. According to experimental results, it can be verified that this study has better accuracy than federated learning in the experiment. Through the Associated Learning feature, it can reduce the latency during inference, improve the user experience, and reduce the computing load of edge devices and the traffic of edge servers under certain circumstances. Finally, the part of this research that can be optimized is proposed, and the sustainable research directions of future scholars are summarized, including security, more general architecture, and more realistic simulation.
描述: 碩士; 國立政治大學; 資訊科學系; 1107531132023-10-03T02:49:01Z開發與評估教育聊天機器人:以與課程相關的內容即時支援非資訊領域大學生解決程式設計問題
https://ah.lib.nccu.edu.tw/handle/140.119/147038
題名: 開發與評估教育聊天機器人:以與課程相關的內容即時支援非資訊領域大學生解決程式設計問題; Development and Evaluation of an Educational Chatbot: Providing Real-Time and Contextual Support for Non-IT University Students Facing Programming Problems
Authors: 林昱辰; Lin, Yu-Chen
摘要: 隨著科技發展,國高中課綱將資訊科技納入必修課程中,希望培養學生的邏輯能力和運算思維。然而,由於課綱修改前後的學生存在學識斷層,進入大學後程式設計程度參差不齊,使得教師在設計個人化教學內容和學習資源方面面臨挑戰;學生們常因害羞或擔心同儕評價而不敢向老師或助教提問。教育聊天機器人的開發可以為學生提供個人化的學習支援,減輕教師和助教的工作負擔,提供學生便利的學習資源的同時給予了較低壓力的環境,讓他們更自在地提問和尋找解答。本研究開發的聊天機器人適用的教學場域為講授基礎Python程式設計觀念的資訊通識課程。研究中使用詞嵌入技術透過餘弦相似度選出與使用者的訊息相近的課程投影片內容來輔助聊天機器人,讓使用者與聊天機器人的對話能夠聚焦於課程討論。; With the development of technology, information technology has been included in the curriculum of junior and senior high schools, aiming to cultivate students` logical reasoning and computational thinking skills. However, due to the disparity in students` knowledge before and after the curriculum revision, there is a significant variation in their programming abilities when they enter university. This poses a challenge for teachers in designing personalized teaching content and learning resources. Additionally, students often hesitate to ask questions of their teachers or teaching assistants due to shyness or concerns about peer evaluation.\nThe development of an educational chatbot can provide personalized learning support to students, alleviate the workload of teachers and teaching assistants, and offer students convenient learning resources in a low-pressure environment. This enables them to feel more comfortable asking questions and seeking answers. The chatbot developed in this research is designed for the educational field of teaching fundamental Python programming concepts in an introductory information technology course. In the research, word embedding techniques are used, and cosine similarity is employed to select course slide content that closely matches the user`s input. This assists the chatbot in focusing on course discussions during interactions with users.
描述: 碩士; 國立政治大學; 資訊科學系; 1107531632023-09-01T07:25:48Z基於命名資料網路在延遲容忍網路之任播機制設計
https://ah.lib.nccu.edu.tw/handle/140.119/147037
題名: 基於命名資料網路在延遲容忍網路之任播機制設計; Design of Anycast Mechanism for NDN in DTN
Authors: 蘇俊憲; Su, Chun-Hsien
摘要: 從人文創新的角度,建立分散式的資料蒐集和邊緣運算平台可以更好地滿足使用者的需求,提供個性化和有效的智慧應用服務。本研究專注於設計一個適用於無線隨意網路的任播路由機制。此機制旨在滿足資料生產者或使用者能夠以更低的請求成本與更高的成功率,透過智慧移動裝置將產出的內容或推薦請求,透過無線網路將資料上傳至距離最近最好的邊緣伺服器或進行所需的推薦運算,並將結果回傳給使用者。\n本研究採取命名資料網路(Named Data Networking,NDN),使用以內容為中心的路由演算法,考慮到節點稀疏可能會造成連線斷線,導入了延遲容忍網路(Delay-tolerant networking, DTN)的概念去儲存-攜帶-轉發封包,並針對 NDN 實作 DTN 的困難處提出解決方法,此外為了降低原有泛洪極度耗費成本的做法,本研究提出了一個利用可達性(Possibility)的路由演算法來幫助減少 Overhead 與達成任播的目的。\n最後實驗使用商圈中的地下街作為研究情境,將方法配合情境使用在任播機制上。實驗數據表明,本研究提出的方法與其他方法相比,不論在邊緣伺服器的數量或是人群的數量增減下,本研究提出的方法都能夠有效控制開銷,並且效率能夠幾乎比肩 Flooding 與 Epidemic Routing。; From a perspective of humanistic innovation, establishing a decentralized data collection and edge computing platform can better meet to user`s needs and provide personalized smart application services.As for data collection, this research focuses on designing an anycast routing mechanism applicable to wireless ad hoc networks. The mechanism aims to satisfy the requirement of data producers or users to achieve lower request costs and higher success rates. It enables the upload of content or recommendation requests generated by smart mobile devices via wireless networks to the nearest and optimal edge server that facilitates the required recommendation calculations. And sends the results back to the users.\nWe adopt Named Data Networking (NDN) with content-centric routing algorithms. Considering the disconnections of nodes due to sparsity, we incorporate the concept of Delay-Tolerant Networking (DTN) for storing-carrying-forwarding packets. We propose solutions to address the challenges of implementing DTN within NDN. Additionally, to reduce the excessive overhead associated with flooding, we introduce a routing algorithm based on "possibility" to help minimize overhead and achieve the objectives of anycast.\nIn our experiments, we utilize an underground mall in a commercial district as the research scenario and apply our methods to the anycast mechanism within this context. The results demonstrate that the proposed approach effectively controls costs and achieves efficiency comparable to Flooding and Epidemic Routing, regardless of variations in the number of edge servers or the size of the population.
描述: 碩士; 國立政治大學; 資訊科學系; 1107531582023-09-01T07:25:35Z