1. NCCU Academic Hub
  2. Publications
  3. College of Informatics
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 結合高密度連結子圖檢索以提升大型語言模型的知識圖譜提示能力
Enhancing Knowledge Graph Prompting for Large Language Models Based on Densely Connected Subgraph Retrieval
作者 陳品伃
Chen, Pin-Yu
貢獻者 沈錳坤
Shan, Man-Kwan
陳品伃
Chen, Pin-Yu
關鍵詞 知識圖譜
子圖檢索
醫療問答
提示工程
大型語言模型
Knowledge Graph
Subgraph Retrieval
Medical Question Answering
Prompt Engineering
Large Language Model
日期 2025
上傳時間 1-Sep-2025 16:58:06 (UTC+8)
摘要 隨著大型語言模型的應用日益廣泛，儘管展現出優異的語言理解與生成能力，但在需要複雜推理的專業領域，推理品質及可解釋性略顯不足，例如醫療和法律領域。為提升模型在專業領域的推理品質及可解釋性，本研究採用知識圖譜作為外部知識來源，以輔助大型語言模型進行推理。為此，我們提出以Densely Connected Subgraph Retrieval為核心的知識圖譜檢索架構，從知識圖譜中檢索出結構緊密且具高度關聯性的子圖，再結合Shortest Path Search與Importance-Based Path Search等方法來選取推理路徑。最後，將這些路徑轉換為自然語言，並以提示工程引導大型語言模型進行推理。本研究以醫療問答任務為專業應用場景，採用患者主訴與醫師回覆的問診對話作為實驗基礎，並採用GPT-4o Ranking與BERT Score評估大型語言模型在醫療問答任務上的推理品質與可解釋性。結果顯示，本研究提升了大型語言模型於醫療問答任務中的推理品質與可解釋性，展現專業應用的潛力。
Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, yet their effectiveness in professional domains such as healthcare and law remains limited by challenges in complex reasoning and explainability. To address these limitations, we propose a knowledge graph–assisted framework for reasoning, which leverages Densely Connected Subgraph Retrieval to extract structurally cohesive and semantically relevant subgraphs. Within these subgraphs, reasoning paths are identified through a combination of shortest-path search and node-importance-based methods, and subsequently transformed into natural language to guide LLMs via prompt engineering. We evaluate this approach on a medical question-answering task using doctor–patient dialogues, assessing reasoning quality and explainability with GPT-4o Ranking and BERTScore. Experimental results demonstrate that our method significantly improves both the reasoning accuracy and interpretability of LLM outputs, underscoring its potential for deployment in high-stakes professional applications.
參考文獻 [1] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. Bang, A. Madotto and P. Fung, “Survey of Hallucination in Natural Language Generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 1-38, 2023. [2] S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang and X. Wu, “Unifying Large Language Models and Knowledge Graphs: A Roadmap,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no.7, pp. 3580-3599, July 2024. [3] L. Luo, Y. Li, G. Haffari, and S. Pan, “Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning,” International Conference on Learning Representations, Vienna, Austria, 2024. [4] Y. Wen, Z. Wang and J. Sun, “MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models,” 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand, 2024. [5] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody and S. Truitt, “From Local to Global: A Graph RAG Approach to Query-Focused Summarization,” arXiv preprint arXiv:2404.16130, 2024. [6] M. Sozio and A. Gionis, “The Community-Search Problem and How to Plan a Successful Cocktail Party,” 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10), pp. 939–948, Washington, DC, USA, 2010. [7] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le and D. Zhou, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, USA, 2022. [8] T. Kojima, S. Gu, M. Reid, Y. Matsuo and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, USA, 2022. [9] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery and D. Zhou, “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” arXiv preprint arXiv:2203.11171, 2023. [10] X. Xu, C. Tao, T. Shen, C. Xu, H. Xu, G. Long and J. Lou, “Re-Reading Improves Reasoning in Large Language Models,” 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15549-15575, Miami, FL, USA, 2024. [11] S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao and K. Narasimhan, “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” 37th Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA, 2023. [12] M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk and T. Hoefler, “Graph of Thoughts: Solving Elaborate Problems with Large Language Models,” 38th AAAI Conference on Artificial Intelligence, vol. 38, no. 16, Vancouver, Canada, 2024. [13] J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, L. Ni, H. Shum and J. Guo, “Think-On-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph,” 12th International Conference on Learning Representations, Vienna, Austria, 2024. [14] B. Jiang, Y. Wang, Y. Luo, D. He, P. Cheng and L. Gao, “Reasoning on Efficient Knowledge Paths: Knowledge Graph Guides Large Language Model for Domain Question Answering,” 2024 IEEE International Conference on Knowledge Graph (ICKG), Abu Dhabi, United Arab Emirates, 2024. [15] M. Jia, J. Duan, Y. Song and J. Wang, “medIKAL: Integrating Knowledge Graphs as Assistants of LLMs for Enhanced Clinical Diagnosis on EMRs,” arXiv preprint arXiv:2406.14326, 2024. [16] L. Wei, G. Xiao and M. Balazinska, “RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph,” arXiv preprint arXiv:2409.14556, 2024. [17] M. Dehghan, M. Alomrani, S. Bagga, D. Alfonso-Hermelo, K. Bibi, A. Ghaddar, Y. Zhang, X. Li, J. Hao, Q. Liu, J. Lin, B. Chen, P. Parthasarathi, M. Biparva and M. Rezagholizadeh, “EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems,” arXiv preprint arXiv:2406.10393, 2024. [18] W. Xie, X. Liang, Y. Liu, K. Ni, H. Cheng and Z. Hu, “WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs,” arXiv preprint arXiv:2408.07611, 2024. [19] V. Sanh, L. Debut, J. Chaumond and T. Wolf, “DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter,” arXiv preprint arXiv:1910.01108, 2019. [20] Y. Li, Z. Li, K. Zhang, R. Dan, S. Jiang and Y. Zhang, “ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge,” Cureus, vol. 55, no. 6, pp. e40895, 2023. [21] S. Brin and L. Page, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” Computer Networks and ISDN Systems, vol. 30, no. 1-7, pp. 107–117, 1998.
描述 碩士
國立政治大學
資訊科學系
112753204
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0112753204
資料類型 thesis
dc.contributor.advisor 沈錳坤zh_TW
dc.contributor.advisor Shan, Man-Kwanen_US
dc.contributor.author (Authors) 陳品伃zh_TW
dc.contributor.author (Authors) Chen, Pin-Yuen_US
dc.creator (作者) 陳品伃zh_TW
dc.creator (作者) Chen, Pin-Yuen_US
dc.date (日期) 2025en_US
dc.date.accessioned 1-Sep-2025 16:58:06 (UTC+8)-
dc.date.available 1-Sep-2025 16:58:06 (UTC+8)-
dc.date.issued (上傳時間) 1-Sep-2025 16:58:06 (UTC+8)-
dc.identifier (Other Identifiers) G0112753204en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/159417-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學系zh_TW
dc.description (描述) 112753204zh_TW
dc.description.abstract (摘要) 隨著大型語言模型的應用日益廣泛，儘管展現出優異的語言理解與生成能力，但在需要複雜推理的專業領域，推理品質及可解釋性略顯不足，例如醫療和法律領域。為提升模型在專業領域的推理品質及可解釋性，本研究採用知識圖譜作為外部知識來源，以輔助大型語言模型進行推理。為此，我們提出以Densely Connected Subgraph Retrieval為核心的知識圖譜檢索架構，從知識圖譜中檢索出結構緊密且具高度關聯性的子圖，再結合Shortest Path Search與Importance-Based Path Search等方法來選取推理路徑。最後，將這些路徑轉換為自然語言，並以提示工程引導大型語言模型進行推理。本研究以醫療問答任務為專業應用場景，採用患者主訴與醫師回覆的問診對話作為實驗基礎，並採用GPT-4o Ranking與BERT Score評估大型語言模型在醫療問答任務上的推理品質與可解釋性。結果顯示，本研究提升了大型語言模型於醫療問答任務中的推理品質與可解釋性，展現專業應用的潛力。zh_TW
dc.description.abstract (摘要) Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, yet their effectiveness in professional domains such as healthcare and law remains limited by challenges in complex reasoning and explainability. To address these limitations, we propose a knowledge graph–assisted framework for reasoning, which leverages Densely Connected Subgraph Retrieval to extract structurally cohesive and semantically relevant subgraphs. Within these subgraphs, reasoning paths are identified through a combination of shortest-path search and node-importance-based methods, and subsequently transformed into natural language to guide LLMs via prompt engineering. We evaluate this approach on a medical question-answering task using doctor–patient dialogues, assessing reasoning quality and explainability with GPT-4o Ranking and BERTScore. Experimental results demonstrate that our method significantly improves both the reasoning accuracy and interpretability of LLM outputs, underscoring its potential for deployment in high-stakes professional applications.en_US
dc.description.tableofcontents 第一章 緒論 8 1.1 研究背景 9 1.2 研究動機 10 1.3 研究目的 10 第二章 相關研究 11 2.1 提示工程增強大型語言模型效能 11 2.2 知識圖譜增強的大型語言模型推理 11 第三章 研究方法 14 3.1 研究架構 14 3.2 Knowledge Graph Construction 15 3.3 Entities Recognition 15 3.3.1 Entities Extraction 15 3.3.2 Entities Expansion 16 3.4 Densely Connected Subgraph Retrieval 17 3.4.1 GreedyDist Algorithm 17 3.4.2 Steiner Tree Algorithm 19 3.5 Path Selection 20 3.5.1 Shortest Path Search 21 3.5.2 Importance-Based Path Search 21 3.5.3 Multi-Hop Paths 轉換為自然語言 22 3.6 LangChain Reasoning 22 第四章 實驗 25 4.1 資料集 25 4.2 實驗設計與評估方法 26 4.2.1 現有大型語言模型規格比較 26 4.2.2 實驗設計 27 4.2.3 評估指標 28 4.3 實驗結果 30 4.3.1 整體表現總覽分析 31 4.3.2 依模型回答類型進行分析：診斷、用藥推薦與檢測建議 32 4.3.3 基於患者主訴焦點之模型表現分析：疾病、症狀、藥物與檢測項目 36 4.4 消融實驗 52 4.4.1 Key Entities的順序對模型回答的影響 52 4.4.2 Key Entities之間的距離限制對模型回答的影響 54 4.4.3 路徑數量選擇對模型回答的影響 54 4.4.4 Densely Connected Subgraph Size的大小對模型回答的影響 56 4.4.5 PageRank使用範圍對模型回答的影響 57 4.5 小語言模型的回答表現 58 4.5.1 整體表現總覽分析 59 4.5.2 依模型回答類型進行分析：診斷、用藥推薦與檢測建議 60 4.5.3 基於患者主訴焦點之模型表現分析：疾病、症狀、藥物與檢測項目 62 第五章 結論 64 參考文獻 65zh_TW
dc.format.extent 4960427 bytes-
dc.format.mimetype application/pdf-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0112753204en_US
dc.subject (關鍵詞) 知識圖譜zh_TW
dc.subject (關鍵詞) 子圖檢索zh_TW
dc.subject (關鍵詞) 醫療問答zh_TW
dc.subject (關鍵詞) 提示工程zh_TW
dc.subject (關鍵詞) 大型語言模型zh_TW
dc.subject (關鍵詞) Knowledge Graphen_US
dc.subject (關鍵詞) Subgraph Retrievalen_US
dc.subject (關鍵詞) Medical Question Answeringen_US
dc.subject (關鍵詞) Prompt Engineeringen_US
dc.subject (關鍵詞) Large Language Modelen_US
dc.title (題名) 結合高密度連結子圖檢索以提升大型語言模型的知識圖譜提示能力zh_TW
dc.title (題名) Enhancing Knowledge Graph Prompting for Large Language Models Based on Densely Connected Subgraph Retrievalen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) [1] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. Bang, A. Madotto and P. Fung, “Survey of Hallucination in Natural Language Generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 1-38, 2023. [2] S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang and X. Wu, “Unifying Large Language Models and Knowledge Graphs: A Roadmap,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no.7, pp. 3580-3599, July 2024. [3] L. Luo, Y. Li, G. Haffari, and S. Pan, “Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning,” International Conference on Learning Representations, Vienna, Austria, 2024. [4] Y. Wen, Z. Wang and J. Sun, “MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models,” 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand, 2024. [5] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody and S. Truitt, “From Local to Global: A Graph RAG Approach to Query-Focused Summarization,” arXiv preprint arXiv:2404.16130, 2024. [6] M. Sozio and A. Gionis, “The Community-Search Problem and How to Plan a Successful Cocktail Party,” 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10), pp. 939–948, Washington, DC, USA, 2010. [7] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le and D. Zhou, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, USA, 2022. [8] T. Kojima, S. Gu, M. Reid, Y. Matsuo and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, USA, 2022. [9] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery and D. Zhou, “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” arXiv preprint arXiv:2203.11171, 2023. [10] X. Xu, C. Tao, T. Shen, C. Xu, H. Xu, G. Long and J. Lou, “Re-Reading Improves Reasoning in Large Language Models,” 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15549-15575, Miami, FL, USA, 2024. [11] S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao and K. Narasimhan, “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” 37th Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA, 2023. [12] M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk and T. Hoefler, “Graph of Thoughts: Solving Elaborate Problems with Large Language Models,” 38th AAAI Conference on Artificial Intelligence, vol. 38, no. 16, Vancouver, Canada, 2024. [13] J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, L. Ni, H. Shum and J. Guo, “Think-On-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph,” 12th International Conference on Learning Representations, Vienna, Austria, 2024. [14] B. Jiang, Y. Wang, Y. Luo, D. He, P. Cheng and L. Gao, “Reasoning on Efficient Knowledge Paths: Knowledge Graph Guides Large Language Model for Domain Question Answering,” 2024 IEEE International Conference on Knowledge Graph (ICKG), Abu Dhabi, United Arab Emirates, 2024. [15] M. Jia, J. Duan, Y. Song and J. Wang, “medIKAL: Integrating Knowledge Graphs as Assistants of LLMs for Enhanced Clinical Diagnosis on EMRs,” arXiv preprint arXiv:2406.14326, 2024. [16] L. Wei, G. Xiao and M. Balazinska, “RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph,” arXiv preprint arXiv:2409.14556, 2024. [17] M. Dehghan, M. Alomrani, S. Bagga, D. Alfonso-Hermelo, K. Bibi, A. Ghaddar, Y. Zhang, X. Li, J. Hao, Q. Liu, J. Lin, B. Chen, P. Parthasarathi, M. Biparva and M. Rezagholizadeh, “EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems,” arXiv preprint arXiv:2406.10393, 2024. [18] W. Xie, X. Liang, Y. Liu, K. Ni, H. Cheng and Z. Hu, “WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs,” arXiv preprint arXiv:2408.07611, 2024. [19] V. Sanh, L. Debut, J. Chaumond and T. Wolf, “DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter,” arXiv preprint arXiv:1910.01108, 2019. [20] Y. Li, Z. Li, K. Zhang, R. Dan, S. Jiang and Y. Zhang, “ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge,” Cureus, vol. 55, no. 6, pp. e40895, 2023. [21] S. Brin and L. Page, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” Computer Networks and ISDN Systems, vol. 30, no. 1-7, pp. 107–117, 1998.zh_TW