Publications-Theses

題名 電腦輔助克漏詞多選題出題系統之研究
A Study on Computer Aided Generation of Multiple-Choice Cloze Items
作者 王俊弘
Wang , Chun-Hung
貢獻者 劉昭麟
Liu , Chao-Lin
王俊弘
Wang , Chun-Hung
關鍵詞 電腦輔助語言學習
自動產生試題
試題編寫工具
詞義辨析
自然語言處理
Computer-Assisted Language Learning
Automatic Item Generation
Authoring Tools
Word Sense Disambiguation
Natural Language Processing
日期 2003
上傳時間 17-Sep-2009 13:54:53 (UTC+8)
摘要 多選題測驗試題已證明能有效地評估學生的學習成效,然而,以人為方式建立題庫是一件耗時費力的工作。藉由電腦高速運算的能力,電腦輔助產生試題系統能有效率地建置大規模的題庫,同時減少人為的干預而得以保持試題的隱密性。受惠於網路上充裕的文字資源,本研究發展一套克漏詞試題出題系統,利用既有的語料自動產生涵蓋各種不同主題的克漏詞試題。藉由分析歷屆大學入學考試的資料,系統可產生類似難度的模擬試題,並且得到出題人員在遴選測驗標的方面的規律性。在產生試題的過程中導入詞義辨析的演算法,利用詞典與selectional preference模型的輔助,分析句子中特定詞彙的語義,以擷取包含測驗編撰者所要測驗的詞義的句子,並以collocation為基礎的方法篩選誘答選項。實驗結果顯示系統可在每產生1.6道試題中,得到1道可用的試題。我們嘗試產生不同類型的試題,並將這套系統融入網路線上英文測驗的環境中,依學生的作答情形分析試題的鑑別度。
Multiple-choice tests have proved to be an efficient tool for measuring students’ achievement. Manually constructing tests items, however, is a time- consuming and labor-intensive task. Harnessing the computing power of computers, computer-assisted item generation offers the possibility of creating large amount of items, thereby alleviating the problem of keeping the items secure. With the abundant text resource on the Web, this study develops a system capable of generating cloze items that cover a wide range of topics based on existing corpra. By analyzing training data from the College Entrance Examinations in Taiwan, we identify special regularities of the test items, and our system can generate items of similar style based on results of the analysis. We propose a word sense disambiguation-based method for locating sentences in which designated words carry specific senses, and apply collocation-based methods for selecting distractors. Experimental results indicate that our system was able to produce a usable item for every 1.6 items it returned. We try to create different types of items and integrate the reported item generator in a Web-based system for learning English. The outcome of on-line examinations is analyzed in order to estimate the item discrimination of the test items generated by our system.
參考文獻 [1] 郭生玉,心理與教育測驗,台北:精華書局,1989。
[2] 鄭恆雄,陳凌霞,宋麗梅等,八十七年度英文命題實驗計畫研究報告,台北:大學入學考試中心,1999。
[3] Berger, A. L., Della Pietra, S. A. & Della Pietra, V. J. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22 (1), 39–71.
[4] Coniam, D. (1997). A preliminary inquiry into using corpus word frequency data in the automatic generation of English language cloze tests. Computer Assisted Language Instruction Consortium, 16 (2–4), 15–33.
[5] Deane, P. & Sheehan, K. (2003). Automatic item generation via frame semantics. Education Testing Service: http://www.ets.org/research/dload/ncme03-deane.pdf .
[6] Dennis, I., Handley, S., Bradon, P., Evans, J. & Nestead, S. (2002). Approaches to modeling item-generative tests. In: Item Generation for Test Development [12], 53–72.
[7] Dorr, B. J., Levow, G.-A. & Lin, D. (2000). Large-scale construction of a Chinese- English semantic hierarchy. Technical report, University of Maryland, College Park, MD.
[8] Fillmore, C. J. & Atkins, B.T. (1992). Towards a frame based lexicon: the semantics of RISK and its neighbors. Frames, Fields and Constraints (Lawrence Erlbaum Associates), 75–102.
[9] Florian, R. & Wicentowski, R. (2002). Unsupervised Italian word sense disambiguation using WordNets and unlabeled corpora. Proceedings of the SIGLEX/SENSEVAL Workshop on Word Sense Disambiguation: Recent Success and Future Directions, 67–73.
[10] Gao, Z.-M. & Liu, C.-L. (2004). A Web-based assessment and profiling system for college English. Proceedings of the Eleventh International Conference on Computer Assisted Instruction, CD–ROM.
[11] Huang, S.-M., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted learning for English dictation. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.
[12] Irvine, S. H. & Kyllonen, P. C. (2002). Item Generation for Test Development (Lawrence Erlbaum Associates).
[13] Johns, T. http://web.bham.ac.uk/johnstf/timcall.htm.
[14] Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. Proceedings of the Special Group for Design of Communications Conference, 24–26.
[15] Lin, D. (1998). Dependency-based evaluation of MINIPAR. Proceedings of the Workshop on the Evaluation of Parsing Systems in the First International Conference on Language Resources and Evaluation.
[16] Liu, C.-L. (2004). Using mutual information for adaptive student assessments. Proceedings of the Fourth IEEE International Conference on Advanced Learning Technologies, to appear.
[17] Madsen, H. (1983). Techniques in Testing (Oxford University Press).
[18] Manning, C. D. & Schütze, H. (1999). Foundations of statistical natural language processing (MIT Press).
[19] Marcus, M. P., Santorini, B. & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19 (2), 313–330.
[20] Martin, B. & Mitrovic A. (2002). Automatic problem generation in constraint-based tutors. Proceedings of the Conference on Intelligent Tutoring System, 388–398.
[21] Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38 (11), 39–41.
[22] Mitkov, R. & Ha, L. A. (2003). Computer-aided generation of multiple-choice tests. Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications Using Natural Language Processing, 17–22.
[23] Ohlsson, S. (1991). Constraint-based student modeling. Proceedings of the NATO Advanced Research Workshop on Student Modeling, 167–189.
[24] Oranje, A. (2003). Automatic item generation applied to the national assessment of educational progress: Exploring a multilevel structural equation model for categorized variables. Education Testing Service: http://www.ets.org/research/dload/ncme03-andreas.pdf.
[25] Poel, C. J. & Weatherly, S. D. (1997). A cloze look at placement testing. Shiken: JALT (Japanese Assoc. for Language Teaching) Testing & Evaluation SIG Newsletter, 1 (1), 4–10.
[26] Ratnaparkhi, A. (1996). A maximum entropy part-of-speech tagger. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 133–142.
[27] Resnik, P. (1997). Selectional preference and sense disambiguation. Proceedings of the Applied Natural Language Processing Workshop on Tagging Text with Lexical Semantics: Why, What and How, 52–57.
[28] Reynar, J. C. & Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. Proceedings of the Conference on Applied Natural Language Processing, 16–19.
[29] Sheehan, K. M., Deane, P. & Kostin, I. (2003). A partially automated system for generating passage-based multiple-choice verbal reasoning items. Paper presented at the National Council on Measurement in Education Annual Meeting.
[30] Steven, V. (1991). Classroom concordancing: vocabulary materials derived from relevant authentic text. English for Specific Purposes, 10 (1), 35–46.
[31] Wang, C.-H., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted item generation for English vocabulary tests. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.
[32] Wilks, Y. & Stevenson, M. (1997). Combining independent knowledge sources for word sense disambiguation. Proceedings of the Conference on Recent Advances in Natural Language Processing, 1–7.
[33] Wilson, E. (1997). The automatic generation of CALL exercises from general corpora. Proceedings of the Conference on Teaching and Language corpora, 116–130.
[34] Yang, X.-F. & Li, T.-Q. (2002). A study of semantic disambiguation based on HowNet. Computational Linguistics and Chinese Language Processing, 7 (1), 47–78.
[35] Yarowsky, D. (1992). Word-sense disambiguation using statistical models of Roget`s categories trained on large corpora. Proceedings of COLING-92, 454–460.
描述 國立政治大學
資訊科學學系
91753024
92
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0091753024
資料類型 thesis
dc.contributor.advisor 劉昭麟zh_TW
dc.contributor.advisor Liu , Chao-Linen_US
dc.contributor.author (Authors) 王俊弘zh_TW
dc.contributor.author (Authors) Wang , Chun-Hungen_US
dc.creator (作者) 王俊弘zh_TW
dc.creator (作者) Wang , Chun-Hungen_US
dc.date (日期) 2003en_US
dc.date.accessioned 17-Sep-2009 13:54:53 (UTC+8)-
dc.date.available 17-Sep-2009 13:54:53 (UTC+8)-
dc.date.issued (上傳時間) 17-Sep-2009 13:54:53 (UTC+8)-
dc.identifier (Other Identifiers) G0091753024en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/32641-
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊科學學系zh_TW
dc.description (描述) 91753024zh_TW
dc.description (描述) 92zh_TW
dc.description.abstract (摘要) 多選題測驗試題已證明能有效地評估學生的學習成效,然而,以人為方式建立題庫是一件耗時費力的工作。藉由電腦高速運算的能力,電腦輔助產生試題系統能有效率地建置大規模的題庫,同時減少人為的干預而得以保持試題的隱密性。受惠於網路上充裕的文字資源,本研究發展一套克漏詞試題出題系統,利用既有的語料自動產生涵蓋各種不同主題的克漏詞試題。藉由分析歷屆大學入學考試的資料,系統可產生類似難度的模擬試題,並且得到出題人員在遴選測驗標的方面的規律性。在產生試題的過程中導入詞義辨析的演算法,利用詞典與selectional preference模型的輔助,分析句子中特定詞彙的語義,以擷取包含測驗編撰者所要測驗的詞義的句子,並以collocation為基礎的方法篩選誘答選項。實驗結果顯示系統可在每產生1.6道試題中,得到1道可用的試題。我們嘗試產生不同類型的試題,並將這套系統融入網路線上英文測驗的環境中,依學生的作答情形分析試題的鑑別度。zh_TW
dc.description.abstract (摘要) Multiple-choice tests have proved to be an efficient tool for measuring students’ achievement. Manually constructing tests items, however, is a time- consuming and labor-intensive task. Harnessing the computing power of computers, computer-assisted item generation offers the possibility of creating large amount of items, thereby alleviating the problem of keeping the items secure. With the abundant text resource on the Web, this study develops a system capable of generating cloze items that cover a wide range of topics based on existing corpra. By analyzing training data from the College Entrance Examinations in Taiwan, we identify special regularities of the test items, and our system can generate items of similar style based on results of the analysis. We propose a word sense disambiguation-based method for locating sentences in which designated words carry specific senses, and apply collocation-based methods for selecting distractors. Experimental results indicate that our system was able to produce a usable item for every 1.6 items it returned. We try to create different types of items and integrate the reported item generator in a Web-based system for learning English. The outcome of on-line examinations is analyzed in order to estimate the item discrimination of the test items generated by our system.en_US
dc.description.tableofcontents 第一章 緒論 1
第二章 文獻回顧與探討 7
第三章 語料來源與詞典 15
第四章 歸納克漏詞試題的特徵 22
第五章 產生克漏詞試題 29
第六章 評估與應用 42
第七章 結論 53
參考文獻 55
zh_TW
dc.format.extent 255952 bytes-
dc.format.extent 256766 bytes-
dc.format.extent 258879 bytes-
dc.format.extent 289314 bytes-
dc.format.extent 328571 bytes-
dc.format.extent 365549 bytes-
dc.format.extent 374292 bytes-
dc.format.extent 360576 bytes-
dc.format.extent 463046 bytes-
dc.format.extent 1273110 bytes-
dc.format.extent 281776 bytes-
dc.format.extent 312882 bytes-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.format.mimetype application/pdf-
dc.language.iso en_US-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0091753024en_US
dc.subject (關鍵詞) 電腦輔助語言學習zh_TW
dc.subject (關鍵詞) 自動產生試題zh_TW
dc.subject (關鍵詞) 試題編寫工具zh_TW
dc.subject (關鍵詞) 詞義辨析zh_TW
dc.subject (關鍵詞) 自然語言處理zh_TW
dc.subject (關鍵詞) Computer-Assisted Language Learningen_US
dc.subject (關鍵詞) Automatic Item Generationen_US
dc.subject (關鍵詞) Authoring Toolsen_US
dc.subject (關鍵詞) Word Sense Disambiguationen_US
dc.subject (關鍵詞) Natural Language Processingen_US
dc.title (題名) 電腦輔助克漏詞多選題出題系統之研究zh_TW
dc.title (題名) A Study on Computer Aided Generation of Multiple-Choice Cloze Itemsen_US
dc.type (資料類型) thesisen
dc.relation.reference (參考文獻) [1] 郭生玉,心理與教育測驗,台北:精華書局,1989。zh_TW
dc.relation.reference (參考文獻) [2] 鄭恆雄,陳凌霞,宋麗梅等,八十七年度英文命題實驗計畫研究報告,台北:大學入學考試中心,1999。zh_TW
dc.relation.reference (參考文獻) [3] Berger, A. L., Della Pietra, S. A. & Della Pietra, V. J. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22 (1), 39–71.zh_TW
dc.relation.reference (參考文獻) [4] Coniam, D. (1997). A preliminary inquiry into using corpus word frequency data in the automatic generation of English language cloze tests. Computer Assisted Language Instruction Consortium, 16 (2–4), 15–33.zh_TW
dc.relation.reference (參考文獻) [5] Deane, P. & Sheehan, K. (2003). Automatic item generation via frame semantics. Education Testing Service: http://www.ets.org/research/dload/ncme03-deane.pdf .zh_TW
dc.relation.reference (參考文獻) [6] Dennis, I., Handley, S., Bradon, P., Evans, J. & Nestead, S. (2002). Approaches to modeling item-generative tests. In: Item Generation for Test Development [12], 53–72.zh_TW
dc.relation.reference (參考文獻) [7] Dorr, B. J., Levow, G.-A. & Lin, D. (2000). Large-scale construction of a Chinese- English semantic hierarchy. Technical report, University of Maryland, College Park, MD.zh_TW
dc.relation.reference (參考文獻) [8] Fillmore, C. J. & Atkins, B.T. (1992). Towards a frame based lexicon: the semantics of RISK and its neighbors. Frames, Fields and Constraints (Lawrence Erlbaum Associates), 75–102.zh_TW
dc.relation.reference (參考文獻) [9] Florian, R. & Wicentowski, R. (2002). Unsupervised Italian word sense disambiguation using WordNets and unlabeled corpora. Proceedings of the SIGLEX/SENSEVAL Workshop on Word Sense Disambiguation: Recent Success and Future Directions, 67–73.zh_TW
dc.relation.reference (參考文獻) [10] Gao, Z.-M. & Liu, C.-L. (2004). A Web-based assessment and profiling system for college English. Proceedings of the Eleventh International Conference on Computer Assisted Instruction, CD–ROM.zh_TW
dc.relation.reference (參考文獻) [11] Huang, S.-M., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted learning for English dictation. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.zh_TW
dc.relation.reference (參考文獻) [12] Irvine, S. H. & Kyllonen, P. C. (2002). Item Generation for Test Development (Lawrence Erlbaum Associates).zh_TW
dc.relation.reference (參考文獻) [13] Johns, T. http://web.bham.ac.uk/johnstf/timcall.htm.zh_TW
dc.relation.reference (參考文獻) [14] Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. Proceedings of the Special Group for Design of Communications Conference, 24–26.zh_TW
dc.relation.reference (參考文獻) [15] Lin, D. (1998). Dependency-based evaluation of MINIPAR. Proceedings of the Workshop on the Evaluation of Parsing Systems in the First International Conference on Language Resources and Evaluation.zh_TW
dc.relation.reference (參考文獻) [16] Liu, C.-L. (2004). Using mutual information for adaptive student assessments. Proceedings of the Fourth IEEE International Conference on Advanced Learning Technologies, to appear.zh_TW
dc.relation.reference (參考文獻) [17] Madsen, H. (1983). Techniques in Testing (Oxford University Press).zh_TW
dc.relation.reference (參考文獻) [18] Manning, C. D. & Schütze, H. (1999). Foundations of statistical natural language processing (MIT Press).zh_TW
dc.relation.reference (參考文獻) [19] Marcus, M. P., Santorini, B. & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19 (2), 313–330.zh_TW
dc.relation.reference (參考文獻) [20] Martin, B. & Mitrovic A. (2002). Automatic problem generation in constraint-based tutors. Proceedings of the Conference on Intelligent Tutoring System, 388–398.zh_TW
dc.relation.reference (參考文獻) [21] Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38 (11), 39–41.zh_TW
dc.relation.reference (參考文獻) [22] Mitkov, R. & Ha, L. A. (2003). Computer-aided generation of multiple-choice tests. Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications Using Natural Language Processing, 17–22.zh_TW
dc.relation.reference (參考文獻) [23] Ohlsson, S. (1991). Constraint-based student modeling. Proceedings of the NATO Advanced Research Workshop on Student Modeling, 167–189.zh_TW
dc.relation.reference (參考文獻) [24] Oranje, A. (2003). Automatic item generation applied to the national assessment of educational progress: Exploring a multilevel structural equation model for categorized variables. Education Testing Service: http://www.ets.org/research/dload/ncme03-andreas.pdf.zh_TW
dc.relation.reference (參考文獻) [25] Poel, C. J. & Weatherly, S. D. (1997). A cloze look at placement testing. Shiken: JALT (Japanese Assoc. for Language Teaching) Testing & Evaluation SIG Newsletter, 1 (1), 4–10.zh_TW
dc.relation.reference (參考文獻) [26] Ratnaparkhi, A. (1996). A maximum entropy part-of-speech tagger. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 133–142.zh_TW
dc.relation.reference (參考文獻) [27] Resnik, P. (1997). Selectional preference and sense disambiguation. Proceedings of the Applied Natural Language Processing Workshop on Tagging Text with Lexical Semantics: Why, What and How, 52–57.zh_TW
dc.relation.reference (參考文獻) [28] Reynar, J. C. & Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. Proceedings of the Conference on Applied Natural Language Processing, 16–19.zh_TW
dc.relation.reference (參考文獻) [29] Sheehan, K. M., Deane, P. & Kostin, I. (2003). A partially automated system for generating passage-based multiple-choice verbal reasoning items. Paper presented at the National Council on Measurement in Education Annual Meeting.zh_TW
dc.relation.reference (參考文獻) [30] Steven, V. (1991). Classroom concordancing: vocabulary materials derived from relevant authentic text. English for Specific Purposes, 10 (1), 35–46.zh_TW
dc.relation.reference (參考文獻) [31] Wang, C.-H., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted item generation for English vocabulary tests. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.zh_TW
dc.relation.reference (參考文獻) [32] Wilks, Y. & Stevenson, M. (1997). Combining independent knowledge sources for word sense disambiguation. Proceedings of the Conference on Recent Advances in Natural Language Processing, 1–7.zh_TW
dc.relation.reference (參考文獻) [33] Wilson, E. (1997). The automatic generation of CALL exercises from general corpora. Proceedings of the Conference on Teaching and Language corpora, 116–130.zh_TW
dc.relation.reference (參考文獻) [34] Yang, X.-F. & Li, T.-Q. (2002). A study of semantic disambiguation based on HowNet. Computational Linguistics and Chinese Language Processing, 7 (1), 47–78.zh_TW
dc.relation.reference (參考文獻) [35] Yarowsky, D. (1992). Word-sense disambiguation using statistical models of Roget`s categories trained on large corpora. Proceedings of COLING-92, 454–460.zh_TW