以語意分析及Bloom理論為基礎之線上測驗輔助及智慧型評分系統

Abstract

隨著電子化學習(E-Learning)環境技術的普及，線上學習與線上測驗已成為資訊教育的重要議題。但是因為填充題及問答題等測驗類型在線上測驗系統上實施有許多問題需克服，當線上測驗系統提供填充題及問答題等題型測驗時，將會產生嚴重的測驗評分等化(Equation)問題。目前線上測驗系統大多仍以是非題、單選題及複選題等題型為主，雖有少數線上測驗系統提供填充題及其他開放式填答的測驗類型，但仍未針對受測者填答之答案進行的語意自動評分。<br>另外，現有線上測驗系統未提供教師設定個人化的評分風格，對於多位教師共用測驗系統平台時所產生的評分規則認知衝突，系統也未提供支援與解決，為了解決上述問題，並使線上測驗能具備與傳統測驗相同的評量效力，本研究使用模糊理論、相似語意詞庫及人工智慧概念等，發展一個線上測驗及智慧評分子系統，此系統除了包括一般測驗系統所提供的是非、單選、複選等題型外，也包含採用智慧評分機制來評分的填充題，完成雛形系統的建置後，本研究再針對傳統紙筆測驗、一般型評分機制、及本研究的智慧型評分機制進行評分效力比較的實證研究。<br>此部分的實證研究結果顯示，在包含填充題型的測驗中，不同的評分機制在測驗成績的評分結果上會有顯著差異，而智慧型評分機制運作初期雖然可以減少與紙筆評分間的差異，並改善一般型評分機制的評分效力，但仍無法在統計上獲得具有相同評分效力的結果。但是智慧評分機制在擴充詞彙語意後，「已擴充語意後的智慧評分機制」與「紙筆評分」的評分結果並無顯著差異，其顯示出若在包括填充題型的線上測驗系統中加入具有擴充詞彙語意關係知識的功能，並提供多功能的智慧型模糊評分機制，允許教師輸入代表個人評分風格習慣的評分規則參數，則線上評分系統將有可能具有與紙筆評分相同的評分效力來處理具有填充題題型的測驗工作。<br>然而線上測驗並不只是在測驗後給予受測者一個分數而已，而應該讓學習者了解自己在知識向度及認知向度的學習結果，因此測驗系統的試題若能包含Bloom教育目標分類資訊，將促使測驗活動能給予學習者更大的幫助。為了降低教師製作試題的負擔，本研究也以本體論、詞彙網路、Bloom分類理論、中文語意庫、人工智慧為基礎，提出一個輔助教師產製題庫的系統架構，並使電腦所產製的試題能涵蓋新版Bloom認知領域教育目標分類中的知識向度及認知向度概念。本研究在電腦輔助教師產製題庫的成果上，不僅能減少教師人工出題的負擔，系統產製的試題也能評量事實、概念及程序等三種知識及記憶、了解、應用、分析及評鑑等五種認知向度能力。受限於線上測驗系統能自動評分的四種題型，本研究尚無法產製屬於創造認知層次的試題，但是卻已能產製出包含基本知識概念的試題，並能提供具有Bloom概念的測驗題庫來評量學習成效。此外，本研究亦針對電子化學習環境，提出適用於線上測驗系統的試題品質及評分等化能力評估概念模式，強調Bloom理論在線上測驗系統環境中的使用範圍與限制。此模式也透過測驗等化觀念，針對教育領域測驗理論在評估試題品質概念上提出新的觀點。

Since the rapid E-learning development, the online learning and testing have been important topics of information education. Currently teachers still need to spend much time on creating and maintaining on-line testing item banks. Some researches have applied the new Bloom`s taxonomy to design meaningful learning assessments. This research has applied ontology, Bloom`s taxonomy, Chinese semantic database, artificial intelligence, semantic web, to design an on-line course learning system to assist teachers in creating test items.<br>Most of present on-line testing only has multiple choice items and true-false items. Though some provide fill-in-blank items, they can only recognize the answers either all right or all wrong through the simple computer binary pattern matching. In order to have the same evaluation effects as the traditional paper-and-pencil testing, this research will adopt the concepts of fuzzy theory, thesaurus, set, and artificial intelligence to develop the fuzzy scoring mechanism. The proposed on-line testing system will have true-false, multiple-choice, and fil-in-blank items. The latter will be graded through fuzzy judgment that is naturally endowed by the human teachers.<br>In addition, the past research indicated that e-learning students would learn more if provided appropriate feedback messages. In this research will add feedback messages to the proposed system according to different situations. The proposed on-line testing system will not only grade the test items, but also explain the answers and provide related materials to the testers.<br>The result of study are: (1) we could design the test items that would need a particular cognitive process to a particular type of knowledge, though we still could not have items to test “creative” level of cognitive process; (2) the test items could be used to assess the learning level meaningfully; (3) the computer would assist teachers to create a large number items, and save time of making item; (4) that different scoring mechanisms have a significant effect on test scores; (5) at the beginning, though our fuzzy on-line testing system is significantly better than the usual on-line testing system, it could not achieve the same effect as the paper-and-pencil testing; (6) after expanding semantic vocabularies from feedbacks, our fuzzy scoring mechanism is equivalent to paper-and-pencil.