加入作答時間之試題反應模型在能力上的研究

Abstract

本研究旨在探討作答時間是否適合作為受試者能力值估計的一項因素。本文從制定一種加入作答時間的評分規則為出發點，建立一個包含作答反應與作答時間的模型，再以最大概似估計法估計能力值與難度值，並透過實際數據之計算結果，分析能力估計值在加入作答時間前與後是否有所不同。最後，探討在作答反應組型相同時，能否以此新模型進一步區分受試者能力值的高低。\n透過模擬數據進行模型驗證，在估計受試者能力值時，與IRT模式比較，我們發現本文所建立之模型對高能力群之受試者，有較佳的估計精準度，且受試者人數越多，精準度越好。\n同時，經過實證分析，本文歸納出以下三項結果：\n1.答錯時，使用本文所建立之評分規則的分數與作答時間呈現正相關。\n在篩選的題目中，無論是簡單、中等或是困難的試題，我們發現答錯之受試者的能力估計值與作答時間均呈現正相關，因此對於作答時間較短的受試者應給予較少的得分(即扣分較多)，而作答時間較長的受試者反而應給予較多的得分(即扣分較少)。\n2.作答時間宜採用伽瑪分布。\n在篩選的試題中，經由適合度檢定，我們發現部分試題的作答時間並不適合指數分布，宜改用較一般化的伽瑪分布。\n3.作答反應組型相同時，以本文所建立之模型能進一步區分受試者的能力值。\n無論是古典測驗理論或是試題反應理論，皆無法從作答反應組型相同的試題中區分受試者的能力值，但採用加入作答時間之試題反應模型後，我們不僅可經由這幾道試題進一步區分受試者的能力值，而且其與依全部試題所估計之能力值呈現高度的秩相關。

This study aims to explore whether the response time is an important factor for estimating the abilities of examinees. After giving a scoring rule, which incorporates both item response and response time, we build a new model and can then estimate the ability of any examinee and the difficulty of any item by using the method of maximum likelihood estimation. Through the real data, we compare examinees’ abilities based on the IRT and that based on our new speed-accuracy response model (NSARM). Finally, we explore whether this new model can further distinguish the abilities of two examinees when their response patterns are the same.\nThrough the simulations, we find that, on ability estimate, our NSARM shows more accurate than IRT model among those examinees with high ability. In addition, it is even more accurate when the number of examinees increases.\nAfter analizing our real data, we further summarize the following three results:\n1.When the item is responded incorrectly, the score based on our new model is posi-tively correlated with the response time.\nAmong the randomly selected items, no mater they are simple, medium or difficult, we found that the estimated abilities of the examinees who incorrectly answered the items, are positively correlated with their response times. Therefore, an examinee taking shorter response time should be given a lower score (i.e., deduct more additional score), and an examinee taking longer response time should be given a higher score (i.e., deduct less additional score).\n2.The gamma distribution is more appropriate for modeling the response time.\nUsing the goodness of fit test, we found that the exponential distribution, which is used by many authors, is not appropriate to model the response time of some items in our data set. However, we further found that the gamma distribution, which is a generalization of the exponential distribution, is appropriate.\n3.Our new model can further distinguish the abilities of examinees.\nNeither classical test theory nor item response theory can distinguish the abilities of examinees when their response patterns are the same.