Please use this identifier to cite or link to this item:

Title: 加入作答時間之試題反應模型在能力上的研究
A study of the ability after incorporating response time in the item response model
Authors: 曾定柏
Tseng, Ting-Po
Contributors: 姜志銘

Jiang, Zhi-Ming
Song, Chwan-Chin

Tseng, Ting-Po
Keywords: 試題反應理論
Response time
Scoring rule
Date: 2020
Issue Date: 2020-09-02 12:14:35 (UTC+8)
Abstract: 本研究旨在探討作答時間是否適合作為受試者能力值估計的一項因素。本文從制定一種加入作答時間的評分規則為出發點,建立一個包含作答反應與作答時間的模型,再以最大概似估計法估計能力值與難度值,並透過實際數據之計算結果,分析能力估計值在加入作答時間前與後是否有所不同。最後,探討在作答反應組型相同時,能否以此新模型進一步區分受試者能力值的高低。
This study aims to explore whether the response time is an important factor for estimating the abilities of examinees. After giving a scoring rule, which incorporates both item response and response time, we build a new model and can then estimate the ability of any examinee and the difficulty of any item by using the method of maximum likelihood estimation. Through the real data, we compare examinees’ abilities based on the IRT and that based on our new speed-accuracy response model (NSARM). Finally, we explore whether this new model can further distinguish the abilities of two examinees when their response patterns are the same.
Through the simulations, we find that, on ability estimate, our NSARM shows more accurate than IRT model among those examinees with high ability. In addition, it is even more accurate when the number of examinees increases.
After analizing our real data, we further summarize the following three results:
1.When the item is responded incorrectly, the score based on our new model is posi-tively correlated with the response time.
Among the randomly selected items, no mater they are simple, medium or difficult, we found that the estimated abilities of the examinees who incorrectly answered the items, are positively correlated with their response times. Therefore, an examinee taking shorter response time should be given a lower score (i.e., deduct more additional score), and an examinee taking longer response time should be given a higher score (i.e., deduct less additional score).
2.The gamma distribution is more appropriate for modeling the response time.
Using the goodness of fit test, we found that the exponential distribution, which is used by many authors, is not appropriate to model the response time of some items in our data set. However, we further found that the gamma distribution, which is a generalization of the exponential distribution, is appropriate.
3.Our new model can further distinguish the abilities of examinees.
Neither classical test theory nor item response theory can distinguish the abilities of examinees when their response patterns are the same.
Reference: 中文文獻
(1) 余民寧 (2009)。試題反應理論(IRT)及其應用。臺北:心理出版社股份有限公司。
(2) 陳俊宏、陳淑英 (2010)。試題作答時間在電腦適性測驗上的應用---改善具洩題資訊受試者的能力估計精確性。測驗學刊第五十七輯第四期P.459~483。
(3) 陳茂雄 (2017)。作答時間與答題機率關係的探討~以小數的乘法單元為例。國立台中教育大學,教育資訊與測驗統計研究所。
(4) 黃聖筠、陳淑英 (2009)。「試題作答時間」在洩題偵測上的應用。測驗學刊第五十六輯第四期P.543~571。
(5) 張雅媛 (2007)。融合kernel smoothing 之MMLE法於IRT參數估計之應用。國立台中教育大學,教育測驗統計研究所。
(6) 郭伯臣、謝典佑、吳慧珉、林佳樺 (2012)。一因子高層次試題反應理論模式之評估。測驗學刊第五十九輯第三期P.329~348。
(1) Allen, W. J. and Yen, W. M. (2001). Introduction to measure theory (2nd ed.). Monterey, CA: Brooks/ Cole.
(2) Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
(3) Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In: Lord, F.M. and Novick, M.R., Eds., Statistical Theories of Mental Test Scores, Addison-Wesley, Reading, 397-479.
(4) Casella, G., and Berger, L. (1990). Statistical inference. Pacific Grove, CA: Brooks/Cole.
(5) Crocker, L. and Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
(6) de la Torre, J. and Song, H. (2009). Simultaneous estimation of overall and domain abilities: A higher-order IRT model approach. Applied Psychological Measurement 33 (8), 620-639.
(7) Dennis, I. and Evans, J. (1996). The speed-error trade-off problem in psychometric testing. British Journal of Psychology, 87, 105–129.
(8) Ferrando, P. J., and Lorenzo-Seva, U. (2007). An item-response model incorporating response time data in binary personality items. Applied Psychological Measurement, 31, 525–543.
(9) Gaviria, J. L. (2005). Increase in precision when estimating parameters in computer assisted testing using response times. Quality & Quantity, 39, 45–69.
(10) Guion, R. M. and Ironson, G. H. (1983). Latent trait theory for organizational research. Organizational Behavior and Human Performance, 31, 54-87.
(11) Lord, F. M. (1952). A theory of test scores. Psychometric Monograph, No. 7.
(12) Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
(13) Lord, F. M. and Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
(14) Maris, G. and Van der Maas, H. L. (2012). Speed-Accuracy Response Models: Scoring Rules based on Response Time and Accuracy. Psychometrika, 77(4), 615–633.
(15) Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Chicago:University of Chicago Press.
(16) Roskam, E. E. (1987). Toward a psychometric theory of intelligence. In E. E. Roskam and R. Suck (Eds.), Progress in mathematical psychology (pp. 151–171). Amsterdam: North Holland.
(17) Thissen, D. (1983). Timed testing: An approach using item response theory. In D. J. Weiss(Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing.(pp. 179–203) New York: Academic Press.
(18) Thurstone, L. L. (1937). Ability, motivation, and speed. Psychometrika, 2, 249–254.
(19) Tucker, L. R. (1946). Maximum validity of a test with equivalent items. Psychometrika, 11, 1-13.
(20) Van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika,72, 287–308.
(21) Van der Linden, W. J. (2009). Conceptual Issues in Response-Time Modeling. Journal of Educational Measurement, Vol. 46, No. 3, pp. 247–272.
(22) Van der Maas, H. L. and Wagenmakers, E. J. (2005). A psychometric analysis of chess expertise. The American Journal of Psychology, 118(1), 29–60.
(23) Verhelst, N. D., Verstralen, H. H. F. M., and Jansen, M. G. (1997). A logistic model for timelimit tests. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 169–185). New York: Springer.
(24) Wang, T., and Hanson, B. A. (2005). Development and calibration of an item response model that incorporates response time. Applied Psychological Measurement, 29, 323–339.
(25) Wickelgren, W. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85.
Description: 碩士
Source URI:
Data Type: thesis
Appears in Collections:[應用數學系] 學位論文

Files in This Item:

File Description SizeFormat
101801.pdf4582KbAdobe PDF0View/Open

All items in 學術集成 are protected by copyright, with all rights reserved.

社群 sharing