Linking statistics of betting behavior to difficulties of test items: An exploration

Abstract

Item difficulty is a key parameter for test items. Learning this parameter in an IRT-based context with statistical methods or machine learning-based approaches is a typical research topic in the field of educational assessment. This paper reports an exploration of using a betting mechanism to assess test takers` intrinsic uncertainty about the answers to a special type of Chinese cloze tests. Uncertainty about the answers influences test takers` behavior in betting on the candidate answers, so statistics of betting behavior serve as a conceivable incarnation of item difficulties. The proposed approach is innovative in that there is no known previous work that employed economics-based methods for this educational data mining problem. The proposed method was evaluated with more than a thousand realistic test items and with 48 native speakers of Chinese. Experimental results show encouraging connections between betting behavior and item difficulty. More specifically, we observed that participants may even lose money in very difficult tests, and they spent longer time in more challenging tests. © 2013 IEEE.