A Statistical Analysis of Testlets - A Parametric Approach

Abstract

Based on the basic concepts in classical test theory (CTT), Song et al. (2014) proposed a parametric method to develop the computational formulas of difficulty index and discrimination index for independent test items. In this article, modeling testlets with appropriate probability structures, we generalize their results to items in testlets. This parametric approach considers the dependence between the items within each testlet. It would also take the effect of performance of middle-scoring group on these two index values into account. In addition, we provide an efficient computing algorithm for obtaining both of them by using the probability generating function technique. Real data taken from the English Test Items of the Second Basic Competence Test for Junior High School Students in 2007 in Taiwan are used for empirical study, and the results are compared with those obtained by the traditional nonparametric method. Discrepancies between these two methods are also discussed in this study.