Title: An IRT-based comparison of an item- and an examinee-centered method for standard setting
Authors: Van Nijlen, DaniĆ«l ×
Janssen, Rianne
Volckaert, Bartel #
Issue Date: 2004
Conference: Paper presented at the Biennial Conference of the Society for Multivariate Analysis in the Behavioural Sciences, Jena, Germany
Abstract: In item-centered methods for standard setting, judges indicate for each item the expected level of performance of a minimally competent student (MCS). The item judgements are combined into a cutoff on the test score scale. In examinee-centered methods, judges classify the examinees into different categories of mastery, such as masters versus nonmasters. A cutoff score is derived from a model that predicts category membership on the basis of the examinees' test performance in both groups.

In the present study, the standards derived from an item-centered and an examinee-centered method are compared on a test assessing the Flemish curriculum standards in biology for primary education. Using the revised Angoff method, primary school teachers indicated for each test item whether a MCS should be able to solve it correctly. Using the contrasting-group method, the same teachers also classified the pupils of their class into masters and nonmasters.

The two methods are compared using item response theory (IRT). Using the two-parameter IRT solution for the test, the judgements of the revised Angoff method can be transformed into an estimate of the location of the MCS on the ability scale (Van der Linden, 1982). A logistic regression of the estimated abilties of the students on their dichotomous classification by their teacher leads to a cutoff on the ability scale as well (Longford, 1996). A unified IRT model for the analysis of both methods of standard setting will be presented. The model is estimated in a Bayesian way using data augmented Gibbs sampling.
Description: 12
Publication status: published
KU Leuven publication type: IMa
Appears in Collections:Leuven Institute for Educational Research (-)
Education and Training
× corresponding author
# (joint) last author

Files in This Item:

There are no files associated with this item.

Request a copy


All items in Lirias are protected by copyright, with all rights reserved.