NCME Annual Meeting edition:74 location:Vancouver (CA) date:12-16 april 2012
Changing the order of items between alternate test forms is a common practice in achievement testing to prevent copying and to enhance test security. However, these changes in item order may affect test characteristics or item characteristics. Several procedures have been proposed for studying these item order effects. The present study proposes a new approach, and explores the use of item response theory (IRT) in descriptive and explanatory models (De Boeck & Wilson, 2004) for investigating and modeling the effects of test item position on item discrimination and difficulty. Combining differential item functioning (DIF) and linear logistic test models (LLTM), we detected and modeled the effects of item position in a one-step procedure. In addition, the IRT framework allows us to consider individual differences in the position effect on item difficulty by introducing an additional position dimension. As an illustration, the framework is applied to a listening comprehension test for French as a foreign language, and data from the PISA 2006 study.
The present study was supported by several grants from the Flemish Ministry of Education.
For the data-analysis, we used the infrastructure of the VSC - Flemish Supercomputer Center, funded by the Hercules foundation and the Flemish Government - department EWI.