International Meeting of the Psychometric Society edition:79 location:Madison (WI), USA date:21-25 July 2014
In achievement testing, it is commonly assumed that item and person characteristics are invariant with respect to administration conditions. However, it has been repeatedly shown that the position of an item within a test form affects item performance. Performance can increase (‘learning/practice effect’) or decrease (‘fatigue effect’) with item-position. Hartig and Bochhulz (2012) and Debeer and Janssen (2013) proposed an IRT model that deals with this issue by adding a ‘position dimension’ (i.e., persistence) to the model. Aside a general effect of item position, this model allows for individual differences in persistence, and estimates the correlation between a test-taker’s ability and persistence.
This paper presents a multilevel extension of the model that decomposes the variance in persistence (and ability) as well as the correlation between the two dimensions, in hierarchical data. The model is applied to data from the 2009 PISA reading assessment. Data were analyzed separately for each country. The (1) general effect of item position, (2) the individual differences in persistence within and between schools, as well as (3) the pattern of correlations between students’ performance and persistence are compared across countries.