Author:

Keywords:

IRT, examinee effort, position effect, PISA, Social Sciences, Education & Educational Research, Social Sciences, Mathematical Methods, Psychology, Mathematical, Mathematical Methods In Social Sciences, Psychology, testing, item response theory, ITEM RESPONSE-TIME, MISSING-DATA, MOTIVATION, ACHIEVEMENT, PERFORMANCE, POSITION, BEHAVIOR, MODEL, 0104 Statistics, 1303 Specialist Studies in Education, 1701 Psychology, Social Sciences Methods, 3904 Specialist studies in education, 4905 Statistics

Abstract:

In this article, the change in examinee effort during an assessment, which we will refer to as persistence, is modeled as an effect of item position. A multilevel extension is proposed to analyze hierarchically structured data and decompose the individual differences in persistence. Data from the 2009Program of International Student Achievement (PISA) reading assessment from N ¼ 467,819 students from 65 countries are analyzed with the proposed model, and the results are compared across countries. A decrease in examinee effort during the PISA reading assessment was found consistently across countries, with individual differences within and between schools. Both the decrease and the individual differences are more pronounced in lower performing countries. Within schools, persistence is slightly negatively correlated with reading ability; but at the school level, this correlation is positive in most countries. The results of our analyses indicate that it is important to model and control examinee effort in low-stakes assessments.