Title: Reliability measures in item response theory: Manifest versus latent correlation functions
Authors: Milanzi, Elasma ×
Molenberghs, Geert
Alonso Abad, Ariel
Verbeke, Geert
De Boeck, Paul #
Issue Date: Feb-2015
Publisher: J. Wright & Sons
Series Title: British Journal of Mathematical and Statistical Psychology vol:68 issue:1 pages:43-64
Article number: 10.1111/bmsp.12033
Abstract: For item response theory (IRT) models, which belong to the class of generalized linear or non-linear mixed models, reliability at the scale of observed scores (i.e., manifest correlation) is more difficult to calculate than latent correlation based reliability, but usually of greater scientific interest. This is not least because it cannot be calculated explicitly when the logit link is used in conjunction with normal random effects. As such, approximations such as Fisher's information coefficient, Cronbach's α, or the latent correlation are calculated, allegedly because it is easy to do so. Cronbach's α has well-known and serious drawbacks, Fisher's information is not meaningful under certain circumstances, and there is an important but often overlooked difference between latent and manifest correlations. Here, manifest correlation refers to correlation between observed scores, while latent correlation refers to correlation between scores at the latent (e.g., logit or probit) scale. Thus, using one in place of the other can lead to erroneous conclusions. Taylor series based reliability measures, which are based on manifest correlation functions, are derived and a careful comparison of reliability measures based on latent correlations, Fisher's information, and exact reliability is carried out. The latent correlations are virtually always considerably higher than their manifest counterparts, Fisher's information measure shows no coherent behaviour (it is even negative in some cases), while the newly introduced Taylor series based approximations reflect the exact reliability very closely. Comparisons among the various types of correlations, for various IRT models, are made using algebraic expressions, Monte Carlo simulations, and data analysis. Given the light computational burden and the performance of Taylor series based reliability measures, their use is recommended.
ISSN: 0007-1102
Publication status: published
KU Leuven publication type: IT
Appears in Collections:Leuven Biostatistics and Statistical Bioinformatics Centre (L-BioStat)
× corresponding author
# (joint) last author

Files in This Item:
File Description Status SizeFormat
explicit15.pdfOA article Published 199KbAdobe PDFView/Open
425.pdf Published 358KbAdobe PDFView/Open Request a copy

These files are only available to some KU Leuven Association staff members


All items in Lirias are protected by copyright, with all rights reserved.

© Web of science