Behavior Research Methods vol:41 issue:1 pages:192-203
The importance of accurate estimation and powerful statistical tests is widely recognized but rarely brought into practice in the social and behavioral sciences. This is especially true for estimation and testing when dealing with multilevel designs, not in the least because of having to deal with variances and research units at several levels. The complexity further increases for imbalanced designs, often necessitating simulation studies to perform accuracy and power calculations. It will be shown, however, using such simulation studies, that the distortion of balance can be ignored in most cases, making efficiency studies simpler and the use of existing software valid. An exception is suggested for imbalanced data with a large majority of small groups. Furthermore, the empirical sampling distribution of variance parameters may show substantial skewness and kurtosis depending on the number of groups and for the random slope also the group’s size, adding another caveat to the recommendation to ignore imbalance.