Behavior Research Methods vol:45 issue:2 pages:547-559
In this study, we focus on a three-level meta-analysis for combining data from studies using multiple-baseline across participants designs. A complicating factor in such designs is that results might be biased if the dependent variable is affected by not explicitly modeled external events, such as the illness of a teacher, an exciting class activity or the presence of a foreign observer. In multiple-baseline designs, external effects can become apparent if they simultaneously have an effect on the outcome score(s) of the participants within a study. This study presents a method to adjust the three-level model for external events and evaluates the appropriateness of the modified model. Therefore we use a simulation study, and we illustrate the new approach with real datasets.
The results indicate that ignoring an external event effect results in biased estimates of the treatment effects, especially when there is only a small number of studies and measurement occasions involved. The mean squared error, as well as the standard error and coverage proportion of the effect estimates are improved with the modified model. Moreover, the adjusted model results in less biased variance estimates. If there is no external event effect, we find no differences in results between the modified and unmodified models.