Title: Combining several types of single-case experimental designs using a three-level meta-analysis
Authors: Moeyaert, Mariola ×
Ugille, Maaike
Onghena, Patrick
Van Den Noortgate, Wim #
Issue Date: Apr-2014
Conference: Annual Meeting of the American Educational Research Association (AERA) location:Philadelphia, Pennsylvania, USA date:3 - 7 April 2014
Abstract: The purpose of the study is to further extend the multilevel meta-analysis of single-subject experimental designs (SSEDs) by including complex SSEDs such as alternating treatment designs (ATDs) and ABAB reversal designs in addition to the multiple-baseline across participants design (MBDs). While in recent years, there is attention for the synthesis of MBDs using the two-level model (Ferron, Bell, Hess, Rendina-Giobioff, & Hibard, 2009; Van den Noortgate & Onghena, 2003) or the three-level model (Owens & Ferron, 2010; Moeyaert, Ugille, Ferron, Beretvas, & Van den Noortgate, in press), combining data from ABAB designs and ATDs hardly received any attention. Combining data from different designs studying the same research question increases both the validity and reliability.
The method we propose to combine data from these designs involves three steps. First we suggest calculating case-specific effect sizes using ordinary least square regression. Note that in the ABAB design, an immediate treatment effect and a treatment effect on the slope for the first and the second AB pair is calculated (Kazdin, 2011). Also for the ATDs an immediate treatment effect and a treatment effect on the slope for the first and the second treatment is calculated (Kazdin, 2011). Whereas we have one immediate treatment effect and one treatment effect on the time trend per MBD, we have two immediate treatment effects and two treatment effects on the slope for the ABAB designs and the ATDs. In a second step we standardize the case-specific effect sizes by using the estimated within-case residual variance (i.e., obtained by the OLS analysis). In a last step, we suggest conducting two separate three-level meta-analyses or one multivariate three-level meta-analysis to estimate the average immediate treatment effect and the treatment effect on the slope, and their variation across cases and studies. A predictor indicating the design type is included at the third level of the three-level model because (part of) the variation in outcome scores between studies might be explained by the design.
We illustrated the proposed method using a published meta-analysis of single-cases including the three types of SSEDs. We also performed a simulation study to empirically validate the suggested method. We simulated data using the three-level model proposed by Van den Noortgate and Onghena (2003), under a variety of realistic conditions, varying the sample sizes at each of the levels (i.e., the number of measurement occasions per case, the number of cases per study, and the number of studies), the between-study, the between-case variance, and the within-case variance, the size of the immediate treatment effects, and the treatment effects on the slope. We simulated 2,000 datasets for each condition. By looking at the bias, the mean squared error, the standard error, the confidence intervals of the 95% significant level and the power, we evaluate the three-level meta-analysis of several types of SSEDs. In addition to the analysis across designs, we also perform the analysis per design type which might be of interest when the overall analysis gives biased results.
Publication status: accepted
KU Leuven publication type: IMa
Appears in Collections:Methodology of Educational Sciences
Faculty of Psychology and Educational Sciences, Campus Kulak Kortrijk – miscellaneous
Faculty of Psychology and Educational Sciences - miscellaneous
× corresponding author
# (joint) last author

Files in This Item:

There are no files associated with this item.


All items in Lirias are protected by copyright, with all rights reserved.