Missing Data in Randomized Pretest Posttest Studies: A Growth Curve Perspective

The randomized pretest posttest (RPP) study is a common experimental design in psychology. However, missing data at posttest is a ubiquitous occurrence with this type of design. Although missing data handling has seen advances over the past several decades, effective and practical solutions for handling missing data in RPP designs are lacking. This dissertation aims to fill this gap in the literature by comparing a variety of analysis models for estimating the average treatment effect (ATE) and its standardized counterpart, the standardized mean difference (SMD), under two different missing data handling techniques, listwise deletion (LD) and multiple imputation (MI). The analysis models of interest were studied under three specific patterns of missing at random (MAR) data, to model plausible scenarios by which data may be missing. Moreover, data were generated from a growth–curve perspective, under the assumption that change is a within–person phenomenon, regardless of the type of design used to measure the construct of interest. Although data were generated from growth–curve models, model assumptions (e.g., linearity and homogeneity of regression) that may have more deleterious consequences when violated under missing data are specified with respect to the pretest posttest relationship at the between–person level. Thus, it was first assessed whether the aforementioned assumptions were violated at the between–person level under various random effect specifications.

Results indicated that LD provided unbiased and efficient estimates of the ATE across all sample sizes and effect sizes, as long as the analysis model was appropriately specified to handle the violated linearity or homogeneity of regression assumption. For example, although standard analysis of covariance provides an unbiased ATE estimate with complete data under heterogeneity of regression, including the pretest by group interaction in the analysis model and centering pretest with respect to the mean of all participants were necessary when estimating the ATE using LD under these conditions in most missingness patterns. Moreover, the type of missingness pattern within MAR had implications for which analysis models were effective. When the goal was the estimate the SMD, patterns of results indicated that MI was more effective than LD in some conditions, although results were more mixed.