Jointly Modeling Participant-Level Data and Summary Statistics for Treatment Differences

Integrative data analysis (IDA) — the combination and joint analysis of participant-level data from multiple studies (Curran & Hussong, 2009) — and meta- analysis (MA) — the combination and joint analysis of summary statistics from multiple studies (Glass, 1976) — offer statistical methods for psychological researchers to leverage multiple data sets to jointly describe or test phenomena of interest. While adaptation of IDA continues to grow within psychology, MA remains the prevalent method of joint data analysis. This may be attributable to the relative difficulty of obtaining participant-level data (IPD) in practice, while (published) sufficient statistics or aggregated data (AD) remain more accessible (H. Cooper & Patall, 2009). The integration of both IPD and AD, however, remains virtually unexplored in the psychology literature despite recent methodological developments and applications in the medical literature (e.g., Jackson et al., 2006; Riley, Lambert, et al., 2008; Riley et al., 2007). Instead, multi-study applications have tended to exclusively use either IPD studies or AD studies. Methods combining IPD and AD remain under-evaluated and their ability to estimate different relationships or effects of interest and the quality of inferences remains an open question. This dissertation will introduce and evaluate multiple models for jointly analyzing cross-sectional IPD and AD studies when comparisons between levels of a dichotomous predictor (e.g., a treatment group and a control group) with respect to a continuous outcome are of interest. In Chapter 2, key estimands of interest are defined, modeling frameworks for joint analysis of IPD and AD are introduced, and which estimands are estimable using each approach are analytically investigated. In Chapter 3, an application of IDA, MA, and joint IPD-AD models and a Monte Carlo empirical simulation comparing modeling approaches are presented using an international sample of 47 studies of the Big Five personality traits (Johnson, 2014). In Chapter 4, a simulation study is conducted to evaluate the performance (i.e., convergence rate, parameter estimation bias, parameter coverage rate, and rejection rate) of IDA, MA, and IPD-AD models with varied numbers of studies (i.e., nine to 24) and relative amounts of available IPD and AD studies (i.e., IPD only, 33% IPD, 67% IPD, AD only) under different data generation assumptions. Chapter 5 presents key contributions and findings and suggests new directions for researchers interested in integrating both IPD and AD studies.