Post-Hoc Comparisons and Classification Accuracy in the Factor Mixture Model and Taxometric Procedures

Researchers in the social and behavioral sciences are frequently concerned whether the constructs they study should be represented as categorical types (classes) or continuous traits (factors). Two approaches to testing such type versus trait hypotheses are Meehlian taxometric procedures and the factor mixture model (FMM). The use of taxometric procedures and the more recently developed FMM has dramatically increased in the last ten years, and initial comparisons have shown that the latent structures that can be detected by taxometric procedures are a subset of the latent structures than can be detected by the FMM (lubke_latent_2009). Users of both the FMM and taxometric procedures frequently assign subjects to latent classes and then perform post-hoc comparisons to estimate class covariate effects. It is obvious that the accuracy of this approach will depend on class assignment accuracy, though this has not been previously studied. A simulation study was conducted to compare the two approaches on their relative performance for accuracy in post-hoc estimation of class covariate effects. Results show that the FMM can detect classes better than taxometric procedures, and that parameter estimates for post-hoc procedures can be quite biased, especially in conditions where assignment accuracy is poor. It is also shown that direct estimation of class covariate effects using the FMM provides more accurate results. Even though direct estimation of covariate effects in the FMM is possible, there some situations where post-hoc comparisons may be advantageous. This includes situations where it is desirable that the interpretation and estimation of the classes not be conditional on class covariates. Before taking the post-hoc comparison approach, classification must be accurate. In practice class assignment accuracy is unknown. Bootstrap methods are developed for examining the sampling variability of estimated individual posterior probabilities. These bootstrap methods provide the means to estimate individual classification accuracy using confidence intervals or empirical standard errors. Simulation results suggest that accurate classification is possible in some conditions for some subjects, though it is expected that classification accuracy may often be poor in many applied settings. In practice, classification accuracy should be checked prior to engaging in post-hoc comparisons.