Multiple-Attempt Procedures: Models, Computerized Adaptive Testing, and Differential Item Functioning

Multiple-attempt items are an innovative item type that remains under-studied in psychometrics and educational measurement. This dissertation advances the field by (a) extending sequential item-response theory for multiple-choice, multiple-attempt items (SIRT-MM), (b) designing computerized adaptive testing that incorporates multiple-attempt items, and (c) clarifying and detecting differential item functioning for such items. Chapter 2 introduces two extensions of the SIRT-MM model. The first permits the slope of each item-category response function to vary, while the second freely estimates a pseudo guessing parameter to capture different success rates due to guessing. These models allow a wider range of response-function shapes and are more likely to fit empirical data. Model-selection strategies and parameter estimation methods for the new formulations are also proposed and evaluated. Chapter 3 explores the integration of multiple-choice, multiple-attempt test items within the Computerized Adaptive Testing (CAT) framework, named as MM-CAT. Using the sequential item response theory model for multiple-choice, multiple-attempt items (Lu, Fowler, & Cheng, 2025), a simulation study was conducted to investigate the effectiveness of a MM-CAT design in improving ability estimation accuracy compared to traditional CAT, which relies on single-attempt, dichotomously scored items. Results show that MM-CAT substantially reduces the standard error of measurement (SEM), bias and root mean square error (RMSE), particularly for examinees with lower ability levels. Furthermore, we examine the impact of item exposure control procedures and find that while both the Sympson-and-Hetter method (SH; Shealy & Stout, 1993) and the Randomesque method (Kingsbury & Zara, 1989) are useful, the SH method is particularly effective in exposure control when paired with MM-CAT, minimizing the severeness of over-exposed items without sacrificing the measurement precision. Taken together, these findings suggest that MM-CAT is a promising approach for enhancing the precision and fairness of adaptive testing, especially in educational contexts where multiple attempts may support both assessment and learning. While multiple-attempt procedures and items have been widely studied, limited research has addressed Differential Item Functioning (DIF) in the context of multiple-attempt items. Chapter 4 formalizes the concept of attempt-level DIF, which captures attempt-specific mechanisms underlying DIF. We present example scenarios to illustrate how attempt-level DIF can arise and propose several detection methods capable of identifying it. Simulation results demonstrate that these methods yield higher true positive rates (i.e., greater power) compared to traditional DIF detection approaches. Their advantage is particularly evident when the sample size and variance of item responses are reduced in the specific attempt where DIF exists.<p></p>