IRT and SVD: Implementing Psychometric Methods in New and Complex Situations

Doctoral Dissertation

Abstract

Most psychometric techniques used to analyze assessment data are designed to work with complete data. The rapid increase in the availability and power of technology has contributed to the growing use of computerized tests and related methods. The data arising from these new and complex situations challenge traditional psychometric techniques because of their size (as there is much more data) and their vast missingness (as students respond to only a small subset of possible items). This dissertation focuses on the effect of missing data on psychometric techniques.

When individuals respond to different items of varying difficulty, the psychometric techniques that rely on complete data can perform poorly. This dissertation proposes using Singular Value Decomposition (SVD), a matrix decomposition technique often seen in data mining, as a psychometric tool. The major result is that SVD is a viable psychometric tool that appears largely robust to missing data and to the missing mechanism. This document provides analytical and empirical justification for SVD’s use with psychometric data under missing data.

Chapter 1 introduces relevant IRT techniques such as nonparametric IRT and a nonparametric item fit statistic. Then, in Chapter 2, SVD is introduced and as well as an Alternating-Least-Squares (ALS) algorithm that extends the decomposition to missing data. Chapter 3 investigates the large sample properties of using SVD with psychometric data. SVD is shown to be a consistent ordinal estimator of student ability and a consistent ordinal estimator of item easiness.

Chapter 4 presents simulation results that show that when students respond to different items of varying difficulty, whether the missingness is related to their ability or not, SVD can rank the students better than proportion correct and can better estimate the true relationship between student ability and the probability of a correct response. When missingness is related to student ability, SVD can rank the students, in most conditions, better than a parametric IRT model, even when the parametric model is correctly specified.

Attributes

Attribute NameValues
URN
  • etd-04132015-134701

Author Quinn Nathaniel Lathrop
Advisor Ying Cheng
Contributor Guangjian Zhang, Committee Member
Contributor Scott Maxwell, Committee Member
Contributor Ying Cheng, Committee Chair
Contributor Zhiyong Zhang, Committee Member
Degree Level Doctoral Dissertation
Degree Discipline Psychology
Degree Name PhD
Defense Date
  • 2015-04-10

Submission Date 2015-04-13
Country
  • United States of America

Subject
  • missing data

  • singular value decomposition

  • data mining

  • computation statistics

  • educational data

  • Item response theory

  • psychometrics

Publisher
  • University of Notre Dame

Language
  • English

Record Visibility and Access Public
Content License
  • All rights reserved

Departments and Units

Files

Please Note: You may encounter a delay before a download begins. Large or infrequently accessed files can take several minutes to retrieve from our archival storage system.