Gallery-Free Methods for Detecting and Recognizing People and Groups of Interest in the Wild

Doctoral Dissertation


The 2013 Boston Marathon investigation presented an ample opportunity to apply advanced face recognition technology. Unfortunately, the crowded scenes proved too difficult for existing facial biometrics methods and databases. In this dissertation, we identify some of the limitations of this technology and present approaches for overcoming these deficiencies. These approaches underlie a suite of novel algorithms for detecting faces in unconstrained imagery; clustering images of faces into identity groups; finding people who appear frequently across a collection of scenes; and detecting groups of individuals who often appear together.

Face detection typically serves as the first step in a facial analysis pipeline. We elucidate the difficulties encountered on uncontrolled imagery by a simple yet strong-performing face detector. Although pose has traditionally been the focus of much of the research on face detection, we show that occlusion and blur have at least as significant of an impact for data collected from natural scenes. The results also indicate that classifiers with distinct error modes can result from blur-based perturbations of a detector training set, enabling the successful application of fusion techniques.

Identity clustering plays a similar role to face detection insofar as it can drive automatic “pattern of life” analyses. We introduce an active clustering scheme, the Framework for Active Clustering with Ensembles (FACE), with an emphasis on clustering face images. FACE forms high-fidelity identity clusters by integrating a minimal amount of human feedback with automatic face recognition results. Recognition errors are mitigated through the solicitation of human decisions regarding ambiguously matched faces. This scheme precludes the need for a pre-existing gallery or database of known identities, since the gallery is essentially defined by the results of the clustering. Our experiments show that the FACE algorithm is more accurate and parsimonious than the state-of-the-art in active clustering, regardless of whether the human feedback is noisy. At the same time, the FACE algorithm promotes high performance in appearance frequency and affiliation analyses. These performance trends illustrate the potential efficacy of the proposed suite of algorithms in aiding data mining efforts encountered by the counter-terrorism and law enforcement communities.


Attribute NameValues
  • etd-12012014-092905

Author Jeremiah Ross Barr
Advisor Patrick J. Flynn
Contributor Aaron Striegel, Committee Member
Contributor Kevin W. Bowyer, Committee Co-Chair
Contributor Patrick J. Flynn, Committee Co-Chair
Contributor Douglas Thain, Committee Member
Degree Level Doctoral Dissertation
Degree Discipline Computer Science and Engineering
Degree Name PhD
Defense Date
  • 2014-09-21

Submission Date 2014-12-01
  • United States of America

  • clustering

  • semi-supervised learning

  • face detection

  • cluster analysis

  • Face recognition

  • University of Notre Dame

  • English

Record Visibility Public
Content License
  • All rights reserved

Departments and Units


Please Note: You may encounter a delay before a download begins. Large or infrequently accessed files can take several minutes to retrieve from our archival storage system.