Advancing Biometrics and Image Forensics Through Vision and Learning Systems

Doctoral Dissertation
Thumbnail

Abstract

The internet is a vast and rapidly changing landscape. For every new algorithm invented to increase the security and reliability of our society, an adversarial group coalesces in reaction. Some may argue that this constant arms race between defenders and attackers is currently being won by the latter. To shift the tides, we need scalable approaches to counter their attacks, whatever they may be, at the scale and speed with which the internet moves. This dissertation focuses on developing computer vision algorithms to help ensure security within three main areas:

1) In the video surveillance chapter (Ch. 2), we focus on the problem of Re-Identification, or finding all appearances of a given subject across a given camera system. Most all existing ReID approaches employ local and global body features (e.g., clothing color and pattern, body symmetry, etc.). These `body ReID’ methods implicitly assume that facial resolution is too low to aid in the ReID process. We assert that faces, even when captured in low resolution environments, may contain unique and stable features for ReID. Such `facial ReID’ approaches are relatively unexplored in the literature. We explore facial ReID using a new dataset that was collected from a real surveillance network in a municipal rapid transit system

2) In the facial recognition chapter (Ch. 3), we focus on the problem of off-pose faces. If a Convolutional Neural Network (CNN) is intended to tolerate facial pose, then we face an important question: should this training data be diverse in its pose distribution, or should face images be normalized to a single pose in a pre-processing step? To address this question, we evaluate a number of facial landmarking algorithms and a popular frontalization method to understand their effect on facial recognition performance.

3) In the Image Forensics and retrieval chapters (Ch. 4, 5, and 6), we focus on building novel image retrieval algorithms that can retrieve and trace modified images back to their origins. Images from social media can reflect diverse viewpoints, heated arguments, and expressions of creativity. They are often composites, borrowing content from many different sources to create a single image. Taking into account these attributes introduces new complexity to image search tasks. We propose multiple new frameworks for image retrieval that model object-level regions, allowing for fine-grained object level retrieval results. These retrieval results can help aid in the task of verifying the forensic integrity of an image. We then utilize these algorithms to perform real-world analysis on data scraped from Twitter and Instagram from the 2019 Indonesian Election.

Attributes

Attribute NameValues
Author Joel Robert Gray Brogan
Contributor Edward Delp, Committee Member
Contributor Patrick Flynn, Committee Member
Contributor Walter J. Scheirer, Research Director
Contributor Kevin W. Bowyer, Research Director
Contributor Adam Czajka, Committee Member
Degree Level Doctoral Dissertation
Degree Discipline Computer Science and Engineering
Degree Name Doctor of Philosophy
Banner Code
  • PHD-CSE

Defense Date
  • 2019-06-11

Submission Date 2019-07-11
Subject
  • ReIdentification

  • Machine Learning

  • Image Phylogeny

  • Image Retrieval

  • Image Forensics

  • Meme Analysis

  • Image Provenance

  • Computer Vision

  • Biometrics

  • Facial Recognition

Language
  • English

Record Visibility and Access Public
Content License
  • All rights reserved

Departments and Units
Catalog Record

Files

Please Note: You may encounter a delay before a download begins. Large or infrequently accessed files can take several minutes to retrieve from our archival storage system.