Studying Unconstrained Degraded Face Recognition and Redaction with Applications in Real Surveillance Environment

We highlight the contributions described in this dissertation and envision the future works. First, a comprehensive background, related works, and dataset information are given at the beginning of this dissertation. The latter part of the dissertation focuses on researching the LRFR task from several aspects. First, we curate two face datasets, the VBOLO dataset which was collected from a real surveillance camera network, and the NDFaceReID dataset which was collected under a near-surveillance environment.

Second, we employ deep neural networks and research on two major face recognition tasks in digital forensics, face re-identification, and face identification. For each task, we propose new deep architectures and training strategies to improve the performance of the real surveillance datasets. We also conduct studies on the capacity of the deep models on the LQ face images from these datasets utilizing an efficient model searching engine to get the insight on performance change as gradually change the complexity of the deep network and the input size.

Third, considering the fact of lacking training data at scale, we develop a practical way to solve the face recognition task under this consideration. We address weakly-supervised LRFR in videos by incorporating pre-trained general face recognition models as prior and propose an approach to gradually adapting a general face recognition model into a more robust LQ face recognition model.

Finally, we explore the face redaction task in surveillance quality videos. We address the challenges compared to a redaction system for HQ face images in video frames. By employing a state-of-the-art stacked hour-glass as well as intermediate supervision during training, we are able to evaluate the performance of a face redaction system with very LQ surveillance videos. By providing a qualitative result, we are able to foresee the potential directions to improve the existing method. First, a more reliable facial keypoint detector needs to specifically curated for LQ face images, contour or template matching could be applied when not detailed information on the face could be catching due to the image degradation. This could be fused with the SOTA face landmark detectors.