Bias in Face Recognition: From Causes to Mitigation

Face recognition technology has recently become a topic of controversy over concern about possible bias across demographics. However, media coverage has not always been concerned with an accurate understanding of face recognition technology and the underlying causes for the observed bias. In this dissertation, we perform a detailed examination of face recognition bias. First, we explore the problem by looking into differences in face recognition accuracy across race, where our experiments show a mixed set of results for African-American and Caucasians, but African-Americans usually have higher false match rates (FMR) and lower false non-match rates (FNMR). We investigate the speculation that darker skin tones somehow causes higher FMR, but our experiments show no evidence to support this speculation. Second, we investigate face recognition accuracy across age groups. Varying face recognition accuracy across age ranges receives almost no attention compared to varying accuracy across race or gender. Also, it is the most difficult demographic aspect to investigate, since datasets with correct meta-data and enough data across middle and older age ranges are required for investigation. Using the best available dataset, we present an extensive analysis of face recognition accuracy across age groups, where our results show that there is a drop in performance when subjects are older. Third, we investigate face recognition bias across gender, which is the main focus of this dissertation. Our results show consensus higher FMR and higher FNMR for women ('gender-gap'), agreeing with previous works. We investigate various speculated causes for this gender gap, and find face visibility to be the main cause for females' higher FNMR. We speculate that differences in face shape similarity are a main cause for females' higher FMR. We arrive at this speculation for the FMR because other possible causes that were investigated did not result in a major difference in FMR. We also show results of extensive experiments on the correlation of gender balance in training datasets and accuracy on test sets. Using face segmentation methods, we show that face regions have different effects across demographics, suggesting that matchers should weight face regions differently. Moreover, motivated by the apparent media confusion between facial analytics (such as ``gender from face'') and face recognition, we present an evaluation of the correlation of gender classification errors and face recognition errors. Finally, we investigate the effect that the training margin during network training has on gender bias. We train models with different margins for each gender, and analyze the effect that this has on training and testing accuracy. When margins are the same for both genders, we show that the gender-gap is present in both training and test datasets. However, when margins are different during training, with females given a larger margin than males, the gender-gap in test accuracy is reduced.