Surveying Racial Bias in Facial Recognition: Balancing Datasets and Algorithmic Enhancements
Andrew Sumsion, Shad Torrie, Dah-Jye Lee, Zheng Sun
Abstract
Facial recognition systems frequently exhibit high accuracies when evaluated on standard test datasets. However, their performance tends to degrade significantly when confronted with more challenging tests, particularly involving specific racial categories. To measure this inconsistency, many have created racially aware datasets to evaluate facial recognition algorithms. This paper analyzes facial recognition datasets, categorizing them as racially balanced or unbalanced while limiting racially balanced datasets to have each race be represented within five percentage points of all other represented races. We investigate methods to address concerns about racial bias due to uneven datasets by using generative adversarial networks and latent diffusion models to balance the data, and we also assess the impact of these techniques. In an effort to mitigate accuracy discrepancies across different racial groups, we investigate a range of network enhancements in facial recognition performance across human races. These improvements encompass architectural improvements, loss functions, training methods, data modifications, and incorporating additional data. Additionally, we discuss the interrelation of racial and gender bias. Lastly, we outline avenues for future research in this domain.