Face Perception : Man Machine Comparison Studies
The University of Texas at Dallas
O'Toole, A.J., Phillips, P.J., Jiang, F., Ayyad, J., Penard, and Abdi, H. (accepted with minor revisions) IEEE: Transactions on Pattern Ananlysis and Machine Intelligence.
There has been significant progress in improving the performance of computer-based face recognition algorithms over the last decade. Although algorithms have been tested and compared extensively with each other, there has been virtually no work comparing the accuracy of computer-based face recognition systems with humans. We compared seven state-of-the-art face recognition algorithms with humans on a face-matching task. Humans and algorithms determined whether pairs of face images, taken under different illumination conditions, were pictures of the same person or of different people. Three algorithms surpassed human performance matching face pairs prescreened to be
O'Toole, A.J., Abdi, H., Jiang, F. and, Phillips, P.J. (accepted with minor revisions) IEEE: Transactions on Systems, Man \& Cybernetics
It has been demonstrated recently that state-of-the-art face recognition algorithms can surpass human accuracy at matching faces over changes in illumination. The ranking of algorithms and humans by accuracy, however, does not provide information about whether algorithms and humans perform the task comsparably or whether algorithms and humans can be fused to improve performance. Here, we fused humans and algorithms using partial least squares (PLS) regression. In the first experiment, we applied PLS to face-pair similarity scores generated by seven algorithms participating in the Face Recognition Grand Challenge (FRGC). PLS produced an optimal weighting of the similarity scores, which we tested for generality with a jackknife procedure. Fusing algorithms¹ similarity scores using the optimal weights produced a two-fold reduction of error rate over the most accurate algorithm. Next, human subject-generated similarity scores were added to the PLS analysis. Fusing humans and algorithms increased performance to near-perfect classification accuracy. These results are discussed in terms of maximizing face recognition accuracy with hybrid systems consisting of multiple algorithms and humans.
O'Toole, A.J. Jiang, F., Abdi, H. and, Phillips, P.J. (in press) Proceedings of the International Symposium on Visual Computing
Recent work indicates that state-of-the-art face recognition algorithms can surpass humans matching identity in pairs of face images taken under different illumination conditions. It has been demonstrated further that fusing algorithm- and human-derived face similarity estimates cuts error rates substantially over the performance of the best algorithms. Here we employed a pattern-based classification procedure to fuse individual human subjects and algorithms with the goal of determining whether strategy differences among humans are strong enough to suggest particular man-machine combinations. The results showed that error rates for the pairwise man-machine fusions were reduced an average of 47 percent when compared to the performance of the algorithms individually. The performance of the best pairwise combinations of individual humans and algorithms was only slightly less accurate than the combination of individual humans with all seven algorithms. The balance of man and machine contributions to the pairwise fusions varied widely, indicating that a one-size-fits-all weighting of human and machine face recognition estimates is not appropriate.
Work supported by TSWG funding to A. O'Toole and H. Abdi.